sparc64/93530: Incorrect checksums when using pf's route-to on sparc64

[Pieter de Boer]

The following reply was made to PR sparc64/93530; it has been noted by GNATS.

From: Pieter de Boer <pieter at thedarkside.nl>
To: bug-followup at freebsd.org,
 pieter at thedarkside.nl
Cc:  
Subject: Re: sparc64/93530: Incorrect checksums when using pf's route-to on sparc64
Date: Sun, 26 Feb 2006 15:48:18 +0100

 I've investigated some more and found an interesting heisenbug.
 
 =46rom pf.c, pf_route():
 
                 ip->ip_sum =3D 0;
                 if (sw_csum & CSUM_DELAY_IP) {
                         /* From KAME */
                         if (ip->ip_v =3D=3D IPVERSION &&
                             (ip->ip_hl << 2) =3D=3D sizeof(*ip)) {
                                 ip->ip_sum =3D in_cksum_hdr(ip);
                         } else {
                                 ip->ip_sum =3D in_cksum(m0, ip->ip_hl << 2);
                         }
                 }
 
 In the tests I've run, the in_cksum_hdr()-function is called, not the=20
 in_cksum() function. Ok,  I inserted some printf's to find the problem. My=
 =20
 new code looks like this:
 
                 ip->ip_sum =3D 0;
                 if (sw_csum & CSUM_DELAY_IP) {
 //                      printf("pf_route(): B1\n");
                         /* From KAME */
                         if (ip->ip_v =3D=3D IPVERSION &&
                             (ip->ip_hl << 2) =3D=3D sizeof(*ip)) {      =20
 //                              printf("pf_route: B2\n");     =20
                                 ip->ip_sum =3D in_cksum_hdr(ip);    =20
                         } else {
                         //      printf("pf_route: B3\n");
                                 ip->ip_sum =3D in_cksum(m0, ip->ip_hl << 2);
                         }
                 }
 With the printf B1 and B2 commented out, the checksums are wrong. With eith=
 er=20
 printf B1 or B2 not commented out, then the checksums are correct. My theor=
 y=20
 is that there's some caching issue between the ip->ip_sum =3D 0; at the top=
  and=20
 the assembly-code of in_cksum_hdr(). When a printf is inserted between the=
 =20
 ip->ip_sum =3D 0; and the in_cksum_hdr(), the cache is invalidated long bef=
 ore=20
 in_cksum_hdr() is called.=20
 
 Perhaps a Sparc64-hacker could take a look at the assembly output of=20
 pf_route() and determine whether this could be the case? :)