Bug in FreeBSD VirtIO network driver (only with pf enabled)

[John Nielsen]

The virtio maintainer is bryanv (CC'ed).

I have KVM+CentOS hosts available at $WORK so let me know if anyone needs help reproducing this or testing fixes.

On Aug 27, 2014, at 1:02 PM, Brian Rak <brak at gameservers.com> wrote:

> I have a FreeBSD 10 x64 guest installed inside a KVM instance on Linux.  When pf is active, and the server is sending data it causes the Linux host to report warnings related to GSO.
> 
> I've talked to some Linux developers, and they believe it to be a bug inside the FreeBSD VirtIO drivers.  Based on what I'm seeing, I'm inclined to agree with them.
> 
> Reproduction is mildly annoying, you need:
> 1) A KVM guest setup with bridged networking, with FreeBSD running side using the VirtIO network interface.  (We tested with CentOS, but the exact distribution should not matter)
> 2) pf enabled (I'm using a single rule: "scrub in all")
> 3) Some method of sending a bit of traffic from the guest (I use netcat)
> 
> So, when I do this:
> 
> # service pf start
> # cat /root/test | nc vultr.com 80
> 
> The Linux kernel on the host will report:
> 
> kernel: WARNING: CPU: 7 PID: 7772 at net/core/dev.c:2246 skb_warn_bad_offload+0xc3/0xd0()
> kernel: igb: caps=(0x0000000640114bb3, 0x0000000000000000) len=1498 data_len=0 gso_size=1380 gso_type=5 ip_summed=0
> 
> If I do:
> 
> # service pf stop
> # cat /root/test | nc vultr.com 80
> 
> No such warning is reported.  I can only reproduce this with pf enabled.  The contents of the /root/test don't matter, I'm using 4k of data from /dev/urandom.  The test file just needs to be bigger then the MTU of the host's network interface.
> 
> I was able to track this down to the virtio_net_hdr being sent by the FreeBSD guest.  With pf enabled, this outbound traffic has the following header:
> 
> flags =  0
> gso_type = VIRTIO_NET_HDR_GSO_TCPV4
> hdr_len = 66
> gso_size =  1440
> csum_start = 0
> csum_offset = 0
> 
> But, this is not a valid configuration.  With VIRTIO_NET_HDR_GSO_TCPV4 enabled, you should also be setting VIRTIO_NET_HDR_F_NEEDS_CSUM and populating the csum_start and csum_offset fields.