Bug in FreeBSD VirtIO network driver (only with pf enabled)

[Bryan Venteicher]

On Wed, Aug 27, 2014 at 2:02 PM, Brian Rak <brak at gameservers.com> wrote:

> I have a FreeBSD 10 x64 guest installed inside a KVM instance on Linux.
> When pf is active, and the server is sending data it causes the Linux host
> to report warnings related to GSO.
>
> I've talked to some Linux developers, and they believe it to be a bug
> inside the FreeBSD VirtIO drivers.  Based on what I'm seeing, I'm inclined
> to agree with them.
>
> Reproduction is mildly annoying, you need:
> 1) A KVM guest setup with bridged networking, with FreeBSD running side
> using the VirtIO network interface.  (We tested with CentOS, but the exact
> distribution should not matter)
> 2) pf enabled (I'm using a single rule: "scrub in all")
> 3) Some method of sending a bit of traffic from the guest (I use netcat)
>
> So, when I do this:
>
> # service pf start
> # cat /root/test | nc vultr.com 80
>
> The Linux kernel on the host will report:
>
> kernel: WARNING: CPU: 7 PID: 7772 at net/core/dev.c:2246
> skb_warn_bad_offload+0xc3/0xd0()
> kernel: igb: caps=(0x0000000640114bb3, 0x0000000000000000) len=1498
> data_len=0 gso_size=1380 gso_type=5 ip_summed=0
>
> If I do:
>
> # service pf stop
> # cat /root/test | nc vultr.com 80
>
> No such warning is reported.  I can only reproduce this with pf enabled.
> The contents of the /root/test don't matter, I'm using 4k of data from
> /dev/urandom.  The test file just needs to be bigger then the MTU of the
> host's network interface.
>
> I was able to track this down to the virtio_net_hdr being sent by the
> FreeBSD guest.  With pf enabled, this outbound traffic has the following
> header:
>
> flags =  0
> gso_type = VIRTIO_NET_HDR_GSO_TCPV4
> hdr_len = 66
> gso_size =  1440
> csum_start = 0
> csum_offset = 0
>
> But, this is not a valid configuration.  With VIRTIO_NET_HDR_GSO_TCPV4
> enabled, you should also be setting VIRTIO_NET_HDR_F_NEEDS_CSUM and
> populating the csum_start and csum_offset fields.
>
>

​It would appear that somewhere in pf the CSUM_TCP flag is getting clear
for CSUM_TSO packets. I don't believe the stack otherwise sets CSUM_TSO
without CSUM_TCP. Now, it is probably safe to assume CSUM_TCP when CSUM_TSO
is set: the mbuf's m_pkthdr.csum_data better be valid in such cases though.



> http://www.spinics.net/lists/netdev/msg293976.html gives more detail on
> this.  I don't fully understand this, so I'd probably mangle the
> explanation if I tried to give more detail.  I can reproduce this at will
> now, but fixing it is beyond my abilities.  Is there a better place to
> report this?  I'm not entirely sure who is responsible for maintaining the
> virtio driver.
> _______________________________________________
> freebsd-hackers at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe at freebsd.org"
>