Bug in FreeBSD VirtIO network driver (only with pf enabled)
Brian Rak
brak at gameservers.com
Wed Aug 27 19:07:44 UTC 2014
I have a FreeBSD 10 x64 guest installed inside a KVM instance on Linux.
When pf is active, and the server is sending data it causes the Linux
host to report warnings related to GSO.
I've talked to some Linux developers, and they believe it to be a bug
inside the FreeBSD VirtIO drivers. Based on what I'm seeing, I'm
inclined to agree with them.
Reproduction is mildly annoying, you need:
1) A KVM guest setup with bridged networking, with FreeBSD running side
using the VirtIO network interface. (We tested with CentOS, but the
exact distribution should not matter)
2) pf enabled (I'm using a single rule: "scrub in all")
3) Some method of sending a bit of traffic from the guest (I use netcat)
So, when I do this:
# service pf start
# cat /root/test | nc vultr.com 80
The Linux kernel on the host will report:
kernel: WARNING: CPU: 7 PID: 7772 at net/core/dev.c:2246
skb_warn_bad_offload+0xc3/0xd0()
kernel: igb: caps=(0x0000000640114bb3, 0x0000000000000000) len=1498
data_len=0 gso_size=1380 gso_type=5 ip_summed=0
If I do:
# service pf stop
# cat /root/test | nc vultr.com 80
No such warning is reported. I can only reproduce this with pf
enabled. The contents of the /root/test don't matter, I'm using 4k of
data from /dev/urandom. The test file just needs to be bigger then the
MTU of the host's network interface.
I was able to track this down to the virtio_net_hdr being sent by the
FreeBSD guest. With pf enabled, this outbound traffic has the following
header:
flags = 0
gso_type = VIRTIO_NET_HDR_GSO_TCPV4
hdr_len = 66
gso_size = 1440
csum_start = 0
csum_offset = 0
But, this is not a valid configuration. With VIRTIO_NET_HDR_GSO_TCPV4
enabled, you should also be setting VIRTIO_NET_HDR_F_NEEDS_CSUM and
populating the csum_start and csum_offset fields.
http://www.spinics.net/lists/netdev/msg293976.html gives more detail on
this. I don't fully understand this, so I'd probably mangle the
explanation if I tried to give more detail. I can reproduce this at
will now, but fixing it is beyond my abilities. Is there a better place
to report this? I'm not entirely sure who is responsible for
maintaining the virtio driver.
More information about the freebsd-hackers
mailing list