misc/165059: virtio_kmod: networking breaks with a router using virtio net driver on KVM host

[David Talkington]

>Number:         165059
>Category:       misc
>Synopsis:       virtio_kmod: networking breaks with a router using virtio net driver on KVM host
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Sun Feb 12 21:20:11 UTC 2012
>Closed-Date:
>Last-Modified:
>Originator:     David Talkington
>Release:        9.0-RELEASE
>Organization:
>Environment:
FreeBSD atlantis02.flyingjoke.org 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:46:30 UTC 2012     root at farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
>Description:
When the router for a FreeBSD guest on KVM is also a FreeBSD guest on the same KVM host, and which is using the virtio network driver from virtio_kmod, ping will work between guests on different subnets, but no userland network daemons will respond. If I switch to the e1000 driver on the router, but change nothing else, everything works correctly.


>How-To-Repeat:
I created three FreeBSD guests on one Linux KVM host. I am using bridged networking on the KVM host, as br0 and br1. One of the guests has two network interfaces and acts as a router between two subnets, as follows:

router1: br0, 192.168.1.1; br1, 192.168.2.1
client1: br0, 192.168.1.100; default route 192.168.1.1
client2: br1, 192.168.2.100; default route 192.168.2.1

I configured virtio network interfaces on all three hosts. I enabled forwarding on router1, but no packet filtering. No NAT is in use.

Result:

    * client1 can ping client2, and vice versa.
    * ssh works from router1 to client1 and vice versa, and from router1 to client2 and vice versa.
    * ssh from client1 to client2 will fail (and vice versa); the client simply hangs indefinitely while trying to connect. 
    * tcpdump on client2 will show that the SYN is arriving at client2 port 22, but client2 never replies, nor generates any debug or log output that suggests it ever saw the connection attempt.
    * any other userland network service I try (both tcp and udp) will show the same thing -- packets arrive at client2 from client1, but the daemon seems to never see them. Since ping works, I know the kernel is getting them.
    * If I switch back to the e1000 driver on router1, but make no other changes, and make no changes at all to client1 and client2, then ssh will work properly from client1 to client2 and the problem is resolved.
    * If I let router1 continue to use virtio interfaces, but move router1 onto a different KVM host -- so that the traffic from client1 to client2 must leave the KVM host via the bridged interface and then return on a different interface - then ssh will work properly from client1 to client2 and the problem is resolved.

KVM guests: FreeBSD 9
virtio-kmod: 0.228301
KVM host: Ubuntu 11.10
qemu-kvm: 0.14.1
>Fix:
Unknown.

>Release-Note:
>Audit-Trail:
>Unformatted: