kern/175954: XENHVM xn network driver extreme packet loss during a slow download

[Paul]

>Number:         175954
>Category:       kern
>Synopsis:       XENHVM xn network driver extreme packet loss during a slow download
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Feb 08 10:50:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator:     Paul
>Release:        9.1
>Organization:
>Environment:
FreeBSD local.server 9.1-RELEASE FreeBSD 9.1-RELEASE #0: Fri Feb  8 01:12:53 UTC 2013     root at local.server:/usr/obj/usr/src/sys/XENHVM  amd64
>Description:
The xn0 PV network driver for freebsd has a bug where if the client is downloading slowly leading to really poor network performance.

I am guessing httpd is filling up the network buffer space with the large file, but the client is not downloading it fast enough.

Increasing the various network send/recv/mbuf buffers with sysctl has no effect.

#ping running from inside the freebsd guest ping the IP of xen host dom0
64 bytes from 10.0.0.1: icmp_seq=1303 ttl=64 time=234.603 ms
64 bytes from 10.0.0.1: icmp_seq=1304 ttl=64 time=204.760 ms
64 bytes from 10.0.0.1: icmp_seq=1305 ttl=64 time=261.683 ms
64 bytes from 10.0.0.1: icmp_seq=1306 ttl=64 time=47.769 ms
ping: sendto: No buffer space available
64 bytes from 10.0.0.1: icmp_seq=1308 ttl=64 time=436.514 ms
64 bytes from 10.0.0.1: icmp_seq=1309 ttl=64 time=72.970 ms
64 bytes from 10.0.0.1: icmp_seq=1310 ttl=64 time=82.359 ms
64 bytes from 10.0.0.1: icmp_seq=1311 ttl=64 time=272.854 ms
64 bytes from 10.0.0.1: icmp_seq=1312 ttl=64 time=328.472 ms
64 bytes from 10.0.0.1: icmp_seq=1313 ttl=64 time=122.783 ms
ping: sendto: No buffer space available
64 bytes from 10.0.0.1: icmp_seq=1315 ttl=64 time=84.061 ms
64 bytes from 10.0.0.1: icmp_seq=1316 ttl=64 time=70.805 ms
^C
--- 10.0.0.1 ping statistics ---
1317 packets transmitted, 1173 packets received, 10.9% packet loss
round-trip min/avg/max/stddev = 18.828/200.423/659.228/117.169 ms

>How-To-Repeat:
On the freebsd guest, run a http server such as apache or nginx and serve a large file, mine was 300mb.

On the xen host (dom0) run this command (10.0.0.2 is the IP of the freebsd guest)
wget -O /dev/null --limit-rate=1000K http://10.0.0.2/300mb.bin
>Fix:
The problem does not occur while using the GENERIC kernel.
It also does not occur if the wget command is run from a different VM, but thats preventing me from running a reverse proxy on the dom0.

>Release-Note:
>Audit-Trail:
>Unformatted: