Virtio network: poor network with KVM hypervisor (latest Proxmox)

[Gala IT]

Hi Alexey,

Can’t bring in much light, but here are our settings under QEMU-KVM 2.1.0 (qemu-system-x86_64):

kern.ipc.maxsockbuf=4194304
net.inet.tcp.sendbuf_max=4194304
net.inet.tcp.recvbuf_max=4194304
net.inet.tcp.recvspace=262144
net.inet.tcp.sendspace=262144
net.inet.tcp.mssdflt=1460
net.inet.tcp.minmss=1300

net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=524288

We dind’t change any of the eth. settings:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex> (this is autodetected, we’re not enforcing it in any way)
        status: active

We didn’t have as huge of a problem as you do, but we usually didn’t get over 10-15 Mbps both inbound and outbound for a single TCP connection. Now we easily see 50Mbps ratios, which is way more reasonable, although not impressive.

IIRC the biggest change came after changing:

net.inet.tcp.mssdflt=1460
net.inet.tcp.minmss=1300

net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=524288

We’re on 9.3-RELEASE-p5.

I can’t seem to enable/disable TSO or LRO via ifconfig, neither I can find any sysctl tunables related to this (man vtnet reads that there should be some):

# sysctl hw. | grep vtnet
# (no output)

Hope it helps. Please let us know if you find out anything else, as we’re also interested in getting some more throughput out of those interfaces!

Best,
David

> El 1 març 2016, a les 12:30, Alexey Tarasov <me at lexasoft.ru> va escriure:
> 
> Hi all! 
> 
> I am using the latest Proxmox 4.1 with all updates installed. 
> I have several VM's with FreeBSD guests and 1 VM with Ubuntu 14 (all KVM). 
> Host system file download speed: 60 MBps. 
> FreeBSD guest download speed: 2 MBps on virtio network with TSO enabled, 5-9 MBps with TSO disabled; 12 MBps on e1000 network. 
> Ubuntu guest: 60 MBps with virtio. 
> 
> I've tried the following: 
> 1) Different FreeBSD versions: 9.3, 10.2, 10.3-BETA3. 
> 2) Different TSO settings, enabling/disabling RXCSUM. 
> 3) Different TSO settings on host system. 
> 
> The best results I got described above :( 
> 
> Does anyone have any ideas how to get full network performance inside FreeBSD guests? 
> 
> --
> Alexey Tarasov
> 
> (\__/) 
> (='.'=) 
> E[: | | | | :]З 
> (")_(")
>