Re: VirtIO Net Tuning
- In reply to: Michael Tuexen : "Re: VirtIO Net Tuning"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Wed, 25 Mar 2026 21:56:39 UTC
> On Mar 25, 2026, at 11:55 AM, Michael Tuexen <tuexen@freebsd.org> wrote: > > >> >> On 25. Mar 2026, at 16:20, Mark Saad <nonesuch@longcount.org> wrote: >> >> >> >>> On Wed, Mar 25, 2026 at 10:54 AM Michael Tuexen <tuexen@freebsd.org> wrote: >>>> On 25. Mar 2026, at 15:06, Mark Saad <nonesuch@longcount.org> wrote: >>> >>> Hello All >>> I have been playing with FreeBSD 14.3-RELEASE and 15.0-REELASE amd64 qemu powered vms on a Linux cloud hosting setup. I have noticed that with VirtIO net ( vtnet ) interfaces there is some sort of tuning required , or there is some sort of limitation on the packets per second and overall throughput the nics can push. I am not able to get any one interface to push more then 6Gb/s bursts and 5.5Gb/s sustained. Before I go off the deep end with details has anyone run into this , is this a known issue ? >> Hi Mark, >> >> as indicated by Mike, there has been a lot of improvements to vtnet in 14.4 and 15.0 >> such that no special tuning is required. >> If you find that some tuning is necessary to improve things, I would like to know, >> since this is not intended. >> I am not saying that the vtnet driver can't be improved, only that the default >> settings should not need tweaking. >> >> Best regards >> Michael >>> >>> -- >>> mark saad | nonesuch@longcount.org >> >> >> Mike / Michael >> I just tried FreeBSD 14.4 and saw the same results . The simple test is this. >> Two VMS in the same network . Each VM has 14.4-RELEASE from using the IMAGE . >> Each VM has 8 cores 16G ram , one vtnet nic and some disk. >> >> fish 10.240.65.6 sending to bird 10.240.65.7. >> >> root@fish:~ # iperf3 -c 10.240.65.7 -p 8080 >> Connecting to host 10.240.65.7, port 8080 >> [ 5] local 10.240.65.8 port 10106 connected to 10.240.65.7 port 8080 >> [ ID] Interval Transfer Bitrate Retr Cwnd >> [ 5] 0.00-1.06 sec 386 MBytes 3.04 Gbits/sec 0 1.20 MBytes >> [ 5] 1.06-2.04 sec 368 MBytes 3.17 Gbits/sec 0 2.29 MBytes >> [ 5] 2.04-3.05 sec 394 MBytes 3.26 Gbits/sec 0 3.44 MBytes >> [ 5] 3.05-4.01 sec 359 MBytes 3.14 Gbits/sec 0 3.92 MBytes >> [ 5] 4.01-5.04 sec 395 MBytes 3.22 Gbits/sec 0 3.92 MBytes >> [ 5] 5.04-6.04 sec 342 MBytes 2.86 Gbits/sec 0 3.92 MBytes >> [ 5] 6.04-7.06 sec 426 MBytes 3.48 Gbits/sec 0 3.92 MBytes >> [ 5] 7.06-8.02 sec 369 MBytes 3.25 Gbits/sec 0 3.92 MBytes >> [ 5] 8.02-9.04 sec 422 MBytes 3.46 Gbits/sec 0 3.92 MBytes >> [ 5] 9.04-10.02 sec 244 MBytes 2.08 Gbits/sec 0 3.92 MBytes >> - - - - - - - - - - - - - - - - - - - - - - - - - >> [ ID] Interval Transfer Bitrate Retr >> [ 5] 0.00-10.02 sec 3.62 GBytes 3.10 Gbits/sec 0 sender >> [ 5] 0.00-10.02 sec 3.61 GBytes 3.10 Gbits/sec receiver > What is limiting the throughput? Are you CPU limited or bandwidth limited or limited > by something else? > > What tuning gives you a better throughput? > > Best regards > Michael >> >> iperf Done. >> root@fish:~ # >> >> Running 15.0-RELEASE was not much of a difference. >> >> >> -- >> mark saad | nonesuch@longcount.org So this is very strange, when the vms are deployed cascade lake cpu hosts with connect-x 5 nics i am never going beyond 5Gb/s . Switch to a Granite rapids host with connect-x 6 nics the issue goes away. Thats the thing that was bothering me, nothing in the vms indicated an issue outside of some weird driver bug . I am not confident what this is , i am going to see if pcaps show anything out of the ordinary . --- Mark Saad | nonesuch@longcount.org