[Bug 254366] Severe IPv6 TCP transfer issues on 13.0-RC2 with virtio network adapter

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254366

            Bug ID: 254366
           Summary: Severe IPv6 TCP transfer issues on 13.0-RC2 with
                    virtio network adapter
           Product: Base System
           Version: 13.0-STABLE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: michael at esosoft.com

Created attachment 223365
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=223365&action=edit
default virtio iperf3 speed ipv6 FreeBSD 13.0-RC2

I installed FreeBSD 13.0-RC2 on some of my database virtual machines using the
VirtIO network adapter. The VMs only use IPv6. After the upgrade MySQL threads
kept filling up with data waiting to be sent. The same setup worked flawlessly
with 12.2. Upon investigation I noticed that the IPv6 transfer speed from the
VM to any other device is in the range of 46Kbps. Usually it is 15-20Gbps from
VM to VM on the same bare metal machine. If IPv4 is used no issues at all on
FreeBSD 13.0-RC2.

This issue was also recently discussed on the freebsd-net list under the
"severe ipv6 tcp transfer issues" subject.

The problem can be easily reproduced by setting up two virtual machines running
the default FreeBSD 13.0-RC2 release candidate. (Note the problem already
exists in BETA4). Configure the network device using virtio and setup IPv6. Run
a simple iperf3 session from VM to VM on IPv6 and the slowness shows up every
time in both directions. I tested it with virt-manager as well as using Proxmox
and both show that issue.

A quick fix was to issue ifconfig vtnet0 -tso6. That brings transfer speeds
back to acceptable levels in the Gbps range. However it is still much slower
for vm-vm-same-host than IPv4. About 4Gbps on IPv6 vs 15-20Gbps IPv4. FreeBSD
12.2 has the same 15-20Gbps speed IPv4/IPv6.

My thoughts were that the recent virtio changes committed by Bryan Venteicher
were the culprit. I started bisecting the changes submitted 2021-01-19.

After building a few kernels it is the following commit causing the issue:
475a60aec7e if_vtnet: Misc Tx path cleanup

review D27926

Building a kernel without that commit brings back IPv4/IPv6 on the same level
as before. Not much of kernel geek so I'm not sure what part of the code is
causing the problem to show up in IPv6 only.

Attached are a couple iperf3 & ifconfig commands to show the slowness. If
anything else is needed, please let me know, but all you need is stock FreeBSD
13.0-RC2 VM using virtio and ipv6.

-- 
You are receiving this mail because:
You are the assignee for the bug.