[Bug 263229] virtio vtnet driver with rxcsum errors

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=263229

            Bug ID: 263229
           Summary: virtio vtnet driver with rxcsum errors
           Product: Base System
           Version: 13.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: mike@sentex.net

I was setting up a VM pf firewall and noticed I was not able to nat out for
some reason. Looking at the pcap, it seems when the vm is in forwarding mode, I
get tcp checksum errors. If I do a

ifconfig vtnet1 -rxcsum

ifconfig vtnet0 -rxcsum

nat then seems to work fine

The setup is a simple VM with the hypervisor libvirt/KVM ubuntu 20 LTS.  Guest
is RELENG_13 from Apr 11/2022



e.g. vtnet1 is facing another internal VM. If I try and connect to the internal
NIC vtnet1 (192.168.199.7) I dont get any errors. But if I try and get out via
nat, it fails. If I switch both nics to Intel em NICs, it works without
modification
public_internet <---> vtnet0-vtnet1 <---> another guest vm

vtnet0 = public IP
vtnet1 = 192.168.199.7/24
guest = 192.168.199.100


 tcpdump -s0 -vnei vtnet1 port 443
tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144
bytes
18:35:05.937364 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 74: (tos 0x10, ttl 64, id 64547, offset 0, flags [DF], proto TCP (6),
length 60)
    192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect
-> 0x4755), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val
1606656477 ecr 0,nop,wscale 7], length 0
18:35:06.939305 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 74: (tos 0x10, ttl 64, id 64548, offset 0, flags [DF], proto TCP (6),
length 60)
    192.168.199.100.56996 > 172.217.1.4.443: Flags [S], cksum 0x3619 (incorrect
-> 0x436b), seq 2994289493, win 64240, options [mss 1460,sackOK,TS val
1606657479 ecr 0,nop,wscale 7], length 0
18:35:21.040936 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 74: (tos 0x10, ttl 64, id 45298, offset 0, flags [DF], proto TCP (6),
length 60)
    192.168.199.100.59876 > 192.168.199.7.443: Flags [S], cksum 0x0fec
(incorrect -> 0xf8f6), seq 1122263205, win 64240, options [mss 1460,sackOK,TS
val 2124691085 ecr 0,nop,wscale 7], length 0
18:35:21.040993 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length
60)
    192.168.199.7.443 > 192.168.199.100.59876: Flags [S.], cksum 0x0fec
(incorrect -> 0x5be5), seq 3948792593, ack 1122263206, win 65535, options [mss
1460,nop,wscale 6,sackOK,TS val 2270714410 ecr 2124691085], length 0
18:35:21.041224 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 45299, offset 0, flags [DF], proto TCP (6),
length 52)
    192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x0fe4
(incorrect -> 0x88b9), ack 1, win 502, options [nop,nop,TS val 2124691086 ecr
2270714410], length 0
18:35:22.843871 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 45300, offset 0, flags [DF], proto TCP (6),
length 52)
    192.168.199.100.59876 > 192.168.199.7.443: Flags [F.], cksum 0x0fe4
(incorrect -> 0x81ae), seq 1, ack 1, win 502, options [nop,nop,TS val
2124692888 ecr 2270714410], length 0
18:35:22.843910 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800),
length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length
52)
    192.168.199.7.443 > 192.168.199.100.59876: Flags [.], cksum 0x0fe4
(incorrect -> 0x7895), ack 2, win 1027, options [nop,nop,TS val 2270716214 ecr
2124692888], length 0
18:35:22.843980 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800),
length 66: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length
52)
    192.168.199.7.443 > 192.168.199.100.59876: Flags [F.], cksum 0x0fe4
(incorrect -> 0x7894), seq 1, ack 2, win 1027, options [nop,nop,TS val
2270716214 ecr 2124692888], length 0
18:35:22.844159 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length
52)
    192.168.199.100.59876 > 192.168.199.7.443: Flags [.], cksum 0x7aa0
(correct), ack 2, win 502, options [nop,nop,TS val 2124692889 ecr 2270716214],
length 0


e.g. after disabling rxcsum (ifconfig vtnet1 -rxcsum ifconfig vtnet0 -rxcsum)

tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144
bytes
18:39:57.425940 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 74: (tos 0x10, ttl 64, id 53398, offset 0, flags [DF], proto TCP (6),
length 60)
    192.168.199.101.55810 > 142.251.32.68.443: Flags [S], cksum 0x5467
(correct), seq 2136333436, win 64240, options [mss 1460,sackOK,TS val
2746738194 ecr 0,nop,wscale 7], length 0
18:39:57.432020 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800),
length 74: (tos 0x0, ttl 123, id 58394, offset 0, flags [none], proto TCP (6),
length 60)
    142.251.32.68.443 > 192.168.199.101.55810: Flags [S.], cksum 0xa929
(correct), seq 292800283, ack 2136333437, win 65535, options [mss
1430,sackOK,TS val 460829236 ecr 2746738194,nop,wscale 8], length 0
18:39:57.432265 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 53399, offset 0, flags [DF], proto TCP (6),
length 52)
    192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xd5dc
(correct), ack 1, win 502, options [nop,nop,TS val 2746738200 ecr 460829236],
length 0
18:39:58.605990 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 53400, offset 0, flags [DF], proto TCP (6),
length 52)
    192.168.199.101.55810 > 142.251.32.68.443: Flags [F.], cksum 0xd145
(correct), seq 1, ack 1, win 502, options [nop,nop,TS val 2746739374 ecr
460829236], length 0
18:39:58.612128 52:54:00:09:6e:82 > 52:54:00:b6:ae:7b, ethertype IPv4 (0x0800),
length 66: (tos 0x0, ttl 123, id 58984, offset 0, flags [none], proto TCP (6),
length 52)
    142.251.32.68.443 > 192.168.199.101.55810: Flags [F.], cksum 0xcd9e
(correct), seq 1, ack 2, win 256, options [nop,nop,TS val 460830416 ecr
2746739374], length 0
18:39:58.612402 52:54:00:b6:ae:7b > 52:54:00:09:6e:82, ethertype IPv4 (0x0800),
length 66: (tos 0x10, ttl 64, id 53401, offset 0, flags [DF], proto TCP (6),
length 52)
    192.168.199.101.55810 > 142.251.32.68.443: Flags [.], cksum 0xcca2
(correct), ack 2, win 502, options [nop,nop,TS val 2746739380 ecr 460830416],
length 0


I am guessing txcsum is fine as I can connect out from the FreeBSD VM, so
packets originating on the firewall's vtnet0 interface is ok.

-- 
You are receiving this mail because:
You are the assignee for the bug.