[Bug 240106] VNET issue with ARP and routing sockets in jails

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240106

            Bug ID: 240106
           Summary: VNET issue with ARP and routing sockets in jails
           Product: Base System
           Version: 12.0-RELEASE
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs at FreeBSD.org
          Reporter: john.westbrook at gmail.com

I'm experiencing an intermittent connectivity issue running FreeBSD 12.0 with
jail using VNET, which appears to be related to lost ARP replies.

There are several discussion threads on forums that appear related:

https://forums.freebsd.org/threads/vnet-arp-replies-are-lost.71082
https://www.ixsystems.com/community/threads/arp-replies-loss-in-vnet.77027
https://www.ixsystems.com/community/threads/jails-eero.59477

One insightful comment from the first thread:

"""On step #2 the reply is mistakenly padded with 14 bytes which is exactly the
number of bytes beyond the 18 bytes in the request (the request was padded with
32 bytes). I bet this is part of the bug. By looking at FreeBSD ARP reply code
it actually creates the reply by editing the request bytes in place. For some
reason it removes only 18 bytes from the request padding. However, this happens
only on VNET interface as noted above."""

I was able to see ARP traffic using tcpdump, but (arp -a) doesn't contain
updated ARP entries. Also, in an affected jail, I can't add static arp entries:

# arp -s 10.0.0.1 XX:XX:XX:XX:XX:XX
arp: writing to routing socket: Cannot allocate memory

whereas, in an unaffected jail the arp command succeeds. Jails are should have
access to routing sockets by default, so perhaps the problem is related to
accessing routing sockets in VNET jails?

The test setup where I'm observing this is using an SR-IOV VF (Chelsio cxlv0)
passed into the jail (via vnet.interface in jail.conf). The test setup has two
jails each on two direct attached hosts. I observe the problem on both hosts,
but it comes and goes with reboots.

-- 
You are receiving this mail because:
You are the assignee for the bug.