Re: realpath() leakage and unix socket collision in -o union mounts + jails

On Tue, Oct 07, 2025 at 08:49:27PM +0200, David 'equinox' Lamparter wrote:
> Hi all,
> 
> 
> I'm working on a network testing system that uses jails to emulate
> virtual routers.  The test target is == the host here, so I'm using
> union mounts to create a "clone" of the host.  That looks like this:
> 
> [root@freebsd15test]: /home/equinox/topotato # uname -a
> FreeBSD r1 15.0-ALPHA4 FreeBSD 15.0-ALPHA4 stable/15-n280334-d2b670b27f37 GENERIC amd64
> [root@freebsd15test]: /home/equinox/topotato # mount | grep ujbfygu3
> / on /tmp/topo_ujbfygu3/root (nullfs, local)
> /tmp/topo_ujbfygu3/___ on /tmp/topo_ujbfygu3/root (nullfs, local, union)
> /home on /tmp/topo_ujbfygu3/root/home (nullfs, local)
> /tmp/topo_ujbfygu3/___home on /tmp/topo_ujbfygu3/root/home (nullfs, local, union)
> /tmp on /tmp/topo_ujbfygu3/root/tmp (nullfs, local)
> /dev on /tmp/topo_ujbfygu3/root/dev (nullfs, local)
> 
> (The last 2 mounts are intentionally not union.  ALPHA5 VM images are
> currently broken for unrelated reasons, hence me being on ALPHA4.)
> 
> With a jail then created in /tmp/topo_ujbfygu3/root, there are 2 issues.
> First, realpath() leaks the full path outside the Jail:
> 
> access("/usr/local/share/yang",X_OK|R_OK)	 = 0 (0x0)
> open("/usr/local/lib/libyang/types",O_RDONLY|O_DIRECTORY|O_CLOEXEC,011) ERR#2 'No such file or directory'
> open("/usr/local/lib/libyang/extensions",O_RDONLY|O_DIRECTORY|O_CLOEXEC,05343024270) ERR#2 'No such file or directory'
> __realpathat(AT_FDCWD,"/usr/local/share/yang","/tmp/topo_3vmk0s5z/root/usr/local/share/yang",1024,0) = 0 (0x0)
> access("/tmp/topo_3vmk0s5z/root/usr/local/share/yang",X_OK|R_OK) ERR#2 'No such file or directory'
> clock_gettime(0,{ 1759858363.616407491 })	 = 0 (0x0)
> writev(2,[{"2025/10/07 17:32:43 ",20},{"MGMTD: ",7},{"libyang: Unable to fully access search directory "/tmp/topo_3vmk0s5z/root/usr/local/share/yang" (No such file or directory).\n",125}],3) = 152 (0x98)
> 
> This can be reproduced invoking the realpath binary directly:
> 
> [root@freebsd15test]: /home/equinox/topotato # jexec 18 /bin/sh
> # mount
> /tmp/topo_ujbfygu3/___ on / (nullfs, local, union)
> # realpath /etc
> /tmp/topo_ujbfygu3/root/etc
> # realpath /
> /
What are you trying to achieve by the MNT_UNION flag?
Or rather, do not use union mounts as root for jails.

> 
> (Interestingly, it does not happen for /.)
> 
> The second issue is quite a bit worse: AF_UNIX sockets "break through"
> between jails.  This can be reproduced with socat:
> 
> [root@freebsd15test]: /home/equinox/topotato # jls
>    JID  IP Address      Hostname                      Path
>     16                  switch-ns                     /tmp/topo_u3crsgjg/root
>     17                  r1                            /tmp/topo_5x1s_smf/root
>     18                  r2                            /tmp/topo_ujbfygu3/root
> [root@freebsd15test]: /home/equinox/topotato # jexec 18 /bin/sh
> # mount
> /tmp/topo_ujbfygu3/___ on / (nullfs, local, union)
> # hostname
> r2
> # socat UNIX-LISTEN:/var/socktest STDIO
> asdf
> # #(above asdf is output from below)
> 
> [root@freebsd15test]: /home/equinox/frr # jexec 17 /bin/sh
> # mount
> /tmp/topo_5x1s_smf/___ on / (nullfs, local, union)
> # hostname
> r1
> # echo asdf | socat UNIX-CONNECT:/var/socktest STDIO
> 
> Neither of these two issues happens with unionfs (which I've now fallen
> back to using, but AIUI that's entirely unmaintained and unsupported...)
> Could someone investigate these?

There is nothing to investigate, unix sockets are bypassed for nullfs,
this is feature.