Fusefs unionfs trouble with hardlinks?

From: <ltning-freebsd-stable_at_anduin.net>
Date: Thu, 21 Sep 2023 18:08:52 UTC

I've been trying to prepare a NomadBSD stick for a raid at a nearby 
laptop store, and have upgraded it to 14-BETA2 using freebsd-update. 
However, when attempting to pkg upgrade it afterwards, I find that some 
ports cause problems; one example being libdeflate.

NomadBSD has a uzip-backed /usr/local with a ZFS-backed fusefs-unionfs 
on top. Hence, installed/upgraded packages and other changes are applied 
to the ZFS dataset only, since the uzip device/filesystem is read-only.

During package upgrade or installation, most packages behave fine, but 
e.g. libdeflate attempts to create hardlinks to itself within 
/usr/local/bin. This fails, with a "File exists"-complaint.

I have not had any similar problems on NomadBSD installations I have 
elsewhere, but they are all on 13.2 - this leads me to believe there may 
be problems specific to 14.

I'm using the standard "latest" FreeBSD package repo. And for clarity, I 
have *not* installed to another disk; this is all done directly on the 
USB stick created from the NomadBSD image.

I can extract more logs/info as required, if anyone feels like looking 
into this. It does seem like a currently-working use case that no longer 
works, so perhaps of interest before 14 ships?