[Bug 264208] make install[kernel|world] from read only /usr/obj throws permission denied warnings.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=264208

            Bug ID: 264208
           Summary: make install[kernel|world] from read only /usr/obj
                    throws permission denied warnings.
           Product: Base System
           Version: 13.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Keywords: toolchain
          Severity: Affects Some People
          Priority: ---
         Component: bin
          Assignee: bugs@FreeBSD.org
          Reporter: mike@sentex.net

On RELENG_13 I am no longer able to build on one server, and then mount
/usr/src and /usr/obj via nfs thats read-only and installkernel/world without
seeing errors. It *seems* to work in the end, but not sure if something is
being missed.  The server

exports on the server
V4:/
/nfs/usr/src -maproot=root -ro -network 192.168.0.0/16
/nfs/usr/obj -maproot=root -ro -network 192.168.0.0/16

% cat /etc/fstab 
# Custom /etc/fstab for FreeBSD VM images
/dev/gpt/rootfs   /       ufs     rw      1       1
/dev/gpt/swapfs  none    swap    sw      0       0
/dev/gpt/efiesp /boot/efi       msdosfs     rw      2       2
/usr/obj    /nfs/usr/obj    nullfs    rw    0    0 
/usr/src    /nfs/usr/src    nullfs    rw    0    0 

I use nullfs so I dont have to export / since this is ufs

if the export is rw it works as expected without throwing errors. 

On RELENG_12, I was able to mount via NFS /usr/obj and src and do an
installworld on read only mounts. However, with RELENG_13 and above, I get
permission denied errors. The install seems to continue just fine, but I am not
sure if something subtle is being missed. Looking at what gets installed, the
kernel modules get installed just fine.  However, it seems installworld has
some problems with static libs (.a files). They dont seem to get re-installed.
However, not sure if thats by design as doing a quick checksum, there is no
difference on the build server and target of the ~ 700 .a files I checked in
/usr/lib32 and /usr/lib. So despite the time stamps being off, maybe it just
does not bother to update the target if they have not changed?

e.g.

make KERNCONF=vtnet installkernel


make warning: /usr/obj/usr/src: Permission denied.
make[1] warning: /usr/src/: Permission denied.
make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.
--------------------------------------------------------------
>>> Install check kernel
--------------------------------------------------------------
--------------------------------------------------------------
>>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022
--------------------------------------------------------------
cd /usr/obj/usr/src/amd64.amd64/sys/vtnet;  MACHINE_ARCH=amd64 MACHINE=amd64 
CPUTYPE= CC="cc -target x86_64-unknown-freebsd13.1
--sysroot=/usr/obj/usr/src/amd64.amd64/tmp
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX="c++  -target
x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  CPP="cpp -target
x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
-B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  AS="as" AR="ar" ELFCTL="elfctl"
LD="ld"  LLVM_LINK="" NM=nm OBJCOPY="objcopy" RANLIB=ranlib STRINGS= 
SIZE="size" STRIPBIN="strip"
PATH=/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin
make  KERNEL=kernel install
make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denied.
thiskernel=`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ;  if [ !
"`dirname "$thiskernel"`" -ef /boot/kernel ] ; then chflags -R noschg
/boot/kernel ;  rm -rf /boot/kernel ;  rm -rf /usr/lib/debug/boot/kernel ; 
else  if [ -d /boot/kernel.old ] ; then  chflags -R noschg /boot/kernel.old ; 
rm -rf /boot/kernel.old ;  fi ;  mv /boot/kernel /boot/kernel.old ;  if [ -n
"/usr/lib/debug" -a  -d /usr/lib/debug/boot/kernel ]; then  rm -rf
/usr/lib/debug/boot/kernel.old ;  mv /usr/lib/debug/boot/kernel
/usr/lib/debug/boot/kernel.old ;  fi ; sysctl
kern.bootfile=/boot/kernel.old/"`basename "$thiskernel"`" ;  fi
kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel
mkdir -p /boot/kernel
install -p -m 555 -o root -g wheel kernel /boot/kernel/
mkdir -p /usr/lib/debug/boot/kernel
install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kernel/
cd /usr/src/sys/modules;
MAKEOBJDIRPREFIX=/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules
KMODDIR=/boot/kernel MACHINE_CPUARCH=amd64 MACHINE=amd64 MACHINE_ARCH=amd64
MODULES_EXTRA="" WITHOUT_MODULES="" ARCH_FLAGS="" DEBUG_FLAGS="-g" __MPATH=""
KERNBUILDDIR="/usr/obj/usr/src/amd64.amd64/sys/vtnet" SYSDIR="/usr/src/sys"
MODULE_TIED=yes WITH_CTF="1" KCSAN_ENABLED="yes" COMPAT_FREEBSD32_ENABLED="yes"
make  install
make[3] warning:
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules: Permission
denied.
===> aac (install)
make[4] warning:
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac:
Permission denied.
install -T release -o root -g wheel -m 555   aac.ko /boot/kernel/
install -T dbg -o root -g wheel -m 555   aac.ko.debug
/usr/lib/debug/boot/kernel/
===> aacraid (install)
make[4] warning:
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacraid:
Permission denied.
install -T release -o root -g wheel -m 555   aacraid.ko /boot/kernel/
install -T dbg -o root -g wheel -m 555   aacraid.ko.debug
/usr/lib/debug/boot/kernel/
===> accf_data (install)
make[4] warning:
/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_data:
Permission denied.


eg.

0{git}% ls -l /usr/lib32/libhei*
-r--r--r--  1 root  wheel  - 53622 Mar 17 20:02 /usr/lib32/libheimbase.a
lrwxr-xr-x  1 root  wheel  -    17 May 24 17:11 /usr/lib32/libheimbase.so ->
libheimbase.so.11
-r--r--r--  1 root  wheel  - 13224 May 24 17:11 /usr/lib32/libheimbase.so.11
-r--r--r--  1 root  wheel  - 54966 Mar 17 20:02 /usr/lib32/libheimbase_p.a
-r--r--r--  1 root  wheel  - 67598 Mar 17 20:02 /usr/lib32/libheimntlm.a
lrwxr-xr-x  1 root  wheel  -    17 May 24 17:11 /usr/lib32/libheimntlm.so ->
libheimntlm.so.11
-r--r--r--  1 root  wheel  - 23840 May 24 17:11 /usr/lib32/libheimntlm.so.11
-r--r--r--  1 root  wheel  - 68746 Mar 17 20:02 /usr/lib32/libheimntlm_p.a
0{git}%


Looking at the pcap, nothing stands out as to what the problem might be.
Looking at the replies, the ERROR is just "ERROR: No such file or directory"
and doesnt correlate to the permission denied errors

reading from file nfs.pcap, link-type EN10MB (Ethernet)
   2 xid  reply ok
3371 xid  reply ok getattr CHR 11734062556 ids
 951 xid  reply ok getattr CHR 12231260544 ids
 626 xid  reply ok getattr ERROR: No such
   2 xid  reply ok getattr FIFO 10432271160 ids
6758 xid  reply ok getattr FIFO 10731272141 ids
 165 xid  reply ok getattr FIFO 11433667553 ids
 214 xid  reply ok getattr FIFO 12231260544 ids
3373 xid  reply ok getattr LNK 10333067563 ids
   2 xid  reply ok getattr LNK 12231267145 ids
   2 xid  reply ok getattr unk-ft 10 10536061550
   2 xid  reply ok getattr unk-ft 13 10334462541
   2 xid  reply ok getattr unk-ft 15 12231261554
 673 xid  reply ok getattr unk-ft 6 10130661545
 879 xid  reply ok getattr unk-ft 6 11433667553
 126 xid  reply ok getattr unk-ft 8 12231260544
   2 xid  reply ok null

-- 
You are receiving this mail because:
You are the assignee for the bug.