Re: make install[kernel|world] from read only /usr/obj possible ?

From: Warner Losh <imp_at_bsdimp.com>
Date: Tue, 24 May 2022 19:04:06 UTC
Sorry for the top post...

It should work. It has worked in the past. This is a regression.

Warner

On Tue, May 24, 2022, 12:59 PM mike tancsa <mike@sentex.net> wrote:

> On RELENG_12, I was able to mount via NFS /usr/obj and src and do an
> installworld on read only mounts. However, with RELENG_13 and above, I
> get permission denied errors. The install seems to continue just fine,
> but I am not sure if something subtle is being missed. Looking at what
> gets installed, the kernel modules get installed just fine.  However, it
> seems installworld has some problems with static libs (.a files). They
> dont seem to get re-installed. However, not sure if thats by design as
> doing a quick checksum, there is no difference on the build server and
> target of the ~ 700 .a files I checked in /usr/lib32 and /usr/lib. So
> despite the time stamps being off, maybe it just does not bother to
> update the target if they have not changed?
>
> e.g.
>
> make KERNCONF=vtnet installkernel
>
>
> make warning: /usr/obj/usr/src: Permission denied.
> make[1] warning: /usr/src/: Permission denied.
> make[2] warning: /usr/obj/usr/src/amd64.amd64: Permission denied.
> --------------------------------------------------------------
>  >>> Install check kernel
> --------------------------------------------------------------
> --------------------------------------------------------------
>  >>> Installing kernel vtnet on Tue May 24 13:50:52 EDT 2022
> --------------------------------------------------------------
> cd /usr/obj/usr/src/amd64.amd64/sys/vtnet;  MACHINE_ARCH=amd64
> MACHINE=amd64  CPUTYPE= CC="cc -target x86_64-unknown-freebsd13.1
> --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin" CXX="c++  -target
> x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  CPP="cpp -target
> x86_64-unknown-freebsd13.1 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp
> -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin"  AS="as" AR="ar"
> ELFCTL="elfctl" LD="ld"  LLVM_LINK="" NM=nm OBJCOPY="objcopy"
> RANLIB=ranlib STRINGS=  SIZE="size" STRIPBIN="strip"
> PATH=/usr/obj/usr/src/amd64.amd64/tmp/bin:/usr/obj/usr/src/amd64.amd64/tmp/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/sbin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin:/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/libexec::/sbin:/bin:/usr/sbin:/usr/bin
>
> make  KERNEL=kernel install
> make[2] warning: /usr/obj/usr/src/amd64.amd64/sys/vtnet: Permission denied.
> thiskernel=`sysctl -n kern.bootfile || echo /boot/kernel/kernel` ;  if [
> ! "`dirname "$thiskernel"`" -ef /boot/kernel ] ; then chflags -R noschg
> /boot/kernel ;  rm -rf /boot/kernel ;  rm -rf /usr/lib/debug/boot/kernel
> ;  else  if [ -d /boot/kernel.old ] ; then  chflags -R noschg
> /boot/kernel.old ;  rm -rf /boot/kernel.old ;  fi ;  mv /boot/kernel
> /boot/kernel.old ;  if [ -n "/usr/lib/debug" -a  -d
> /usr/lib/debug/boot/kernel ]; then  rm -rf
> /usr/lib/debug/boot/kernel.old ;  mv /usr/lib/debug/boot/kernel
> /usr/lib/debug/boot/kernel.old ;  fi ; sysctl
> kern.bootfile=/boot/kernel.old/"`basename "$thiskernel"`" ;  fi
> kern.bootfile: /boot/kernel/kernel -> /boot/kernel.old/kernel
> mkdir -p /boot/kernel
> install -p -m 555 -o root -g wheel kernel /boot/kernel/
> mkdir -p /usr/lib/debug/boot/kernel
> install -p -m 555 -o root -g wheel kernel.debug /usr/lib/debug/boot/kernel/
> cd /usr/src/sys/modules;
> MAKEOBJDIRPREFIX=/usr/obj/usr/src/amd64.amd64/sys/vtnet/modules
> KMODDIR=/boot/kernel MACHINE_CPUARCH=amd64 MACHINE=amd64
> MACHINE_ARCH=amd64 MODULES_EXTRA="" WITHOUT_MODULES="" ARCH_FLAGS=""
> DEBUG_FLAGS="-g" __MPATH=""
> KERNBUILDDIR="/usr/obj/usr/src/amd64.amd64/sys/vtnet"
> SYSDIR="/usr/src/sys" MODULE_TIED=yes WITH_CTF="1" KCSAN_ENABLED="yes"
> COMPAT_FREEBSD32_ENABLED="yes" make  install
> make[3] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules:
> Permission denied.
> ===> aac (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aac:
> Permission denied.
> install -T release -o root -g wheel -m 555   aac.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555   aac.ko.debug
> /usr/lib/debug/boot/kernel/
> ===> aacraid (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/aacraid:
>
> Permission denied.
> install -T release -o root -g wheel -m 555   aacraid.ko /boot/kernel/
> install -T dbg -o root -g wheel -m 555   aacraid.ko.debug
> /usr/lib/debug/boot/kernel/
> ===> accf_data (install)
> make[4] warning:
> /usr/obj/usr/src/amd64.amd64/sys/vtnet/modules/usr/src/sys/modules/accf_data:
>
> Permission denied.
>
>
> eg.
>
> 0{git}% ls -l /usr/lib32/libhei*
> -r--r--r--  1 root  wheel  - 53622 Mar 17 20:02 /usr/lib32/libheimbase.a
> lrwxr-xr-x  1 root  wheel  -    17 May 24 17:11
> /usr/lib32/libheimbase.so -> libheimbase.so.11
> -r--r--r--  1 root  wheel  - 13224 May 24 17:11
> /usr/lib32/libheimbase.so.11
> -r--r--r--  1 root  wheel  - 54966 Mar 17 20:02 /usr/lib32/libheimbase_p.a
> -r--r--r--  1 root  wheel  - 67598 Mar 17 20:02 /usr/lib32/libheimntlm.a
> lrwxr-xr-x  1 root  wheel  -    17 May 24 17:11
> /usr/lib32/libheimntlm.so -> libheimntlm.so.11
> -r--r--r--  1 root  wheel  - 23840 May 24 17:11
> /usr/lib32/libheimntlm.so.11
> -r--r--r--  1 root  wheel  - 68746 Mar 17 20:02 /usr/lib32/libheimntlm_p.a
> 0{git}%
>
>
> Looking at the pcap, nothing stands out as to what the problem might be.
> Looking at the replies, the ERROR is just "ERROR: No such file or
> directory" and doesnt correlate to the permission denied errors
>
> reading from file nfs.pcap, link-type EN10MB (Ethernet)
>     2 xid  reply ok
> 3371 xid  reply ok getattr CHR 11734062556 ids
>   951 xid  reply ok getattr CHR 12231260544 ids
>   626 xid  reply ok getattr ERROR: No such
>     2 xid  reply ok getattr FIFO 10432271160 ids
> 6758 xid  reply ok getattr FIFO 10731272141 ids
>   165 xid  reply ok getattr FIFO 11433667553 ids
>   214 xid  reply ok getattr FIFO 12231260544 ids
> 3373 xid  reply ok getattr LNK 10333067563 ids
>     2 xid  reply ok getattr LNK 12231267145 ids
>     2 xid  reply ok getattr unk-ft 10 10536061550
>     2 xid  reply ok getattr unk-ft 13 10334462541
>     2 xid  reply ok getattr unk-ft 15 12231261554
>   673 xid  reply ok getattr unk-ft 6 10130661545
>   879 xid  reply ok getattr unk-ft 6 11433667553
>   126 xid  reply ok getattr unk-ft 8 12231260544
>     2 xid  reply ok null
>
>
>
>