immutable flag breaks 9.1 to 9.2 make world in a jail

[Julian H. Stacey]

Hi freebsd-jail at freebsd.org
I have a problem with an immutable flag, running make world in a jail,
Any ideas please ?

The 9.1-RELEASE jail I built on a 9.1-RELEASE laptop, with
ports/sysutils/ezjail then I replaced all the shared bits with own
local copies of all files within the chroot, it worked fine, & still
works fine from 2 prisons, those prisons I meantime have upgraded on 
2 different partitions, to 9.2 & 10.0-RELEASE ...  but ...

Each time I run a make world within the jail (running on a 9.2
prison), the jail world hangs, & I have to login to prison, go root
& rescue the jail, though I fail to rescue the world upgrade.

The next jail to upgrade is in a prison I'm not root for, It's an
operational jail, (built the same way as this test jail) & I won't
touch it till I find a solution.  Something I'm missing, forgetting,
or a real bug perhaps ?

Here's an approx. abbrevated transcript of what I've tried.  Ideas please ?

Jail:
	cd /usr/src/lib ; make install
		install -s -o root -g wheel -m 444   -fschg -S  libc.so.7 /lib
		install: rename: /lib/INS at 3Xhe to /lib/libc.so.7: \
			Operation not permitted

	Also fails:
		chflags -R noschg /
		chflags noschg /lib/libc.so.7
Prison :
	chflags noschg /usr1/jail/jstest/lib/libc.so.7
		# That still would not allow make world to run in jail, 

	sysctl security.jail.param.allow.chflags=1
Jail:
	chflags -R noschg /
	cd /usr/src
	make world
		===> lib/libc (install)
		install -C -o root -g wheel -m 444   libc.a /usr/lib
		install -C -o root -g wheel -m 444   libc_p.a /usr/lib
		install -s -o root -g wheel -m 444   -fschg -S  libc.so.7 /lib
		install: /lib/libc.so.7: chflags: Operation not permitted
		*** [_libinstall] Error code 71
Prison:
	sysctl -a | grep chflag
		security.jail.param.allow.chflags: 0
		security.jail.chflags_allowed: 0
	sysctl -d security.jail.param.allow.chflags
		security.jail.param.allow.chflags:	\
			 Jail may alter system file flags
	sysctl -d security.jail.chflags_allowed
		security.jail.chflags_allowed:	\
			 Processes in jail can alter system file flags
	sysctl security.jail.param.allow.chflags=1
		security.jail.param.allow.chflags: 0 -> 0
	sysctl security.jail.chflags_allowed=1
		security.jail.chflags_allowed: 0 -> 1
	sysctl -a | grep chflag
		security.jail.param.allow.chflags: 0
		security.jail.chflags_allowed: 1
	sysctl security.jail.param.allow.chflags=1
		sysctl security.jail.param.allow.chflags=1
		security.jail.param.allow.chflags: 0 -> 0
	cd /lib ; tar cf - libc.so.7 | ( cd /usr1/jail/jstest/lib && tar xf - )
Jail:
	cd /usr/src/lib/libc
	make install
		install -C -o root -g wheel -m 444   libc.a /usr/lib
		install -C -o root -g wheel -m 444   libc_p.a /usr/lib
		install -s -o root -g wheel -m 444   -fschg -S  libc.so.7 /lib
		install: rename: /lib/INS at sS7m to /lib/libc.so.7:	\
			 Operation not permitted
	install -s -o root -g wheel -m 444  -S  /usr/obj/`pwd`/libc.so.7 /lib
		install: rename: /lib/INS at lsAo to /lib/libc.so.7:	\
			 Operation not permitted
	install -s -o root -g wheel -m 444 /usr/obj/`pwd`/libc.so.7 /lib
		install: /lib/libc.so.7: Operation not permitted
	chflags -R noschg /
		chflags: /lib/libc.so.7: Operation not permitted
Prison:
	chflags -R noschg /usr1/jail/jstest
	statv /usr1/jail/jstest/lib/libc.so.7
		Flags                                           <none>
	# http://www.berklix.com/~jhs/src/bsd/jhs/bin/public/statv/statv.c
Jail: 
	install -s -o root -g wheel -m 444 /usr/obj/`pwd`/libc.so.7 /lib
		NO ERROR ! But make world will want more so
	install -s -o root -g wheel -m 444 -fschg -S \
		  /usr/obj/`pwd`/libc.so.7 /lib
		install: /lib/libc.so.7: chflags:	\
			 Operation not permitted
Prison:
	ls -l /usr1/jail/jstest/lib/libc.so.7
	cd /lib ; tar cf - libc.so.7 | ( cd /usr1/jail/jstest/lib && tar xf - )
	ls -l /usr1/jail/jstest/lib/libc.so.7
	statv /usr1/jail/jstest/lib/libc.so.7 | grep Flags
		Flags                                           <none>
Jail: 
	install -s -o root -g wheel -m 444 -fschg -S	\
		  /usr/obj/`pwd`/libc.so.7 /lib
		install: /lib/libc.so.7: chflags: Operation not permitted
	chflags noschg /lib/libc.so.7
		Shared object "libc.so.7" not found, required by "chflags"
Prison:
	cd /lib ; tar cf - libc.so.7 | ( cd /usr1/jail/jstest/lib && tar xf - )
	sysctl -a | grep chflag
		security.jail.param.allow.chflags: 0
		security.jail.chflags_allowed: 1
Jail: 
	sysctl -a | grep chflag
		security.jail.param.allow.chflags: 0
		security.jail.chflags_allowed: 0

PS re. auditdistd:
 Jail vipw does show
  auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologin
 (though I did get bitten by lack of that earlier.)

 Curiously my 9.2 prison did not have that line (maybe deleted by mistake)
 I just added it [back] & started another make world overnight in prison 9.2.

 My 10.0 prison /etc/master.passwd does have that line (though I'm not doing
 jail build from 10 prison)

PPS I have always hated FreeBSD immutable bits, & turned them off.

Cheers,
Julian
-- 
Julian Stacey, BSD Unix Linux C Sys Eng Consultant, Munich http://berklix.com
 Reply below not above, like a play script.  Indent old text with "> ".
 Send plain text.  No quoted-printable, HTML, base64, multipart/alternative.