unionfs and nullfs combination

[Nikos Vassiliadis]

Hi Kurt,

On 02/19/2017 12:59 PM, Kurt Jaeger wrote:
> Ah, to correct myself here: We only used unionfs, not in combination
> with nullfs. Can you describe why nullfs with unionfs does not
> work in 11 ?
>

It panics easily. I use the following shell script to create a working
environment:
> PREFIX=/jails
> BASEJAIL=${PREFIX}/base-jail
> JAILS="mongo-1 mongo-2 mongo-3 mongo-4 mongo-5 mongo-6"
>
> mkdir -p $BASEJAIL
> for jail in $JAILS
> do
> 	mkdir -p ${PREFIX}/$jail
> 	mkdir -p ${PREFIX}/upper/$jail
> 	mount -t nullfs -o ro $BASEJAIL ${PREFIX}/$jail
> 	mount -t unionfs -o noatime ${PREFIX}/upper/$jail ${PREFIX}/$jail
> #	mount -t devfs none ${PREFIX}/$jail/dev
> #	cp /etc/resolv.conf ${PREFIX}/$jail/etc/resolv.conf
> done
>
> #chroot $PREFIX/mongo-1 rm -rv /var
> #chroot $PREFIX/mongo-2 rm -rv /var

Then I can trigger a panic if I run this:
rm -rf /jails/mongo-*/*

And the panic:
> Unread portion of the kernel message buffer:
> panic: Memory modified after free 0xfffff80004d0e000(1024) val=adc0dede @ 0xfffff80004d0e000
>
> cpuid = 0
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe012058ce90
> vpanic() at vpanic+0x186/frame 0xfffffe012058cf10
> panic() at panic+0x43/frame 0xfffffe012058cf70
> trash_ctor() at trash_ctor+0x4b/frame 0xfffffe012058cf80
> uma_zalloc_arg() at uma_zalloc_arg+0x514/frame 0xfffffe012058cfe0
> unionfs_relookup() at unionfs_relookup+0x41/frame 0xfffffe012058d040
> unionfs_mkshadowdir() at unionfs_mkshadowdir+0x120/frame 0xfffffe012058d270
> unionfs_lookup() at unionfs_lookup+0x883/frame 0xfffffe012058d3c0
> VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xda/frame 0xfffffe012058d3f0
> vfs_cache_lookup() at vfs_cache_lookup+0xd6/frame 0xfffffe012058d450
> VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xda/frame 0xfffffe012058d480
> lookup() at lookup+0x6d2/frame 0xfffffe012058d520
> namei() at namei+0x504/frame 0xfffffe012058d5e0
> kern_statat() at kern_statat+0x98/frame 0xfffffe012058d790
> sys_fstatat() at sys_fstatat+0x2c/frame 0xfffffe012058d830
> amd64_syscall() at amd64_syscall+0x2f9/frame 0xfffffe012058d9b0
> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe012058d9b0
> --- syscall (493, FreeBSD ELF64, sys_fstatat), rip = 0x8008ba62a, rsp = 0x7fffffffe728, rbp = 0x7fffffffe7e0 ---
> KDB: enter: panic

Thanks in advance for any ideas,
Nikos