Re: RFC: nfsd in a vnet jail

On 2022-12-01 17:32, Rick Macklem wrote:
> On Thu, Dec 1, 2022 at 8:23 AM Chris <bsd-lists@bsdforge.com> wrote:
> 
>> On 2022-11-29 16:21, Rick Macklem wrote:
>> > On Sun, Nov 27, 2022 at 10:04 AM Peter Eriksson <pen@lysator.liu.se>
>> wrote:
>> >
>> >> Keep the global variables as defaults that apply to all nfsds and allow
>> >> (at least some subset) to be overridden inside the net jails if some
>> things
>> >> need to be changed from the defaults?
>> >>
>> >> This is pretty much a reply to one of the posts selected at random,
>> > but I thought that better than starting a new email thread.
>> >
>> > bz@ and asomers@ have both asked about running mountd within a vnet
>> prison
>> > (one via offlist email and the other on phabricator).
>> >
>> > I think it is worth discussing here...
>> > mountd (rightly or wrongly) does two distinctly different things:
>> > 1 - It pushes the exports into the kernel via nmount() so they
>> >     can be hung off of the "struct mount" for a file system's
>> >     mount point.
>> >     --> This can only work for file system mount points and can
>> >         only be done once for any given file system mount point.
>> >
>> >     At this time, I have it done once globally outside of the prisons.
>> >     The alternative I can see is doing it within each prison, but I
>> >     think that would require that each prison have its own file
>> system(s).
>> >     (ie. The prison's root would always be a file system mount point.)
>> >
>> > 2 - It handles RPC Mount protocol requests from NFSv3 clients.  This one
>> >     is NFSv3 specific, which is why I have done this NFSv4 only at
>> >     this time.  To do this, it must be able to register with rpcbind,
>> >     and I have no idea if running rpcbind in a vnet jail is practical.
>> >
>> > Enforcing the use for separate file systems for each jail also makes
>> > things safer, since the exports are enforced by the kernel. Without
>> > this, a malicious NFSv4 client could "guess" a file handle for a file
>> > outside the jail and gain access to that file. Put another way, without
>> > a separate file system, there is no way to stop a malicious client from
>> > finding files above the Root file handle. (Normal clients will use
>> > PutRootFH and LookupParent and these won't be able to go above the top
>> > of the jail.)
>> >
>> > So, what do others think of enforcing the requirement that each jail
>> > have its own file systems for this?
>> 
>> I don't care for any of it. It looks like additional overhead with the
>> addition of potential security risks. All for a very limited (and as yet
>> unknown) use case.
>> 
> I am thinking that if/when this goes into main, it would be
> under a new kernel build option called something like
> NFSD_VIMAGE. I think that would avoid the overhead/security
> risks for those that do not need/want it.
Brilliant. Count me in. :-)

--chris
> 
> rick
> 
>> 
>> --chris
>> >
>> > rick
>> >
>> >
>> >> - Peter
>> >>
>> >>
>> >> On Fri, Nov 25, 2022, 4:24 PM Rick Macklem <rick.macklem@gmail.com>
>> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> bz@ has encouraged me to fiddle with the nfsd
>> >>> so that it works in a vnet jail.
>> >>> I have now basically done so, specifically for
>> >>> NFSv4, since NFSv3 presents various issues.
>> >>>
>> >>> What I have not yet done is put global variables
>> >>> in the vnet. This needs to be done so that the nfsd
>> >>> can be run in multiple jail instances and/or in and
>> >>> outside of a jail.
>> >>> The problem is that there are 100s of global variables.
>> >>>
>> >>> I can see two approaches:
>> >>> 1 - Move them all into the vnet jail. This would imply
>> >>>     that all the sysctls need to somehow be changed,
>> >>>     which would seem to be a POLA violation.
>> >>>     It also implies a lot of stuff in the vnet.
>> >>> 2 - Just move the global variables that will always
>> >>>     differ from one nfsd to another (this would make
>> >>>     the sysctls global and apply to all nfsds).
>> >>>     This will keep the number of globals in the vnet
>> >>>     smaller.
>> >>>
>> >>> I am currently leaning towards #2, put what do others
>> >>> think?
>> >>>
>> >>> rick
>> >>> ps: Personally, I don't know what use there is of
>> >>>     running the nfsd inside a vnet jail, but bz@ has
>> >>>     some use case.
>> >>>
>> >>
>> >>
>>