Re: RFC: nfsd in a vnet jail

On 2022-11-29 16:21, Rick Macklem wrote:
> On Sun, Nov 27, 2022 at 10:04 AM Peter Eriksson <pen@lysator.liu.se> wrote:
> 
>> Keep the global variables as defaults that apply to all nfsds and allow
>> (at least some subset) to be overridden inside the net jails if some things
>> need to be changed from the defaults?
>> 
>> This is pretty much a reply to one of the posts selected at random,
> but I thought that better than starting a new email thread.
> 
> bz@ and asomers@ have both asked about running mountd within a vnet prison
> (one via offlist email and the other on phabricator).
> 
> I think it is worth discussing here...
> mountd (rightly or wrongly) does two distinctly different things:
> 1 - It pushes the exports into the kernel via nmount() so they
>     can be hung off of the "struct mount" for a file system's
>     mount point.
>     --> This can only work for file system mount points and can
>         only be done once for any given file system mount point.
> 
>     At this time, I have it done once globally outside of the prisons.
>     The alternative I can see is doing it within each prison, but I
>     think that would require that each prison have its own file system(s).
>     (ie. The prison's root would always be a file system mount point.)
> 
> 2 - It handles RPC Mount protocol requests from NFSv3 clients.  This one
>     is NFSv3 specific, which is why I have done this NFSv4 only at
>     this time.  To do this, it must be able to register with rpcbind,
>     and I have no idea if running rpcbind in a vnet jail is practical.
> 
> Enforcing the use for separate file systems for each jail also makes
> things safer, since the exports are enforced by the kernel. Without
> this, a malicious NFSv4 client could "guess" a file handle for a file
> outside the jail and gain access to that file. Put another way, without
> a separate file system, there is no way to stop a malicious client from
> finding files above the Root file handle. (Normal clients will use
> PutRootFH and LookupParent and these won't be able to go above the top
> of the jail.)
> 
> So, what do others think of enforcing the requirement that each jail
> have its own file systems for this?

I don't care for any of it. It looks like additional overhead with the
addition of potential security risks. All for a very limited (and as yet
unknown) use case.

--chris
> 
> rick
> 
> 
>> - Peter
>> 
>> 
>> On Fri, Nov 25, 2022, 4:24 PM Rick Macklem <rick.macklem@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> bz@ has encouraged me to fiddle with the nfsd
>>> so that it works in a vnet jail.
>>> I have now basically done so, specifically for
>>> NFSv4, since NFSv3 presents various issues.
>>> 
>>> What I have not yet done is put global variables
>>> in the vnet. This needs to be done so that the nfsd
>>> can be run in multiple jail instances and/or in and
>>> outside of a jail.
>>> The problem is that there are 100s of global variables.
>>> 
>>> I can see two approaches:
>>> 1 - Move them all into the vnet jail. This would imply
>>>     that all the sysctls need to somehow be changed,
>>>     which would seem to be a POLA violation.
>>>     It also implies a lot of stuff in the vnet.
>>> 2 - Just move the global variables that will always
>>>     differ from one nfsd to another (this would make
>>>     the sysctls global and apply to all nfsds).
>>>     This will keep the number of globals in the vnet
>>>     smaller.
>>> 
>>> I am currently leaning towards #2, put what do others
>>> think?
>>> 
>>> rick
>>> ps: Personally, I don't know what use there is of
>>>     running the nfsd inside a vnet jail, but bz@ has
>>>     some use case.
>>> 
>> 
>>