Re: Behavior of /dev/pts in a jail?

On Wed, 09 Feb 2022 13:22:13 +0100
Alexander Leidinger <Alexander@leidinger.net> wrote:

> Quoting Michael Gmelin <grembo@freebsd.org> (from Wed, 9 Feb 2022  
> 12:56:49 +0100):
> 
> > I was able to reproduce the issue locally.
> >
> > The problem is caused by jexec inheriting the pty from the jail
> > host.
> >
> > If you use a pty that was created inside of the jail,  
> > gpg-agent/pinentry works as expected.
> >
> > This can be accomplished, e.g., by running tmux inside of the jail:
> >
> >     jexec gpgtest
> >     pkg install tmux
> >     tmux
> >     gpg --gen-key
> >
> > Running sshd inside of the jail and connecting to it using ssh has  
> > the same effect.  
> 
> I confirm (with ssh instead of jexec) the behavior.
> 
> What I don't understand is how this works. ls is not build-in to the  
> shell. So how can it be that the jexec-ed shell can fork ls and it  
> sees the content of /dev/pts/, and the ls forked from  
> gpg->gpg-agent->pinentry-wrapper can't?

gpg-agent starts as a daemon, so it detaches from the controlling
terminal, closing all open fds to the pty.

You can test this yourself easily:

    [root@gpgtest ~]# cat >test.sh<<"EOF"
    #!/bin/sh
    sleep .5
    echo
    echo "/dev/pts contains $(ls /dev/pts | wc -l | xargs) files"
    EOF
    [root@gpgtest ~]# chmod 755 test.sh
    [root@gpgtest ~]# ./test.sh
    /dev/pts contains 1 files
    [root@gpgtest /]# daemon ./test.sh
    [root@gpgtest /]# 
    /dev/pts contains 0 files

You can also compare the file descriptors opened by your shell vs those
opened by gpg-agent using `procstat -f <pid>`.

> And how could we fix this (or
> why wouldn't we want to fix it)?

I think that the current behavior is already hacky, but makes jexec more
useful in practice (so you *can* actually do ssh from one jail to
another without having to create a new pty), thanks to entries in
devfs.rules. In the end, jexec just calls jail_attach, it's not
starting a whole new user session.

Expanding this to daemonized processes would make things a lot more
complex (like: is this the pty created from within the jail or one
created outside of it, what happens to it if the user session on the
jail host ends, etc.).

The one thing I could see is adding a parameter to jexec that makes it
allocate a new pty within the jail (similar to what ssh would do). This
would also workaround the issues I had with bhyve (ttyu* not being
allowed to seen inside the jail). Or write a thin wrapper that is
called inside of the jail, to run a program in a newly allocated
pty.

Maybe someone with more insights to how jails work internally could
give their input here.

In the meantime, tmux is probably the most lightweight way of working
around this in your specific use-case, without having to run sshd.

Cheers
Michael

p.s. Once you know what you're looking for, you can find mailing list
and forum entries about it going back some 15 years, which include
pretty much the same workarounds I came up with *sigh*

> 
> Bye,
> Alexander.
> 



-- 
Michael Gmelin