Need advice: Better Jail integration into ps/top, setpwfile gone forever?

From: antranigv <antranigv_at_freebsd.am>
Date: Mon, 23 Aug 2021 10:02:39 UTC
Greetings all,

I am trying to have better integration of top(1) and ps(1) with FreeBSD Jails.

The main problem that I am trying to solve is displaying the correct UID username. Here's an example.

I have a host (srv0), it is running a Jail named "fsoc", The Jail "fsoc" has a user named "romero" with the UID 1001.

If I run `ps auxd` in the Jail, I get the following,

romero@fsoc:~ $ ps auxd
USER    PID %CPU %MEM   VSZ  RSS TT  STAT STARTED    TIME COMMAND
root   4377  0.0  0.0 11376  956  -  SsJ  14:15   0:00.38 /usr/sbin/syslogd -ss
root   5758  0.0  0.1 13128 1352  1  IJ   18:24   0:00.02 /bin/tcsh -i
root   5763  0.0  0.0 12048  960  1  IJ   18:24   0:00.01 - su - romero
romero 5764  0.0  0.1 12120 2268  1  SJ   18:24   0:00.02 `-- -su (sh)
romero 9625  0.0  0.1 11684 2576  1  R+J  09:41   0:00.01   `-- ps auxd

Good!

However, if I try to run it on the host, here's what I get,

root@srv0:~ # ps auxd -J fsoc
USER  PID %CPU %MEM   VSZ  RSS TT  STAT STARTED    TIME COMMAND
root 4377  0.0  0.0 11376  956  -  SsJ  14:15   0:00.38 /usr/sbin/syslogd -ss
root 5758  0.0  0.1 13128 1352  1  IJ   18:24   0:00.02 /bin/tcsh -i
root 5763  0.0  0.0 12048  960  1  IJ   18:24   0:00.01 - su - romero
1001 5764  0.0  0.1 12124 2436  1  I+J  18:24   0:00.02 `-- -su (sh)

As you can see, in the User field it says 1001, because the host does not have a user with that UID.

This seems fine, but it becomes an issue when you have multiple Jail and a large host running.

Here's an example if the host had a user with UID 1001,

root@pingvinashen:~ # ps auxd -J oragir
USER        PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
root        949  0.0  0.0  11344  2584  -  IsJ  Mon19   0:01.13 /usr/sbin/cron -s
root       1962  0.0  0.0  11428  2796  -  SsJ  Mon19   0:01.83 /usr/sbin/syslogd -ss
antranigv 95342  0.0  0.0  11004  2424  -  IsJ  Mon19   0:00.48 daemon: /usr/home/oragir/writefreely/writefreely[9992] (daemon)
antranigv  9992  0.0  0.4 767244 58336  -  IJ   Mon19   2:58.87 - /usr/home/oragir/writefreely/writefreely

Now, you would think that this is good, however, if you run this in the jail,

root@oragir:~ # ps auxd
USER      PID %CPU %MEM    VSZ   RSS TT  STAT STARTED    TIME COMMAND
root      949  0.0  0.0  11344  2584  -  SsJ  Mon15   0:01.13 /usr/sbin/cron -s
root     1962  0.0  0.0  11428  2796  -  SsJ  Mon15   0:01.83 /usr/sbin/syslogd -ss
oragir  95342  0.0  0.0  11004  2424  -  IsJ  Mon15   0:00.48 daemon: /usr/home/oragir/writefreely/writefreely[9992] (daemon)
oragir   9992  0.0  0.4 767244 58336  -  IJ   Mon15   2:58.88 - /usr/home/oragir/writefreely/writefreely
root    88228  0.0  0.0  13336  4004  8  SJ   09:45   0:00.01 /bin/csh -i
root    99502  0.0  0.0  11824  3140  8  R+J  09:45   0:00.00 - ps auxd

As you can see, the UID 1001 was not `antranigv`, instead it was `oragir`.

This has been an issue for me, so I tried writing some code to implement the following.

If the process is in a Jail, then change the passwd db from /etc to /path/of/the/jail/etc.

I thought it would be an easy thing to do, but not so much.

Here's what I've tried.

1) Call jail_attach and run ps inside the Jail. Oh yeah, it's a jail! after attaching to it there is no way to deattach :-) silly me!

2) Change the passwd file for getpwuid/getpwnam. I wanted to use setpwfile(3) but turns out that \
COMPATIBILITY
     The historic function setpwfile(3), which allowed the specification of
     alternate password databases, has been deprecated and is no longer
     available.

Okay, So I look into how other tools like pwd_mkdb is written and I see that everything is defined (pun intended) the following way,

in /usr/include/pwd.h

#define _PATH_PWD               "/etc"
#define _PATH_PASSWD            "/etc/passwd"
#define _PASSWD                 "passwd"
#define _PATH_MASTERPASSWD      "/etc/master.passwd"
#define _MASTERPASSWD           "master.passwd"

#define _PATH_MP_DB             "/etc/pwd.db"
#define _MP_DB                  "pwd.db"
#define _PATH_SMP_DB            "/etc/spwd.db"
#define _SMP_DB                 "spwd.db"

#define _PATH_PWD_MKDB          "/usr/sbin/pwd_mkdb"

and pwd_mkdb does the following

...
strcpy(prefix, _PATH_PWD);
...
                case 'd':
                        dflag++;
                        strlcpy(prefix, optarg, sizeof(prefix));
                        break;
...

Tuns out it parses the DB file, but I don't want to do that in ps/top! :-)

3) Just for fun, I played with chroot. I tried the following code.
# cat getpw.c 

#define MAXHOSTNAMELEN  255
#define MAXPATHLEN      255
#include <pwd.h>
//#include <jail.h>
#include <stdio.h>
#include <unistd.h>
#include <sys/uio.h>
#include <sys/jail.h>
#include <sys/param.h>
#include <sys/types.h>

int main(){
        // Just get root!
        struct passwd *pwd;
        printf("just root: %s\n", (getpwuid(0))->pw_name);

        // let's try with undef/define
#undef _PATH_PWD         
#undef _PATH_PASSWD      
#undef _PASSWD           
#undef _PATH_MASTERPASSWD
#undef _MASTERPASSWD     

#undef _PATH_MP_DB       
#undef _MP_DB            
#undef _PATH_SMP_DB      
#undef _SMP_DB           

#define _PATH_PWD               "/zdata/jails/fsoc/etc"
#define _PATH_PASSWD            "/zdata/jails/fsoc/etc/passwd"
#define _PASSWD                 "passwd"
#define _PATH_MASTERPASSWD      "/zdata/jails/fsoc/etc/master.passwd"
#define _MASTERPASSWD           "master.passwd"

#define _PATH_MP_DB             "/zdata/jails/fsoc/etc/pwd.db"
#define _MP_DB                  "pwd.db"
#define _PATH_SMP_DB            "/zdata/jails/fsoc/etc/spwd.db"
#define _SMP_DB                 "spwd.db"
        pwd = getpwuid(1001);
        if (pwd == NULL) {
                printf("using undef/define: no user found\n");
        } else {
                printf("using undef/define: %s\n", pwd->pw_name);
        }

        // let's try with chroot!
        chroot("/zdata/jails/fsoc");
        pwd = getpwuid(1001);
        if (pwd == NULL) {
                printf("after chroot: no user found\n");
        } else {
                printf("after chroot: %s\n", pwd->pw_name);
        }

        // escape back the chroot ;-)
        chroot("../../../../");
        pwd = getpwuid(1001);
        if (pwd == NULL) {
                printf("after unchroot: no user found\n");
        } else {
                printf("after chroot: %s\n", pwd->pw_name);
        }

        return 42;
}

And I get the following:

# ./getpw 
just root: root
using undef/define: no user found
after chroot: romero
after unchroot: no user found

So, any advice? should I do chroot in ps? (no I don't think that's a good idea), should I add a new call that implements setpwfile(3)? But I really want to know why it was removed, I'm sure there's a story there. Or is there a better way?

Kind regards, have a nice day!

--
antranigv
https://antranigv.am/