How early can jails be started?

[Warren Block]

On Sat, 9 Aug 2014, James Gritton wrote:

> On 8/9/2014 6:53 PM, Warren Block wrote:
>> Is it technically possible to start a jail much earlier in the boot order?
>> 
>> The reason is that a jailed DNS server could be used by the host if it was 
>> started before any of the host's network services needed DNS.  After 
>> /etc/rc.d/netwait, say.
>> 
>> There may be other jailed services that would also benefit from an early 
>> start, but DNS is something of a special case and the only one that comes 
>> to mind.
>
> Sure - jails can go quite early.  Technically, very near the beginning.
>
> You'll want local filesystems, assuming you want your jail chrooted somewhere 
> (you do for normal-use jails, but it's not as obviously true for 
> single-purpose jails).  In the same situation, you'd want to depend on devfs 
> so you can mount a devfs with the proper ruleset.
>
> If you want to add IP address aliases, you'll need networking set up, but if 
> you just want to restrict to already existing addresses or run in an 
> unrestricted IP setup, you don't even need that. Except ...
>
> Other than that, the only restriction is what you want to do with the jail. 
> So for the DNS server example, it's whatever an unjailed DNS server would 
> require.  So yeah, something like netwait.

(Sorry for multiposting--I put this on the ezjail list also.  But it's 
generally applicable to ordinary jails too.)

It works... mostly.  This file is /etc/rc.d/earlyjail:

   #!/bin/sh
   # PROVIDE: earlyjail
   # REQUIRE: netwait
   # KEYWORD:
   # BEFORE:  mountcritremote
   /usr/local/etc/rc.d/ezjail start dns1

That was a quick hack, not expected to work, but it did.  However...

/usr/local/etc/rc.d/ezjail

When /etc/rc.d/jail runs much later in the startup, it tries to start 
that jail again, and gets an error because of it.  Seeing the error, it 
deletes /var/run/jail_dns1.id.  ezjail uses those jail_*.id files to 
detect which jails are running, and is sure that dns1 is not running. 
jls does show things correctly.  I'm not sure if there is a workaround 
short of modifying /etc/rc.d/jail.

The second problem might be simpler to solve.  With sendmail_enable="NO" 
in the dns1 jail (so it can send status email), sendmail on the host is 
blocked:

sm-mta[679]: daemon Daemon0: problem creating SMTP socket
sm-mta[679]: NOQUEUE: SYSERR(root): opendaemonsocket: daemon Daemon0: cannot bind: Can't assign requested address

If the host sendmail is killed and restarted, it works.  And of course 
it also works when sendmail is started on the host first and the jails 
use sendmail_enable="NO".  I'm not really sure what's going on there.