nosh version 1.30

[Jonathan de Boyne Pollard]

The nosh package is now up to version 1.30 .

* http://jdebp.eu./Softwares/nosh/
* 
https://www.freebsd.org/news/status/report-2015-07-2015-09.html#The-nosh-Project
* http://jdebp.info./Softwares/nosh/

service bundles
---------------

As usual, there are more service bundles, including for the UWSGI 
"Emperor" and the new services in FreeBSD/TrueOS 11 such as ypldap. 
There are now services to run Sendmail in the same manner as the 
services that run exim.  Note that this is slightly different to the old 
FreeBSD division of labour.  There are individually controllable 
services for SMTP Submission, SMTP Relay, the Submission queue runner, 
and the Relay queue runner.

doco
----

The Guide has been extended with several new chapters, including a 
gazetteer of interesting directories, a chapter on log file 
post-processing, a chapter on logging security, a chapter on per-user 
service management, and some notes for individual services.  The 
commands list has moved from the blurb into the Guide, too, as it seems 
like something that an administrator might find handy to have available 
when there's no Internet connection.

* http://jdebp.eu./Softwares/nosh/guide.html

service management
------------------

There's now a hardlimit chain-loading command, analogous to softlimit.  
The convert-systemd-services utility now makes use of this and permits 
setting separate hard and soft limits, or only one or the other, with 
settings like LimitOFILE=32:128 and LimitNPROC=:infinity .

There's now a local-reaper chain-loading command, that can turn "local 
reaper" status for the current process on or off.  Have a care when 
using this, per the note on the manual page.  There is a 
LocalReaper=true extension to systemd service units for this.

netlink-datagram-socket-listen is now available on the BSDs for script 
compatibility.  It always aborts with an address family error.

There's a new hangup subcommand of system-control, equivalent to the 
existing -H option to svc .

enhancements to system-control stop/start/reset and single-shot services
------------------------------------------------------------------------

This is the first big item for 1.30 :

The start and stop subcommands of system-control now operate more 
quickly.  Instead of polling once per second, they monitor the 
supervise/status files of each service that is in the process of being 
started and stopped, with kevent().

In addition, system-control now supports the notion of services that 
become ready when their main process has exited, marked with a new flag 
file in the service directory.  convert-systemd-units has been modified 
to convert "oneshot" services to this, instead of to services that put 
all of the run code into the start program.  Thus "oneshot" services 
that are running their actual main programs are reported as "running" by 
svstat, rather than as "starting".

This takes advantage of the extended status information that 
service-manager has been writing to the status file since version 1.28.  
The sharp-eyed may have noticed that in version 1.28 the output of 
"svstat"/"system-control status" gained information about the exit 
statuses of the start, run, restart, and stop programs. This is what 
system-control now uses to detect whether ready-after-run services ran 
before they stopped.  (Detection of ready-after-run services that are 
running with no processes, because they are "remain" services, can be 
and is done with just the daemontools-encore-compatible status information.)

Old-style "oneshot"s will continue to work as before, as of course they 
become ready as soon as the run process is spawned, which is after they 
have run their programs as part of start.

The benefit of this new style, apart from reporting a running service as 
actually "running", which should help with nagios monitoring and the 
like, is that "oneshot" services converted from systemd no longer have 
to be marked as RemainAfterExit=true in order to avoid a dummy "pause" 
process hanging around.  This is the case for old-style "oneshot" 
services.  They have to run something in run, after all, and that 
something has to keep running in order for the service to be considered 
ready and services ordered after it to be unblocked.  A ready-after-run 
service, however, unblocks ordered-after services if it has reached the 
stopped state via a run, thus puts its programs in run, thus doesn't 
have to have a dummy pause process, and can be RemainAfterExit=false 
without adding to the process list.

log file management
-------------------

export-to-rsyslog had a bug that caused it to skip old log files (the 
@nnnnnnnnnnnnnnnnnnnnnnnn.s ones) in catch-up mode.  This has been 
corrected.  There is now a follow-log-directories command that can 
substitute for tail -F .  It knows the actual structure of log 
directories, operates using one or more cursors like export-to-rsyslog 
does, and copes correctly with cyclog/multilog log rotation (which GNU 
tail, at least, apparently has problems with when the timing is 
particularly wrong on a loaded system).

See also http://jdebp.eu./FGA/do-not-use-logrotate.html

build
-----

More warnings are now turned on with clang++ during the build, and a lot 
of the resultant warnings have been eliminated where appropriate.  The 
check for eg++ in preference to g++ is now limited to OpenBSD, where (at 
least on OpenBSD 5.9) eg++ is still ahead of g++ by a wide margin.

Per-user service management
---------------------------

Changes in per-user service management are the second big item for 1.30 :

The per-user service manager instances are now invoked via userenv, so 
all per-user services that you run under nosh service management, D-BUS 
servers or otherwise, will have your own HOME, SHELL, and USER set.  
Several per-user daemon softwares were expecting HOME to be set.

To match what the Desktop Bus people are doing, the dbus socket path for 
the per-user D-BUS broker has changed from 
"/run/user/$USER/dbus/user_bus_socket" to "/run/user/$USER/bus".  In 
theory, this is addressable (in D-BUS speak) as "unix:runtime=yes". In 
practice, there is no version of D-BUS available on stable/release 
FreeBSD, TrueOS, or Debian that understands this address syntax.  So one 
still has to use "unix:path=/run/user/$USER/bus".

The Desktop Bus people and the desktop environments people are also 
switching from per-login D-BUS brokers to per-user D-BUS brokers. The 
nosh toolset has already had this for over a year, since the middle of 
2015.  Each real-person user account has an optional per-user service 
management service (e.g. user-services at fred).  What is new is that 
per-user service bundle areas are now populated with a whole load of 
service bundles for real services, many relating to GUI desktop 
environments, and the per-user D-BUS broker has moved to there, from 
being a system-level service bundle.

The configuration import subsystem creates these new per-user service 
bundles in the home directories of individual real users, under 
~fred/.config/service-bundles/services/ and 
~fred/.config/service-bundles/targets/ (for user fred).  These run 
per-user services for a whole load of things, from GNOME editor and 
emacs through dconf and KDE Notify to urxvtd and GNOME Terminal.

The configuration import subsystem also sets up a bypass for D-BUS's 
broken "bus activation" mechanism, so that instead of attempting to run 
these D-BUS servers directly, the D-BUS broker instead tells the nosh 
per-user service manager to run them.  This takes the form of a 
replacement dbus-daemon-launch-helper, and the per-user D-BUS brokers 
now employ a modified configuration file that invokes it.

There's a full explanation of how this all works in the new chapter on 
demand-starting user-level Desktop Bus services in the nosh Guide.

Notes:
   * For emacs as a per-user service, you must have a very recent emacs 
with its very-late-to-the-party --new-daemon option.
   * GNOME Weather and its interaction with GeoClue2 are only partly 
tested, because the versions of them available for the test platforms 
were attempting to contact a weather service that the U.S. Government 
discontinued in June 2016; and this was hardwired into their code.