Re: Troubles with Kibana

On 03/02/2024 16:24, Andrea Venturoli wrote:
> Hello.
> 
> I've got a 13.2p9/amd64 server where I have a jail dedicated to 
> ElasticSearch+Kibana and I'm having troubles starting and stopping the 
> latter.
> 
>> # pkg info | grep -E "(elastic|kibana)"
>> elasticsearch8-8.11.3          Distributed, RESTful search and 
>> analytics engine
>> kibana8-8.11.3                 Browser based analytics and search 
>> interface to ElasticSearch
>> py39-elasticsearch-7.17.9      Official Python low-level client for 
>> Elasticsearch
>> py39-elasticsearch-dsl-7.3.0   High level Python client for Elasticsearch
> 
>> # cat /etc/rc.conf
>> #cron_enable="NO"
>> kibana_enable="YES"
>> kibana_syslog_output_enable="YES"
>> elasticsearch_enable="YES"

> 
> First problem
> 
> When I start the jail (with "ezjail-admin start") Elastic starts, but 
> Kibana does not.
> syslog shows:
>> [2024-02-03T16:14:21.861+01:00][INFO ][root] Kibana is starting
>> [2024-02-03T16:14:21.939+01:00][INFO ][root] Kibana is shutting down
>> [2024-02-03T16:14:21.939+01:00][FATAL][root] Reason: EACCES: 
>> permission denied, open '/var/run/kibana.pid'
>> Error: EACCES: permission denied, open '/var/run/kibana.pid'
>>
>>  FATAL  Error: EACCES: permission denied, open '/var/run/kibana.pid'
> 
> Same happens if I try "service kibana start" afterwards from inside the 
> jail.
> I can get around this with:
>> # touch /var/run/kibana.pid
>> # chown www /var/run/kibana.pid
>> # service kibana start
> 
> However this must be done manually everytime I need to restart the 
> server or jail.

Both, the first and the second problem are caused by permissions problem 
on PID file.
I no longer use Kibana, but if Kibana is running as user www and tries 
to write to /var/run/ as that user, it must fail.
But a quick look at the rc.d/kibana [1] show that the PID file should be 
written to subdirectory /var/run/kibana/kibana.pid

Do you have any local modification to the path of the PID file? (for 
example, in the kibana.yml file)

/var/run/kibana/ is created by rc.d/kibana with the correct owner and 
permissions, so "it should work".

Try to figure out why your Kibana is trying to write /var/run/kibana.pid 
instead of /var/run/kibana/kibana.pid.

> Second problem
> 
> When I try to stop Kibana, it just hangs indefinitely:
>>  # service kibana stop
>> Stopping kibana.
>> Waiting for PIDS: 28179
> 
> syslog reports:
>> [2024-02-03T16:20:22.309+01:00][INFO ][root] SIGTERM received - 
>> initiating shutdown
>> [2024-02-03T16:20:22.309+01:00][INFO ][root] Kibana is shutting down
>> [2024-02-03T16:20:22.311+01:00][INFO ][plugins-system.standard] 
>> Stopping all plugins.
>> [2024-02-03T16:20:22.314+01:00][INFO 
>> ][plugins.monitoring.monitoring.kibana-monitoring] Monitoring stats 
>> collection is stopped
>> [2024-02-03T16:20:22.323+01:00][INFO ][plugins-system.standard] All 
>> plugins stopped.
>> [2024-02-03T16:20:22.324+01:00][WARN ][environment] Detected an 
>> unhandled Promise rejection: Error: EACCES: permission denied, unlink 
>> '/var/run/kibana.pid'

^^^ PID file cannot be deleted as user www because parent dir is owned 
by root and no one else can write to it.

[1] https://cgit.freebsd.org/ports/tree/textproc/kibana8/files/kibana.in

Kind regards
Miroslav Lachman