AWS CloudWatch Logs Agent

[Rafal Lukawiecki]

Thank you, Colin, for your suggestion. I have never run the Linux emulation system in FreeBSD yet, I suppose this is a good time to try it. However, I did not realise that it would be able to cope with setup of rc/runlevels/deamons. I thought it was only able to run more mundane utilities, rather than assist in system/boot init set-up.

Regarding your comment about CloudWatch metrics, I am *also* using CloudWatch to report in-machine memory and CPU utilisation in my existing Linux machines. I have followed this guide: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html

It was easy. Essentially, I have installed the script from http://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.1.zip and in my crontab I have something along the lines of:

*/5 * * * * /opt/aws-scripts-mon/mon-put-instance-data.pl --mem-util --mem-used --mem-avail --swap-util --swap-used --disk-path=/ --disk-space-util --disk-space-used --disk-space-avail --from-cron

The metrics magically and reliably appear in CloudWatch so that I can set-up alarms etc. It works really well in Linux, I hope to replicate the success with FreeBSD.

As for the CW Logs, unfortunately the script at curl https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py is a bit too daunting for me to try rewriting using my brand-new rc.d scripting skills, with only a hundred lines working (!) so far behind my belt.

Thank you for all your help, which is much appreciated. I hope to be able to give back somehow one day…if I manage to persist in this adventure. Fingers crossed.

Rafal
--
Rafal Lukawiecki
Data Scientist and Director 
Project Botticelli Ltd

> On 31 Aug 2017, at 00:06, Colin Percival <cperciva at tarsnap.com> wrote:
> 
> On 08/30/17 07:07, Rafal Lukawiecki wrote:
>> One nice AWS component is its CloudWatch Logs engine that collects streams
>> of syslog-generated data and centralises them, with a simple but usable UI.
>> AWS provides an automated set-up for the awslogs daemon agent, but from a
>> cursory look at the code, although parts of it are plain Python, much of it
>> is very Linux-focused shell script, with much regard for runlevels,
>> logrotate etc and not immediately usable as an rc.d script.
>> 
>> I was wondering if anyone has hacked that script yet to run on FreeBSD or
>> if you have any other suggestions how to help me get AWS CloudWatch Logs
>> integrated into FreeBSD running on AWS.
> 
> Huh, somehow I never noticed CloudWatch Logs; the aspect of CloudWatch I had
> been thinking could be useful for FreeBSD was to create an agent which would
> record memory/swap/disk usage to CloudWatch.
> 
> Unfortunately I don't have time to work on either right now; but for a very
> simple proof of principle I'd suggest running the Linux CloudWatch Logs agent
> via the Linux emulation system.
> 
> -- 
> Colin Percival
> Security Officer Emeritus, FreeBSD | The power to serve
> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid