Re: Proposal: Disable compression of newsyslog by default

On 2023-12-23 00:51, Miroslav Lachman wrote:
> On 23/12/2023 08:18, Xin Li wrote:
>> Hi,
>>
>> Inspired by D42961, I propose that we move forward with disabling the 
>> compression by default in newsyslog, as implemented in 
>> https://reviews.freebsd.org/D43169
>>
>> Historically, newsyslog has compressed rotated log files to save disk 
>> space. This approach was valuable in the early days where storage 
>> space was limited.  However, the landscape has changed significantly.  
>> Modern file systems, such as ZFS, now offer native compression 
>> capabilities. Additionally, the widespread availability of larger hard 
>> drives has diminished the necessity for additional compression.  
>> Notably, the need to decompress log files for pattern searches poses a 
>> significant inconvenience, further questioning the utility of this 
>> legacy feature.
>>
>> In commit 906748d208d3, flags J, X, Y, Z can now indicate that a log 
>> file is eligible for compression rather than directly enforcing it. It 
>> allows for a more flexible approach, wherein the actual compression 
>> method can be set to "none" or specified as one among bzip2, gzip, xz, 
>> or zstd.
>>
>> Therefore I would propose that we change the default compression 
>> setting to "none" in FreeBSD 15.0.  This change reflects our 
>> adaptation to the evolving technological environment and user needs.  
>> It also aligns with the broader initiative to modernize our systems 
>> while maintaining flexibility and efficiency.
>>
>> I look forward to your thoughts and feedback on this proposal.
> 
> I don't think anything needs to be changed on newsyslog. Those who want 
> to disable compression can do so in the "default" newsyslog.conf file. 
> Why force this change in the newsyslog code?
> I also don't think that the log file should not be compressed even on a 
> compressed filesystem. Compressed log files can be handled by tools like 
> zcat, zless, zgrep, etc. without first decompressing the log file. Text 
> log files can still grow to large sizes, and if you have a daily backup 
> job over a long distance network, if you use a protocol without own 
> compression, it is still better to have compressed log files than 
> uncompressed on a compressed filesystem. To me, it's the same as 
> compressing large database dumps and not relying on filesystem compression.
> YMMV, but I really don't see any benefit of changing the newsyslog code, 
> just change defaults in newsyslog.conf.

I appreciate your perspective on this issue. However, I believe there 
are additional benefits to modifying the newsyslog code (which is 
already done in commit 906748d208d3, by the way) beyond what can be 
achieved by simply adjusting the defaults in newsyslog.conf.

Firstly, updating the newsyslog code provides us with greater 
flexibility when introducing new compression modes, such as 'xz -9e', or 
in supporting other compressors. This approach is more efficient than 
continually adding new letters to represent each compression method in 
the configuration file. It simplifies the process and avoids unnecessary 
complexity.

Furthermore, the previous method of representing compression methods in 
the configuration file can be cumbersome for users who have disabled or 
altered their compression configuration.  Each time there is a change in 
newsyslog.conf, these users face potential conflicts that require manual 
resolution.  Updating the code would make the configuration in one 
central place (/etc/crontab) therefore simplifying maintenance for users 
and administrators alike.

While I acknowledge your points regarding the utility of compressed log 
files, especially in scenarios involving daily backups over 
long-distance networks, I believe the flexibility and ease of 
maintenance offered by updating the code would be a significant 
improvement for many users.

Looking forward to further discussion on this.

Cheers,