Re: Proposal: Disable compression of newsyslog by default

On 2023-12-24 10:03, Rodney W. Grimes wrote:
>> On 2023-12-23 06:52, Konstantin Belousov wrote:
>>> On Fri, Dec 22, 2023 at 11:18:23PM -0800, Xin Li wrote:
>>>> Hi,
>>>>
>>>> Inspired by D42961, I propose that we move forward with disabling the
>>>> compression by default in newsyslog, as implemented in
>>>> https://reviews.freebsd.org/D43169
>>>>
>>>> Historically, newsyslog has compressed rotated log files to save disk space.
>>>> This approach was valuable in the early days where storage space was
>>>> limited.  However, the landscape has changed significantly.  Modern file
>>>> systems, such as ZFS, now offer native compression capabilities.
>>>> Additionally, the widespread availability of larger hard drives has
>>>> diminished the necessity for additional compression.  Notably, the need to
>>>> decompress log files for pattern searches poses a significant inconvenience,
>>>> further questioning the utility of this legacy feature.
>>>>
>>>> In commit 906748d208d3, flags J, X, Y, Z can now indicate that a log file is
>>>> eligible for compression rather than directly enforcing it. It allows for a
>>>> more flexible approach, wherein the actual compression method can be set to
>>>> "none" or specified as one among bzip2, gzip, xz, or zstd.
>>>>
>>>> Therefore I would propose that we change the default compression setting to
>>>> "none" in FreeBSD 15.0.  This change reflects our adaptation to the evolving
>>>> technological environment and user needs.  It also aligns with the broader
>>>> initiative to modernize our systems while maintaining flexibility and
>>>> efficiency.
>>>>
>>>> I look forward to your thoughts and feedback on this proposal.
>>>
>>> This is strange change at best.  I have no opinion about the disabling
>>> of compression of the rotated logs by default, but we already have knobs
>>> to do that.  Adding a knob that disables (or enables) other knobs to work
>>> is weird.
> 
> And counter intuitive!
> 
>>> If you want to change the compression, update the default configuration file.
>>
>> The primary issue with simply updating the default configuration file is
>> the increased workload it imposes during system upgrades. Since the
>> compression method flag is a part of newsyslog.conf, standard conflict
>> resolution tools like diff3 struggle to automatically resolve changes
>> involving these flags without manual intervention. This situation
>> necessitates that users manually reconcile their configuration with
>> every update to newsyslog.conf, even for minor alterations like
>> switching the default compression method.
>>
>> Therefore, the proposal isn't about adding another knob within
>> newsyslog.conf. Rather, it's about introducing a command-line option for
>> newsyslog (to be used in /etc/crontab) that specifies the preferred
>> compression method, or the choice not to compress at all. This approach
>> is more self-contained compared to modifying each line in
>> newsyslog.conf. It offers a simpler, more straightforward solution for
>> administrators to manage their compression settings, reducing the
>> administrative burden during upgrades.
> 
> So now I edit /etc/crontab to change the behavior of newsyslog?
> Thats VERY counter to how things "should be."  I am sorry, but
> this change is just yet another "Oh my god, the defaults dont
> work for me so I am gona force my idea of what the defaults should
> be on everone else."
> 
> And you just pushed the merge conflict to another file, one that
> is not even related to newsyslog, you do get your arguement doesnt
> hold water?  A change in /etc/newsyslog.conf or /etc/crontab BOTH
> lead to the fact people are going to have to merge things.
> 
> Now we are goning go crazy trying to figure out why my logs are
> not compressed and finally gona end up finding this flag in
> /etc/crontab that is causing newsyslog to ignore what I put
> in its proper config file???  Again, very counter intuitive.

Thank you for your feedback.

In response to the concerns you've raised, I've initiated a new change, 
which you can review at https://reviews.freebsd.org/D43174 . This 
introduces a new configuration file option, "<compress>", in the 
newsyslog.conf filem, intended for use at the beginning of the file, and 
updated https://reviews.freebsd.org/D43169 with the default change done 
in the configuration file.

This change aims to provide a more intuitive way to manage compression 
settings directly within the newsyslog configuration, hopefully 
addressing the issues you highlighted.

Cheers,