Re: vfs.zfs.compressed_arc_enabled=0 is INCOMPATIBLE with L2ARC at least in FreeBSD 13 (Was: Crash on adding L2ARC to raidz1 pool)

From: Peter 'PMc' Much <pmc_at_citylink.dinoex.sub.org>
Date: Sat, 13 Jan 2024 22:16:38 UTC
On 2024-01-13, Alexander Burke <alex@alexburke.ca> wrote:
> Hello,
>
> It looks like the issue is fixed in OpenZFS 2.2 (and thus in FreeBSD 14-RELEASE):
>
> https://github.com/openzfs/zfs/issues/15764#issuecomment-1890491789
>
> Cheers,
> Alex
> ----------------------------------------
>
> Jan 13, 2024 12:26:50 Lev Serebryakov <lev@FreeBSD.org>:
>
>> On 08.01.2024 18:34, Lev Serebryakov wrote:
>> 
>>    I've found that all my L2ARC problems (live-locks and crashes) are result of OpenZFS bug which can not support L2ARC with un-compressed ARC (vfs.zfs.compressed_arc_enabled=0).
>> 
>>    It is NOT hardware-depended (and my NVMe is perfectly Ok and healthy) and could be easily reproduced under VM with all-virtual disks.
>> 
>>    I've opened the ticket in OpenZFS project (https://github.com/openzfs/zfs/issues/15764).
>> 
>>    Maybe, FreeBSD need ERRATA entry?
>> 
>> 
>>    Previous threads:
>> 
>>     [1] ZFS pool hangs (live-locks?) after adding L2ARC
>>     [2] Crash on adding L2ARC to raidz1 pool
>> 
>> -- 
>> // Lev Serebryakov
>
>

Just for the records, there is a note in my loader.conf:

> vfs.zfs.compressed_arc_enabled="1" # 27.7.17: since R11.1 l2_cksum_errs if 0

Apparently I didn't bother to open a ticket, since the general stance
was that one shouldn't mess with the defaults.

The more interesting question might be how we managed to improve from
checksum-errors (that were otherwise harmless) to "live-locks and
crashes" ;)