Re: Swap, ZFS & ARC

On Thu, Feb 2, 2023 at 9:17 AM Bakul Shah <bakul@iitbombay.org> wrote:

>
> On Feb 2, 2023, at 6:28 AM, jbo@insane.engineer wrote:
>
>
> Hello folks,
>
> Based on a discussion on the forums not so long ago I tried to figure out
> how swap usage on a ZFS system plays together with ARC. However, I could
> find very little to no information on this which leads me to believe that
> there is some "core concept" I might be oblivious to.
>
> The main question is basically this: Your system starts to swap out data
> from RAM to your swap partition. This swap data on disk ultimately resides
> somewhere in a ZFS pool. If this data then gets accessed, it might be
> cached by ARC essentially eating up memory again which seems counter
> productive.
> Is there any magic which prevents swap partitions from being loaded into
> ARC? Or is this a non-issue for some other reason?
>
>
> I suspect this bug affects FreeBSD as well:
>
> https://github.com/openzfs/zfs/issues/7734
>
> From https://github.com/openzfs/zfs/issues/7734#issuecomment-422082279
>
> I'm not an expert in this area of the code, but I think that swap on ZVOL
> is inherently unreliable due to writes to the swap ZVOL having to go
> through the normal TXG sync and ZIO write paths, which can require lots of
> memory allocations by design (and these memory allocations can stall due to
> a low memory situation). I believe this to be true for swap on ZVOL for
> illumos, as well as Linux, and presumably FreeBSD too (although I have no
> experience using it on FreeBSD, so I could be wrong).
>
>
> FYI. I don't know enough about zfs internals so can't say if this poster
> is right or not but I too have just used a disk partition as opposed to a
> zvolume for swap.
>

They have it right. FreeBSD tries hard (though isn't perfect in this
regard) to not allocate memory in the I/O path, especially if it knows the
request is for swap. It's possible to swap to zvols, and it actually works
when the memory pressures are light (since the stalls usually aren't too
long and usually complete). When memory pressure is high, though, these can
turn into deadlocks.

Warner