How big can ZFS L2ARC grow?

[Karli Sjöberg]

Hey!

As the topic states, I´m wondering about the size of L2ARC and if there
is a limit to how big it is able to grow.

Why I´m asking is because I´ve always thought that if you add a cache
device to the pool, the maximum size of L2ARC would be the size of the
disk you´ve added, but recently I´ve come to know that´s not the case.

Here´s a 9.3-RELEASE system that has 64 GB RAM and two 256 GB large
SSD's added as cache, that I would´ve thought only could´ve grown to
about 512 GB:
# sysctl -n kstat.zfs.misc.arcstats.l2_size 
7696626233344

Another system running 9.2-RELEASE with 32 GB RAM + 240 GB SSD:
# sysctl -n kstat.zfs.misc.arcstats.l2_size
1400038980608

The servers are running a software for graphing so I have seen that the
size numbers can go up and down over time, but clearly goes over the
size of the SSD's that have been added.

We have two more systems configured with cache devices and yet another
two systems configured without.

The problem we have is that the four systems with cache devices (our
primary storage systems) goes completely unresponsive after different
periods of time, depending on how much RAM they have and the load they
´ve been under, I guess. The less RAM, the more frequent they stall, and
I´m starting to wonder if what´s common between them is L2ARC, because
the other two systems without cache devices doesn´t have those issues,
although they aren´t under the same kind of load either, it´s a disaster
recovery system receiving zfs snapshots and the other one is our syslog
server, but still...

What do you think, are the size numbers for L2ARC unusual, and could it
be related to the stalls we´ve been experiencing? And if the size
numbers really are unusual, is there a way to handle it, like limit how
large the L2ARC is able to grow somehow?



-- 

Med Vänliga Hälsningar

-------------------------------------------------------------------------------
Karli Sjöberg
Swedish University of Agricultural Sciences Box 7079 (Visiting Address
Kronåsvägen 8)
S-750 07 Uppsala, Sweden
Phone:  +46-(0)18-67 15 66
karli.sjoberg at slu.se