Re: Wired Memory Increasing about 500MBytes per day

03.08.2021 16:47, Mark Johnston пишет:
>> We noticed the same problem, I'm not sure the exact version, but you can
>> check the output:
>> # vmstat -z | egrep "ITEM|pgcache"
>>
>> The page cache grows until lowmem is not reached. Then it automatically
>> cleans and begins to grow again.
> 
> The pgcache zones simply provide a per-CPU cache and allocator for
> physical page frames.  The sizes of the caches are bounded.  The numbers
> of "used" items from the pgcache zones do not really tell you anything
> since those pages may be allocated for any number of purposes, including
> for other UMA zones.  For instance, if ZFS allocates a buffer page from
> its ABD UMA zone, and that zone's caches are empty, UMA may allocate a
> new slab using uma_small_alloc() -> vm_page_alloc() -> pgcache zone.
> 
> So if there is some wired page leak, the pgcache zones are probably not
> directly responsible.

We don't see any leaks, but our monitoring shows that "free" memory
migrates to "wired" and only these zones are grow. So, we have on the
graphs linear growing of wired memory over 7 days. When free memory
reaches ~4% all returns to normal, and then again linear growing for 7
days. And pgcache zones reset their number of USED items to low value.
This is on the server with 256G RAM.

E.g. This is when 9% of free memory left:

$ vmstat -z | egrep "ITEM|pgcache"
ITEM                   SIZE  LIMIT     USED     FREE      REQ
FAILSLEEP XDOMAIN
vm pgcache:            4096,      0,    5225,     139,  412976,   0,
0,   0
vm pgcache:            4096,      0,28381269,      77,190108006,  24,
0,   0
vm pgcache:            4096,      0,  166358,   11523,1684567513,3054,
 0,   0
vm pgcache:            4096,      0,29548679,     576,780034183,1730,
0,   0
$ bc
>>> 5225+28381269+166358+29548679
58101531
>>> 58101531*4096/1024/1024/1024
221
>>>

This is when lowmem triggered:
% vmstat -z | egrep "ITEM|pgcache"
ITEM                   SIZE  LIMIT     USED     FREE      REQ
FAILSLEEP XDOMAIN
vm pgcache:            4096,      0,    5336,     337,  410052,   0,
0,   0
vm pgcache:            4096,      0, 3126129,     117,56689945,  24,
0,   0
vm pgcache:            4096,      0,   49771,    3910,413657845,1828,
0,   0
vm pgcache:            4096,      0, 4249924,     706,224519238, 562,
0,   0
% bc
>>> 5336+3126129+49771+4249924
7431160
>>> 7431160*4096/1024/1024/1024
28
>>>

Look at the graph:
https://imgur.com/yhqK1p8.png

-- 
WBR, Andrey V. Elsukov