Re: ZFS ARC memory requirements
- In reply to: Andrea Venturoli : "ZFS ARC memory requirements"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Mon, 04 Mar 2024 14:42:57 UTC
On 3/4/2024 2:19 AM, Andrea Venturoli wrote: > Hello. > > I'm planning a machine to be used (almost) only as a backup host. > It will have a "system" SSD-based ZFS pool (possibly 1TB mirror) for > OS and programs and a large dedicated HDD-based ZRAID1 for backups > (initially 160-200 TB, but may grow in the future). > > my guess would be that I don't need that much RAM and 64 or even 32GiB > would be enough. > 64G should be fine. I have a backup server with 102TB across 3 raidZ1 vdevs, CPU E3-1230 v6 @ 3.50GHz with LSI MegaRAID SAS 9272-8i controller with JBOD disks on a SAS2 backplane. Its mostly a sink for backup data and occasionally I will restore something from backups. The RAM would help somewhat at restore time, but I get the sense it will be diminishing returns especially if you are restoring things one at a time or sequentially. If I read a big file (that is not in cache) and dump it to /dev/null I get acceptable (for me) throughput dd if=netflow.2024.02.07.20.00 bs=1m of=/dev/null status=progress 18360565760 bytes (18 GB, 17 GiB) transferred 24.003s, 765 MB/s 17986+1 records in 17986+1 records out 18859928380 bytes transferred in 24.643516 secs (765309973 bytes/sec) I imagine spreading it across more vdevs and using faster HDDs will improve speeds as these are all average grade HDDs but the read speeds meet my needs for now. Here are my arc stats FWIW before and after that big read. ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 4.20 b Mutex Misses: 1.92 m Evict Skips: 136.33 k ARC Size: 84.76% 53.21 GiB Target Size: (Adaptive) 84.78% 53.22 GiB Min Size (Hard Limit): 3.17% 1.99 GiB Max Size (High Water): 31:1 62.78 GiB Compressed Data Size: 49.37 GiB Decompressed Data Size: 54.13 GiB Compression Factor: 1.10 ARC Size Breakdown: Recently Used Cache Size: 52.50% 27.94 GiB Frequently Used Cache Size: 47.50% 25.28 GiB ARC Hash Breakdown: Elements Max: 11.21 m Elements Current: 65.86% 7.38 m Collisions: 2.69 b Chain Max: 12 Chains: 1.85 m ------------------------------------------------------------------------ ARC Efficiency: 60.98 b Cache Hit Ratio: 95.92% 58.49 b Cache Miss Ratio: 4.08% 2.49 b Actual Hit Ratio: 95.63% 58.31 b Data Demand Efficiency: 76.97% 1.60 b Data Prefetch Efficiency: 6.04% 2.20 b CACHE HITS BY CACHE LIST: Anonymously Used: 0.24% 141.86 m Most Recently Used: 6.58% 3.85 b Most Frequently Used: 93.13% 54.47 b Most Recently Used Ghost: 0.03% 17.37 m Most Frequently Used Ghost: 0.02% 12.78 m CACHE HITS BY DATA TYPE: Demand Data: 2.10% 1.23 b Prefetch Data: 0.23% 133.14 m Demand Metadata: 97.59% 57.08 b Prefetch Metadata: 0.08% 46.43 m CACHE MISSES BY DATA TYPE: Demand Data: 14.78% 368.09 m Prefetch Data: 83.12% 2.07 b Demand Metadata: 1.52% 37.87 m Prefetch Metadata: 0.58% 14.47 m --------------------------- ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 4.20 b Mutex Misses: 1.93 m Evict Skips: 136.40 k ARC Size: 84.79% 53.23 GiB Target Size: (Adaptive) 84.78% 53.22 GiB Min Size (Hard Limit): 3.17% 1.99 GiB Max Size (High Water): 31:1 62.78 GiB Compressed Data Size: 49.54 GiB Decompressed Data Size: 61.72 GiB Compression Factor: 1.25 ARC Size Breakdown: Recently Used Cache Size: 52.44% 27.91 GiB Frequently Used Cache Size: 47.56% 25.31 GiB ARC Hash Breakdown: Elements Max: 11.21 m Elements Current: 59.84% 6.71 m Collisions: 2.69 b Chain Max: 12 Chains: 1.60 m ------------------------------------------------------------------------ ARC Efficiency: 60.99 b Cache Hit Ratio: 95.92% 58.50 b Cache Miss Ratio: 4.08% 2.49 b Actual Hit Ratio: 95.63% 58.32 b Data Demand Efficiency: 76.97% 1.60 b Data Prefetch Efficiency: 6.04% 2.20 b CACHE HITS BY CACHE LIST: Anonymously Used: 0.24% 141.88 m Most Recently Used: 6.58% 3.85 b Most Frequently Used: 93.13% 54.48 b Most Recently Used Ghost: 0.03% 17.37 m Most Frequently Used Ghost: 0.02% 12.79 m CACHE HITS BY DATA TYPE: Demand Data: 2.10% 1.23 b Prefetch Data: 0.23% 133.16 m Demand Metadata: 97.59% 57.09 b Prefetch Metadata: 0.08% 46.43 m CACHE MISSES BY DATA TYPE: Demand Data: 14.78% 368.10 m Prefetch Data: 83.12% 2.07 b Demand Metadata: 1.52% 37.88 m Prefetch Metadata: 0.58% 14.47 m ---Mike