Odd behaviour of two identical ZFS servers mirroring via rsync
Date: Fri, 11 Nov 2022 17:42:44 UTC
I have two identical servers, called clustor2 and clustor-backup, each with a ZFS RAIDZ-1 pool containing 9 SAS hard disks plus one spare and two SSDs for the ZIL and ARC functions. clustor2 stores user data from a HPC while clustor2-backup uses rsync to mirrors all the data from clustor2 every 24 hours. However, the disk usage on the mirror server is considerably more than on the other server - attached is a screenshot showing the two servers side by side, with the mirror server on the right, and displaying the contents of the same subdirectory choen at random (named 'ratio_10.0' in this instance); as you can see, the sizes of the files within each of the folders are identical but 'du' reports very different space usages for each folder and 'zpool list' also reports a significant difference in ZFS pool size. I'm not sure if this is relevant but both servers have ZFS pools with no compression although lz4 compression is enabled on the ZFS filesystems & both run FreeBSD 11.3 with ZFS version 5. Perhaps using zfs send/receive instead of rsync for mirroring might solve this disparity? Thanks in advance for any suggestions, Andy