HEADS UP: ZFS v28 merged to 8-STABLE

[David Magda]

On Jun 10, 2011, at 17:24, Bob Friesenhahn wrote:

> Dedup can require a huge amount of RAM, or a dedicated L2ARC SSD, depending on the size of your storage.  You should not enable it unless you are prepared for the consequences.

Under OpenSolaris, each tracking entry for a deduped block (which can be between 512B to 128KB) can be up to 376 bytes (struct ddt_entry): so for one 1 GB (10^9) of deduped data (244140 blocks at 4K), you would need ~91MB of overhead to keep track of it; for 1 TB (10^12) of deduped data  you would need ~91 GB of space to keep track of all the blocks. And if you can't fit the DDT in RAM, it will have to be saved to disk, which means more I/O to fetch the data.

http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h?rev=1.2

If your data is in blocks smaller than 4K you'll need more memory for the DDT; if the data is broken up into blocks larger than 4K you'll probably need less.

Also remember that even though an L2ARC cache may save you from having to go to spinning rust, you still need to use some RAM (struct arc_buf_hdr; ~178B) to reference the DDT stuff in L2ARC:

http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c

A few threads on zfs-dicuss on this:

http://mail.opensolaris.org/pipermail/zfs-discuss/2011-April/thread.html#48026
http://mail.opensolaris.org/pipermail/zfs-discuss/2011-May/thread.html#48185

Also, the above numbers are for OpenSolaris: someone may want to check the structure sizes for FreeBSD to be sure. They should get you in the right ballpark though.