When you decided to use ZFS did you read up on the design of it? Most of the RAM goes to ARC which caches all the read files for you. Beside that you can tame the ARC with parameters and FreeBSD should shrink the ARC when more memory is needed.