what is the best way to use free space on several boxes?

[Bakul Shah]

On Thu, 09 Dec 2010 16:14:12 +0200 Zeus V Panchenko <zeus at ibs.dn.ua>  wrote:
> Hi All,
> 
> we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror 
> UFS dedicated
> 
> but really it's used less than 30% each box ...
> 
> so, advice please, what is the best/correct/right way to use the free
> space on the boxes? nfs? zfs? iscsi?
> 
> the aim is to have storage place ...

This is probably more "researchy" than you want and this is
probably more suited to a much larger number of nodes but how
about some sort of a distributed hash table filesystem? Hash
a block address and map it to N different nodes.  That way
you can add and remove boxes without having to reconfigure.
When a block is initially written, its `address' is computed
from its contents.  One thing you have to worry about is that
the amount of available storage can change dynamically.

Lots of useful information in the wikipedia entry for this.

Even more researchy: even if you move such disks arround to
different nodes, or lose some disk blocks, the remaining data
on such disks can be reattached. A truly resilient storage
system.