memory pool, rfc

Eduardo Morras nec556 at retena.com
Wed Oct 31 18:01:09 PDT 2007


Hello:

    I have some free time and want to do an memory pool. The idea is 
to have a memory zone of N KB (or several MB) compressed in memory. I 
have fast compression algorithms now that can release under BSD 
licence that are faster than hd i/o, so it take less 
compress/decompress a memory zone than read/write it to disk. I don't 
know if it already exist for FreeBSD, so if it's already done i'll 
try to improve it.

- Each memory chunk is compressed separately, so i can decompress and 
use one without decompress anyother one.
- In 4KB chunks of text i get 50-60 % compression (avg 2 - 1.6 KB result)
- In 4KB chunks of binary (application code) i get 30-40 % 
compression (avg 2.8 - 2.4 KB result)
- In both cases, results may vary depending on data type and chunk 
size, greater implies better compression
- Speed once implemented will be very fast. Current speed hogs a PATA 133 disk.

For what can be used?

- Memory pools in applications (like malloc)
- Ram disks
- Disk Cache (permit bigger disk cache)
- 'On the fly' filesystem compression (and it takes less read/write 
compressed data than non-compressed)
- Perhaps add it as Virtual Memory swap cache?
- Other

Don't point me to zlib or libbzip2, they are on another league and 
are much slower than my code.

Thanks



More information about the freebsd-hackers mailing list