Big Problem

[Kathy Quinlan]

Hi Guys and Gals,

First off I am not a troll, this is a serious email. I can not go into 
to many fine points as I am bound by an NDA.

The problem:

I need to hold a text file in ram, the text file in the forseable future 
could be up to 10TB in size.

My Options:

Design a computer (probably multiple AMD 64's) to handle 10TB of memory 
(+ a few extra Gb of ram for system overhead) and hold the file in one 
physical computer system.

Build a server farm and have each server hold a portion eg 4GB each 
Server (250 servers (plus a few extra for system overhead)

The reason the file needs to be in ram is that I need speed of search 
for paterns in the data (less than 1 second to pull out relevent chunks)

I am sure I have missed some options, right now I am just kicking ideas 
around, the software will be based on FreeBSD with some major 
modifications to address the large amount of ram (probably set it up as 
a virtual drive with one file)

Regards,

Kat.