storage selection for embedded devices

[Isaac Levy]

Hi all,

> Olivier Gautherot wrote:
>> Philip, Warner,
>>
>> On Tue, Oct 14, 2008 at 3:00 PM, M. Warner Losh <imp at bsdimp.com>  
>> wrote:
>>
>>> In message: <48F4C02F.1060407 at syx.ca>
>>>           Philip Mullis <philip.mullis at syx.ca> writes:
>>> : I was wondering if anyone has extended experience in this area  
>>> with
>>> : embedded devices.
>>> :
>>> : I have a fixed embedded image which runs happily out of a 1Gig  
>>> compact
>>> : flash card.
>>> :
>>> : However I have some applications that I want to install to my  
>>> device
>>> : that will perform writes alot to the cf.
>>>
>>> I've deployed CF cards into systems for a number of years (since
>>> 2000).  They are way more reliable than spinning media in the
>>> environments that I deploy my company's gear into.
>>>
>>> We have most of the CF dedicated to a read-only partition.  A small
>>> modification partition was also provided.
>>>
>>
>> I wonder if you're talking about the same thing (may be just me...)
>>
>> Philip, what frequency of writes are you talking about? I think this
>> is key to the discussion. Are you planning enough RAM to avoid swap?
>> Can your system count with a RAM disk and regular dump of the content
>> to FLASH? If this is the case, a USB stick should be a safe approach.
>>
>> The algorithm Sandisk is referring to enhances the statistical
>> lifespan by shuffling the cells and using spare ones when the main
>> array wears out (trial-and-error algorithm). The typical lifespan  
>> of a
>> cell is 100k write cycles: try to evaluate whether this is compatible
>> with the use you plan for your device.
>>
>>
>>
>>> I've also tried to wear out a CF part by writing to the part, both
>>> directly and through a filesystem, millions of times.  I was  
>>> unable to
>>> keep a machine running long enough to cause a failure (my mistake  
>>> was
>>> doing it in a lab where people liked to unplug things).
>>>
>>
>> The technology has surely evolved since I last dealt with it in an
>> industrial environment. However, I would not swear by the "millions  
>> of
>> times" as such: Sandisk is famous for leveling the writes over the
>> whole array. Now, if your partition is relatively empty, your device
>> will support more cycles. In any case, using 10% of the FLASH blocks
>> can surely lead you to the millions of cycles without problem.
>>
>>
>>
>>> : Ive read the sandisk wear leveling white paper, yet I also hear  
>>> many
>>> : people such as professional photographers swearing by the write  
>>> once
>>> : rule with cf cards.
>>>
>>
>> That's paranoia, especially with todays technologies.
>>
>>
>>

On Oct 14, 2008, at 1:50 PM, Philip Mullis wrote:

> In a non-busy environment I am looking at just over 100
> writes,consisting of files ranging from a few 100kbytes to a max of
> 40mbytes each.
>
> The calculated max writes that would ever be reached in one day  
> would be
> 8000. (Id love to leave these in memory and sync them of, however  
> there
> is not enough memory on the alix2c3 to do this in most scenarios).
>
> Ive got the whole system down to less than 80mb which gets loaded  
> into a
> read-only memory filesystem on boot leaving me with most of the cf  
> to be
> mounted in as rw.  I used freebsd 6.4 for the image.
>
> The Alix2c3 has 256megs Ram and is a geode 500mhz processor, it runs
> quite nicely, and eats less than 6watts most of the time :)
>

I just wanted to chime in with some metrics, (although lacking total  
context/details):

 From a presentation on the NanoBSD build scripts, on FreeBSD, here's  
this:

--
http://www.bsdcan.org/2006/papers/nanobsd.pdf
Page 12 of the slides states:

Counting flash writes
+ 200.000 writes, worst case:
   –Superblock updated 1/s = 55 hours lifetime
   –File written 1/m = 138 days lifetime
   –File written 1/h = 22 years lifetime
   –File written 1/d = 547 years lifetime

--
Now, the details of the writes is not clear here, but the point is  
clear- writes to CF cards *must* be minimized.  After working with  
Soekris boards for years, and now PCEngines, I can testify that  
typical UNIX installs on CF cards, (lots of writes), wrecks the cards  
fast.

CF cards do funky wear-leveling stuff, so tricks I've seen people try  
which move files around simply exacerbate the problem...

--
Here's a general approach to CF as boot drive use, (and the purpose of  
the NanoBSD build scripts on FreeBSD):

1) obviously: Tiny Kernel, userland, etc... stripped down system to  
taste/sanity

2) 4 notable partitions on CF media:
   - boot sector
   - 1st half, read-only root and userland
   - 2nd half, read-only root and userland
   - a small partition to mount as needed and write config files to  
(is unmounted unless a config needs to be written)
For system updates, build a new image, and dd it to the second  
partition- reset bootloader to boot from second partition, and  
viola... minimal writes.  This is very flexible, can be piped over  
ssh, etc...

3) A couple of memory disks for /etc /var and /tmp, (5mb each is Nano  
default)- /etc gets the files from the small r/w partition above.


That's just a quick conceptual overview of one successful method of  
running UNIX on CF cards- hope it's helpful!  More info specifically  
on NanoBSD can be found here:
http://www.freebsd.org/doc/en/articles/nanobsd/index.html

I'd love to know what similar systems exist for other UNIX systems if  
anybody has done anything interesting?!  (esp. any noteworthy PicoBSD  
differences)

Best,
.ike