restore(8) to UFS on USB key: terrible slow

[Julian H. Stacey]

Matthias Apitz wrote:
> El día Thursday, December 08, 2011 a las 09:57:13AM -0600, Dan Nelson escribió:
> 
> > Cheap USB thumb drives aren't really optimized for small random-I/O writes. 
> > Can you try mounting the filesystem async?  that might help a little.  A
> > workaround would be to use mdconfig to create a block device (backed by
> > either swap or a file on your hard drive) the same size as your flash drive,
> > newfs and restore to that, then umount the filesystem and dd the raw image
> > directly to your flash drive.
> 
> Hello Dan,
> 
> Thanks for your hints. I tend to add that those USB thum drives aren't
> good for anything. I have a certain number of them containing each a
> complete bootable FreeBSD (including 'src', 'obj' and binary packages)
> to install my laptops and netbooks from them;
> 
> after some time these USB keys tend to loose data:
> files are corrupt a bit, dirs are missing and so on; that's
> why I wanted to make dump(8) nackups of them, to restore them from time
> to time; I will drop the idea and will just make dd(1) backups of the
> full /dev/da0;

Additional to all the other good points others wrote earlier,
may I mention: ...

I've found some sticks are slower than others.
Sometimes I do a performance & integrity test with my
	http://berklix.com/~jhs/src/bsd/jhs/bin/public/testblock

One (free promo) stick I found lies, see this comment in my
	http://berklix.com/~jhs/src/bsd/fixes/FreeBSD/src/jhs/etc/devd/jhs.conf
#       The end of it is write only memory !
#       It lets one write the last chunck,
#       It even lets one read that last chunk,
#       but the content of the last chunck is all zeroes.

Another 2G stick was particularly slow: (marked as) Sony,
	bought at a computer Sunday 'flea' market in Croydon England, 
	in retrospect I wonder if manufacturer Sony might have withdrawn them
	from sale, marked the batch for destruction, & possibly some criminal
	'liberated' them for resale again ?

	(Such things do happen, eg In Germany years back, pre USB
	era, CT Mag reported reported Betruger/Placebo cache chips.
	They were just ceramic with no silicon in, it was reported
	importers (in Munich I think) were afraid to sue chinese
	exporters, fear of Triads! maybe last bit was speculation,
	but wan't My speculation, I read it, whatever, can't remember
	more now)

Block Sizes:
	Maybe USB sticks may have different size/ speed front end
	cache chips on USB sticks ? Hans would know I suppose. ?

	Apart from soft updates, one can also choose the block sizes
	newfs creates, I recall FFS is larger than UFS ?.  

	Maybe we should send-pr some suggested size for man newfs
	if targeting images for USB sticks.
	(is that a question to consider jointly with fs@ list ? )

Voltages:
	I've  recently been bitten by appalling problems on a bunch
	of 2 of my externals discs, using 2 different laptops, 2/3
	hubs, & 3 power supplies.  Various combinations come back
	to bad voltage regulation, usually too low, some too high.

	But I assume discs will be more susceptible than sticks.
	
	However next time a motherboard fails for any of you, I
	suggest don't discard, first hacksaw off the double USB socket,
	solder wires across, add extra wires for a meter, so you
	can monitor voltage & current.

Mastering first on hard disc (per Dan's suggestion, mdconfig etc)
	is a good idea, I was considering this earlier when building
	a new stick/ extended Live-FS. .. using mdconfig etc,
	but it's heavy & slow after the initial image create, to keep
	rewriting, even if at large dd bs=

So I use incremental writes 
	I keep personal backups & bins & Live FS & mp3 to play etc
	all on USB sticks.  Still usable though 'cos I rarely change
	too much at one time. & rdist6 updates what's changed.
	(would also correct odd corruption Matthias)

	I even sue gbde encrypted FS (ie more performance degradation)
	.. and updates still happens acceptably if not exactly fast.
	Others could use rsync if they dont fancy rdist6.

Cheers,
Julian
-- 
Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com
 Reply below not above, cumulative like a play script, & indent with "> ".
 Format: Plain text. Not HTML, multipart/alternative, base64, quoted-printable.