gjournal +UFS - anyone actually use it?

[Mark Saad]

On Mar 6, 2013, at 5:55 AM, Wojciech Puchar <wojtek at wojtek.tensor.gdynia.pl> wrote:

>> I was using it to store large MySQL myisam tables , speed was acceptable at the time .  I never had any fs corruption and it worked as expected .
>> 
>> At the time I set it up I remember there was some chatter about how slow gjournal was compared to ufs with softupdate .
> 
> did some tests yesterday on 25GB partition.
> 
> i simulated journal on SSD using 2GB malloc backed ramdisk (md0). UFS partition mounted async as gjournal recommends.
> 
> test: unpacking kernel sources to multiple directories until disk gets full
> 
> simulates write heavy I/O.
> 
> gjournal+SATA drive (seagate constellation ES 500GB) with write cache disabled - 14 minutes 20 seconds.
> 
> write cache enabled - 14 minutes 5 seconds (nearly no difference).
> 
> UFS+journalled softupdates, no gjournal, disk write cache on - 26 minutes 44 seconds. disk write cache off - was too lazy to wait after an hour.
> 
> With gjournal it is not only faster, but it doesn't make other I/O activity crawling.
> 

Interesting I will have to try this; can you post the exact test steps . Also what type of controller were you using and what kernel / version .

> 
>> 
>> Fast forward to today I almost always use ufs with softupdate journal , new in FreeBSD 9.0 and available as a patch to 6.x, 7.x , and 8.x
> 
> The problem is as follows:
> 
> SU+J makes sure that metadata will get consistent. NOT DATA. And this is quite a mess if you get UPS failure under high load.
> 
> gjournal does journal everything.
> 

Not exactly, ufs mounted  with default options insures data is written sync and metadata asynchronous . Standard Softupdate (no journal) improves upon this by limiting what ops need to write to the disk. It had some short falls  for edge case operations; which softupdate journal resolved by journaling the metadata ops that were not protected / covered by standard softupdate .  

See 
http://jeffr-tech.livejournal.com/24357.html



>> This is better supported now , as more people use it in new 9.x builds  .
>> 
>>> i think about journal on SSD.
>> 
>> I believe this is only and option in geom journal ,
> 
> SSDs are not expensive today. i can get 128GB SSD and create 20GB journal just to limit wear. and possibly use the rest of SSD to store read-intensive data.
> 

I wonder if how trim / no trim effects the journal wear . 

> the way gjournal writes to journal device (sequential) is very friendly for SSD too.
> 
> Small SLC-based SSD would be best but i don't see anything on the market with acceptable price for now.
> 
>> I am not sure if you can relocate a suj journal to an alternate disk.
> no you can't. But still you will end with consistent metadata but not consistent data. I recovered it but still it took a time and lots of checking.
> 
> 
> 
> 
> gjournal doesn't seem to be elegant in case of journal failure (i simulated it with forced removal of ramdisk with mdconfig).
> 
> TONS of messages in logs, but still - no data loss, just you have to shutdown system, boot from pendrive, remove journal, fsck (just for sure), and then add journal again


I would be careful of using the md for the journal . Something makes me think it will play nicer when you remove that then a real failure .  Try a USB stick for the journal; and pull it out while running your test case.  That to me seams evil enough to break this setup . 

Let me know what happens .

Also when testing su+j I ran the following test case . Extract ports via portsnap extract , build world with -j4 . Let the box warm up the yank the power and then boot the box back up and see what happens . 
--
Mark saad | mark.saad at longcount.org