File System Performance on FreeBSD

[Bill Moran]

On 8/8/10 10:03:59 AM, Kiswono Prayogo wrote:
> Is there any justification for this benchmark?
>
> http://www.phoronix.com/scan.php?page=article&item=zfs_ext4_btrfs&num=2
> <http://www.phoronix.com/scan.php?page=article&item=zfs_ext4_btrfs&num=2>

Kind of hard to do much with that "benchmark"  First off:
* Does the author even know what he's doing?  All that article does is
   display the charts, then tell you what the chart says.  The author
   shows no understanding of what's going on.
* He's running the tests on a laptop.
* He had a single disk/partition, which was the same disk/partition that
   he was running the OS off.  The difference in speed might have been
   the result of different software being installed on the different OS
   that was competing for disk usage.
* All of his tests involve tiny amounts of data and/or extremely quick
   run times (less than 30s).  On a system with 4G of ram, the different
   caching policies on the different FS can have a huge difference on the
   results.  While it's interesting to study those caching differences,
   it's not anywhere indicative of overall FS performance.  Let him run
   one of those tests for 5 mins and see if the results are still the
   same.

But, most importantly, his benchmarks are useless for any productive
use.

He doesn't describe the tests he's doing with enough detail for anyone
else to attempt to reproduce them and attempt to address the problem.
What does he mean by "gzip test"?  Can I see the command line parameters
involved?  How many runs of each test did he do?  What other programs
were accessing the disk at the time?  What other programs were
_running_?  There's nothing wrong with PC-BSD, but it installs a lot
of stuff at install time -- there may be programs running that are
hurting the results that aren't running on Ubuntu.  Since that was a
laptop, what is the powersave policy for the disks in each case?  Did
he do a single run of each test?  That produces the most unreliable
results ever.

Overall, it's just sloppy reporting if you ask me.  For all I know,
he actually did a really good job of making sure that everything was
set up to be fair, but the article doesn't say that.  It's pretty
typical of most reporting, not enough depth or care to be useful.

I'm sure there are Linux people who will be shouting about this all
over the place.  But to the casual observer, all this tells you is
that Linux's filesystems _may_ be faster for short, bursty work.

To someone technical who might be looking to investigate the results
with an eye toward fixing them, it's useless.

-- 
Bill Moran