Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

[Matthew Tippett]

Bottom post this time to follow Oliver :).

On 12/20/2011 02:54 PM, O. Hartmann wrote:
> On 12/20/11 22:45, Samuel J. Greear wrote:
>> http://www.osnews.com/story/25334/DragonFly_BSD_MP_Performance_Significantly_Improved
>>
>> PostgreSQL tests, see the linked PDF for #'s on FreeBSD, DragonFly, Linux
>> and Solaris. Steps to reproduce these benchmarks provided.
>>
>> Sam
There are still possible issues with those benchmarks.  The Xeon has 
known problems scaling from 6 to 12 cores (well enabling the 
hyperthreading), so you may find that some platforms are penalized in 
performance if HT is turned on.  See the scaling that Phoronix has done in

http://openbenchmarking.org/result/1112166-AR-1112153AR03

Most systems are good with scaling on real cores, the hyperthreading 
(and for that matter the Bulldozer thread affinity) can really break 
performance.   Different platforms have different behaviours.  
Benchmarking is a mucky business..

Note that the benchmarks with Phoronix test suite are repeatable, once 
installed, you can just run "./phoronix-test-suite benchmark 
1112113-AR-ORACLELIN37" to repeat (as close as the system allows) the 
benchmarks that started this thread.
>> Is the postgresql benchmark the only way to benchmark?
pgbench is already included in the Phoronix Test Suite (at least 9.0.1 
TPC-B benchmark.

>>
>> Well, this inspires me to gather together all the benchmarks someone
>> could find. There were lots of compalins about FreeBSD's poor
>> performance with BIND - once a domain of FreeBSD. Network performance
>> seems also to be an issue if it comes to scalability.
>> It would be nice to see what portion of the raw CPU/GPU power the OS
>> (FreeBSD, Linux ...)  delivers to scientific applications.
>>
>> I only know some kind of benchmarks, BYTE UNIX benchmark, LINPACK test
>> ... Does someone know a site to look for a couple of benchmarks to test
>>
>> a) memory system
>> b) scalability (apart from pgbench)
>> c) network performance/throughput/network scalability
>> d) portion of CPU performance the system delivers for numerical
>> applications to the user apart from the system's own consumption
>> e) disk I/O performance and scalability
The majority of these benchmarks are already in Phoronix Test Suite.  
There is monitoring capability (temp, load, CPU states, etc).  The 
question is the mapping from system attribute to benchmark, as well as 
determine what the ambigious terms mean (scaling can mean on increasing 
workloads, as memory is increased, as cpus are increased).

>>
>> it would also be nice to discuss some nice settings and performance
>> tunings for FreeBSD for several scenarios. I guess, starting developing
>> benchmarking test scenarios for several purposes would lead faster to
>> real numbers and non polemic than weird discussions ...
>>
This is what Michael and I are wanting to see.  Adrian Chadd has 
offerered to help facilitate within the FreeBSD community.  As mentioned 
before, what I'd like to see is

     1) Recommendations for more rounded benchmarks from the FreeBSD 
perspective
     2) Tuning guide documented somewhere within the community
     3) Comparative results based on the communities testing.

All concrete, and all achievable.

Regards,

Matthew