Benchmarks: AMD64 vs i386 on Dual 246 Opteron

[Matthew Sullivan]

uidzero wrote:
> ray at redshift.com wrote:
> 
>> Freebsd-AMD64 list:
>>
>> I recently completed benchmarking an evaluation server provided to us 
>> by our
>> hardware vendor in order to see if switching our cluster from Xeon based
>> machines to AMD based machines was worth while/cost effective
>>
>> The machine provided was a Dual Opteron 246 using the Tyan S2881 
>> motherboard.
>> It had 4GB or ram and included a single SATA hard drive.
>>
>> I initially loaded FreeBSD 5.4 AMD64 on the machine, recompiled the 
>> kernel, etc.
>> and applied all the normal tweaks to apache, PHP, etc.  The machine, 
>> while
>> faster than our single 2.4 Ghz Xeon's, wasn't all that much faster 
>> (maybe only
>> 10 to 15 percent). 
>> After speaking with AMD and doing further benchmarks, I was about to 
>> give up on
>> AMD and return the machine.  However, at the last minute, an engineer 
>> from AMD
>> suggested that perhaps loading the 32 bit version of FreeBSD (aka 
>> i386) might
>> improve performance, since it was possible that the overhead from 64 bit
>> pointers was causing the machine to run slower than expected.  He also 
>> explained
>> that the AMD should be running about 3 to 4 times faster than the 
>> single Xeon.
>>
>> While this sounded like a long shot, I loaded FreeBSD 5.4 i386 on the 
>> machine
>> and after applying the exact same configuration to the OS, Apache, PHP 
>> and
>> MySQL, re-ran the benchmarks.  Much to my surprise, just changing the 
>> OS from 64
>> bit to 32 bit caused the machine to double in speed.  The results are 
>> attached
>> in an Excel spreadsheet.  So the exact same machine, running the 
>> identical
>> configuration, performed roughly twice as fast when running FreeBSD 
>> 5.4 i386 vs
>> FreeBSD 5.4 AMD64.  Something about this seems so wrong to me :-)
>>
>> In speaking with one person off the list here, I was told that the 
>> FreeBSD AMD64
>> branch has actually been cleaned up substantially over the i386 code.  So
>> naturally I was expecting much better performance from a 64 bit 
>> machine running
>> the AMD64 code than the "older" i386 code.  I had also originally 
>> expected that
>> since this branch [the AMD64 branch] was specifically built around the 
>> AMD
>> CPU's, that it should run the best (and thus the fastest) on the AMD 
>> opteron
>> CPU's.  However, just the contrary turned out to be the case in 
>> benchmarking.
>>
>> I'm wondering if anyone has any comments or thoughts on this?  The 
>> attached
>> benchmarks show transactions per second across localhost using Apache 
>> AB on the
>> same machine.  The first tests are with plain text files from 64 bytes 
>> to 50K in
>> size.  The next group is using a small and medium size PHP program.  
>> The final
>> set relate to MySQL inserts, selects and updates.  As you can see from 
>> the data,
>> the exact same machine runs about twice as fast just by switching the 
>> OS from
>> AMD64 to i386.  But why?
>>
>> The only answer I have so far as to why this may be the case is that 
>> perhaps
>> i386 uses 32 bit pointers which the CPU(s) can handle faster (thus 
>> less overhead
>> for the CPU).  But it still seems odd to me that if FreeBSD AMD64 is 
>> written
>> specifically for the 64 bit CPU, why doesn't the machine perform 
>> better when
>> running it?
>>
>> I'm also wondering if there is a compiler switch on AMD64 that could 
>> be used
>> (perhaps in /etc/make.conf or something) that would force the AMD64 
>> version to
>> run in 32 bit mode only - since that would be an interesting 
>> comparison as well.
>>
>> I welcome any/all comments, suggestions, insight.
>> Thanks.
>>
>> Ray
>>  
>>
> 
> Good morning. I'm very NEW to the list and NEW to 64bit systems. I 
> installed 5.4-R (i386) on my dual AMD64 Opteron 2.0g (1mb cache) with 1g 
> ram and 4 X 160 (raid10) sata drives server. I was blazing fast, I think 
> the first kernel recompile was 10 minutes or so "time make buildkernel 
> KERNCONF=KERNEL" I was shocked to see how fast it was. (I know you 
> tested with php/etc...) Well, like an idiot, I was thinking I could use 
> the i386 install disc to get the 64bit. Eh.. no go. I then grabbed the 
> boot only 64bit from FreeBSD's ftp site and loaded it. Base install of 
> course. When I did the "first compile" I yet again. was surprised. 
> 6:29:xx!!! Fastest I've ever seen a kernel compile and that was with ONE 
> cpu. (Had to compile in SMP.) Man, never had a system that was this fast.
> 
> Needless to say, I think the 64bit out performs the 32bit OSes. Then 
> again, I'm not as technical as most of you all are, I'm just chimming in 
> from a "different" side.
> 
> Please, put me in my place if need be. :)

This echos my experiences...


Further to that I have seen some amazing feats of sheer processing power - 
processing out Postfix queues of >5k messages (with inline virus checking) on a 
single AMD64 processor in under 5 minutes (most messages being delivered through 
a perl script).  Mailing lists of 500+ people getting getting processed _and_ 
delivered in under 20 seconds from first accept to last delivery (450+ users on 
the list were remote Internet users).  As a mail server with inline virus 
scanning I have no qualms with it receiving 50 messages per second - infact 
there is a 22M Uplink to the net, so far it has kept up with everything to the 
point of the 22M being full.

Hardware:

AMD64 2.0GHz
Asus K8S M/B
1G Ram
Dual 160G SATA (RAID1 config).

Regards,

-- 
Matthew Sullivan
IT Security Manager
The University of Canberra

A member of the Australian Association for the Abolition of
Acronym Abuse, Regional Group Headquarters, Strategic and
Tactical Operations Planning (AAAAARGHSTOP).