Benchmarks: AMD64 vs i386 on Dual 246 Opteron

[uidzero]

ray at redshift.com wrote:

>Freebsd-AMD64 list:
>
>I recently completed benchmarking an evaluation server provided to us by our
>hardware vendor in order to see if switching our cluster from Xeon based
>machines to AMD based machines was worth while/cost effective
>
>The machine provided was a Dual Opteron 246 using the Tyan S2881 motherboard.
>It had 4GB or ram and included a single SATA hard drive.
>
>I initially loaded FreeBSD 5.4 AMD64 on the machine, recompiled the kernel, etc.
>and applied all the normal tweaks to apache, PHP, etc.  The machine, while
>faster than our single 2.4 Ghz Xeon's, wasn't all that much faster (maybe only
>10 to 15 percent).  
>
>After speaking with AMD and doing further benchmarks, I was about to give up on
>AMD and return the machine.  However, at the last minute, an engineer from AMD
>suggested that perhaps loading the 32 bit version of FreeBSD (aka i386) might
>improve performance, since it was possible that the overhead from 64 bit
>pointers was causing the machine to run slower than expected.  He also explained
>that the AMD should be running about 3 to 4 times faster than the single Xeon.
>
>While this sounded like a long shot, I loaded FreeBSD 5.4 i386 on the machine
>and after applying the exact same configuration to the OS, Apache, PHP and
>MySQL, re-ran the benchmarks.  Much to my surprise, just changing the OS from 64
>bit to 32 bit caused the machine to double in speed.  The results are attached
>in an Excel spreadsheet.  So the exact same machine, running the identical
>configuration, performed roughly twice as fast when running FreeBSD 5.4 i386 vs
>FreeBSD 5.4 AMD64.  Something about this seems so wrong to me :-)
>
>In speaking with one person off the list here, I was told that the FreeBSD AMD64
>branch has actually been cleaned up substantially over the i386 code.  So
>naturally I was expecting much better performance from a 64 bit machine running
>the AMD64 code than the "older" i386 code.  I had also originally expected that
>since this branch [the AMD64 branch] was specifically built around the AMD
>CPU's, that it should run the best (and thus the fastest) on the AMD opteron
>CPU's.  However, just the contrary turned out to be the case in benchmarking.
>
>I'm wondering if anyone has any comments or thoughts on this?  The attached
>benchmarks show transactions per second across localhost using Apache AB on the
>same machine.  The first tests are with plain text files from 64 bytes to 50K in
>size.  The next group is using a small and medium size PHP program.  The final
>set relate to MySQL inserts, selects and updates.  As you can see from the data,
>the exact same machine runs about twice as fast just by switching the OS from
>AMD64 to i386.  But why?
>
>The only answer I have so far as to why this may be the case is that perhaps
>i386 uses 32 bit pointers which the CPU(s) can handle faster (thus less overhead
>for the CPU).  But it still seems odd to me that if FreeBSD AMD64 is written
>specifically for the 64 bit CPU, why doesn't the machine perform better when
>running it?
>
>I'm also wondering if there is a compiler switch on AMD64 that could be used
>(perhaps in /etc/make.conf or something) that would force the AMD64 version to
>run in 32 bit mode only - since that would be an interesting comparison as well.
>
>I welcome any/all comments, suggestions, insight. 
>
>Thanks.
>
>Ray
>  
>

Good morning. I'm very NEW to the list and NEW to 64bit systems. I 
installed 5.4-R (i386) on my dual AMD64 Opteron 2.0g (1mb cache) with 1g 
ram and 4 X 160 (raid10) sata drives server. I was blazing fast, I think 
the first kernel recompile was 10 minutes or so "time make buildkernel 
KERNCONF=KERNEL" I was shocked to see how fast it was. (I know you 
tested with php/etc...) Well, like an idiot, I was thinking I could use 
the i386 install disc to get the 64bit. Eh.. no go. I then grabbed the 
boot only 64bit from FreeBSD's ftp site and loaded it. Base install of 
course. When I did the "first compile" I yet again. was surprised. 
6:29:xx!!! Fastest I've ever seen a kernel compile and that was with ONE 
cpu. (Had to compile in SMP.) Man, never had a system that was this fast.

Needless to say, I think the 64bit out performs the 32bit OSes. Then 
again, I'm not as technical as most of you all are, I'm just chimming in 
from a "different" side.

Please, put me in my place if need be. :)

Michael