Peer review of AMD64/FreeBSD article

[Jem Matzan]

Oliver Fromme wrote:

>Jem Matzan <valour at thejemreport.com> wrote:
> > I've just finished writing this article comparing performance between an 
> > Athlon64 in 32-bit and 64-bit mode using FreeBSD:
> > 
> > http://www.thejemreport.com/lab64/amd64vsi386.php
> > 
> > (this is a temporary address which will later redirect to the published 
> > article)
> > 
> > I've checked it over twice for fact accuracy, but I would like other 
> > eyes to look at it before it goes to press. I haven't spell-checked it 
> > yet, so don't worry about that... I just want to make sure I haven't 
> > made any factual errors.
>
>I like the article very much.  Well done.  I also appre-
>ciate the fact that you refrained from spoiling the compa-
>rison with colorful graphics.  :-)
>
>There are just two things which seem a bit unclear to me.
>
>In the very first paragraph it sounds like hyperthreading
>would always be a performance win, but that's not the case.
>I've had applications that ran slightly faster when hyper-
>threading was turned off.  If I remember correctly, soft-
>ware that does many concurrent things and I/O benefits most
>from hyperthreading, while pure numbercrunching jobs run
>faster with hyperthreading switched off.  (I'm not saying
>that you should repeat all your benchmarks with hyper-
>threading off, mind you.  I just think that the remark in
>the first paragraph sounds a little bit misleading.  YMMV.)
>
>The second point is that the gcc "benchmark" seems a bit
>unfair for me, because you're really measuring _different_
>things when compiling something for i386 and for amd64.
>The compiler is producing different code, it has to opti-
>mize differently (particularly because of the different
>register sets of the processors), so you can't really
>compare the results.  Also take into account that the amd64
>code generation engine of gcc is rather new, while the i386
>code generation is very mature.  Apart from that, I would
>rather call this "benchmark" synthetic, because nobody buys
>an Opteron to compile things all day long.  Well, except
>for the FreeBSD package building people, maybe.  :-)
>
>In relation to that, the oggenc benchmark is certainly much
>more realistic.  It would have been nice to have some video
>decoding / encoding benchmarks, too (e.g. mplayer / menco-
>der, transcode, ffmpeg, whatever).
>
>Well, just my 2 cents.  :-)
>
>Regards
>   Oliver
>
>  
>
Hyper-Threading seemed to help with processes that didn't require a 
heavy CPU load. The OpenSSL tests show it being markedly faster in the 
smaller algorithms, but lagging way behind the 64-bit Athlon64 when the 
serious number crunching comes into play. Intel's press kit shows HT 
(and SSE3) giving an advantage when multitasking with four desktop 
programs in Windows XP. It's just too hard to show that reliably though. 
There's a lot of anecdotal evidence to suggest that AMD64 is faster on 
the desktop (in X) in 64-bit mode than the Prescott is in 32-bit, but 
I'm having trouble proving it.

As for the buildworld test, I knew that it would be an issue. I did my 
best to warn readers that it wasn't exactly a good point of comparison 
because of the variable that GCC presented, but it has its place because 
a lot of people spend a lot of time compiling (especially in FreeBSD) 
and they want to know how it performs compared to the others regardless 
of the reason why. It's not synthetic because it's a real-world 
scenario, but it's not as good as the oggenc and openssl tests. I hope 
to have more good tests for round two -- I'll try out the encoders you 
reccommended.

-Jem