Performance comparison, ULE vs 4BSD and AMD64 vs i386

[Kenneth Culver]

Quoting Jem Matzan <jem at thejemreport.com>:

> I've spent the last two weeks benchmarking an AMD64 machine with
> FreeBSD. I've acquired a decent amount of data that seems to show
> conclusively that a kernel using the ULE scheduler has slower compile
> times and inferior performance to a kernel running the 4BSD scheduler.
> The difference is roughly 11% in some cases, although I have not yet run
> all of the numbers through ministat yet. This is consistent for both
> i386 and AMD64.
>
> I tried both 5.2-RELEASE and 5.2.1-RC2 and the performance results were
> identical for this system.
>
> I've also found that the AMD64 edition of FreeBSD is significantly
> slower than the i386 edition when doing a buildworld (with any amount of
> simultaneous makes), but ubench reports a slightly higher CPU score. I'm
> still trying to work with John McCalpin to get Stream to work properly
> for testing, and I have openssl speed tests that I have not yet
> compared.
>
> For testing I used the same make.conf and rc.conf for both systems and
> eliminated every possible option from the kernel that I could, without
> breaking anything. I ran all tests in single user mode with none of the
> usual background process (network, usbd, fsck, etc.) running. The kernel
> config files were as identical as they could possibly be.
>
> I collected three sets of data per run, each being performed immediately
> after a fresh restart, and for the buildworld tests I did three sets of
> those three sets, for a total of nine per scheduler.
>
> I'm willing to type in all of my data on a temporary page before I
> finish testing everything (I have another system to test before the
> review can be written) for the purpose of peer review, but I don't want
> to go to the trouble if no one here has a reason to dispute my findings.
> I thought that ULE was supposed to offer superior performance -- or at
> least, that's the word around the Internet -- but I have yet to discover
> that through my experience. Can anyone offer any possible explanation
> for ULE performing worse than 4BSD? How about AMD64 being slower than
> i386 on the same hardware? By slower, I mean a buildworld -j4 took about
> 400 seconds longer in AMD64 mode.
>
The buildworld problem could just be because it takes longer for the compiler to
generate amd64 code. In fact, I'm almost willing to bet that's the case since
amd64 has 2x the GPR's that x86 does. It's likely that it's harder to optimize
for it. Does anyone who knows compilers care to comment?

Ken