Are hardware vendors starting to bail on FreeBSD ... ?

Thu Jul 13 15:56:24 UTC 2006

--- Nick Withers <nick at nickwithers.com> wrote:

> On Thu, 13 Jul 2006 08:22:03 -0700 (PDT)
> Danial Thom <danial_thom at yahoo.com> wrote:
> 
> > 
> > --- Head in the sand Jerry mumbled:
> 
> Just thought I should metion that this comes
> across as rude to
> me... but maybe that's just me!
> 
> > > > --- Francisco Reyes
> <lists at stringsutils.com>
> > > > wrote:
> > > > 
> > > > > Marc G. Fournier writes:
> > > > > 
> > > > > > the problem is that none of the Tier
> 1
> > > > > hardware manufacturer's support 
> > > > > > FreeBSD, and a growing number of
> places
> > > (ie.
> > > > > Adaptec / Intel) appear to be 
> > > > > > dropping support for it as well ...
> > > > > 
> > > > > But companies like 3Ware and Areca are
> > > > > supporting it and from what I see on 
> > > > > the lists, people are voting with their
> > > money
> > > > > in their favor.
> > > > 
> > > > Mainly because they had drivers that
> required
> > > > little modification from previous
> versions.
> > > Intel
> > > > has a few other things on their plate,
> > > releasing
> > > > more processors to bail out Freebsd's
> paltry
> > > > performance, so give them a break.
> > > > 
> > > > How long are vendors supposed to wait for
> the
> > > > FreeBSD developers to deliver the
> performance
> > > > they've claimed that they can deliver? I
> know
> > > > several network appliance vendors all
> stuck
> > > on
> > > > FreeBSD 4, because 5 and 6 are a step
> > > backwards
> > > > performance-wise. Now they're saying 7
> will
> > > be
> > > > the one. 
> > > > 
> > > > FreeBSD is the OS that cried "WOLF", and
> the
> > > > vendors are starting to ignore the calls.
> The
> > > > infrastructure is so poor (in terms of
> > > process
> > > > switching times and scheduler
> efficiencies),
> > > and
> > > > they seem clueless on how to fix it.
> > > 
> > > Must be a troll.
> > > FreeBSD performance is not what holds it
> back.
> > > It competes well with others out there.
> > > 
> > > ////jerry
> > 
> > No it doesn't, Jerry. Even Robert Watson, who
> > spends most of his time on performance
> issues,
> > readily admits that 
> > 
> > - FreeBSD 6 is faster with 1 processor than 2
> > - FreeBSD 6 is slower with 1 processor than
> > Freebsd 4.x
> 
> Would you mind providing a source for that
> information? I would
> not be at all surprised to hear that a FreeBSD
> 6.x
> dual-CPU set-up provides less than twice the
> performance as that
> of a single CPU FreeBSD 6.x set-up, but I will
> happily eat my
> own (mighty tasty) hat if a dual CPU FreeBSD
> 6.x set-up
> performs worse than a single FreeBSD 6.x
> set-up. That having
> been said, I tend to treat Robert Watson's word
> as gospel, but
> I'd like to see it in a form I can trust
> (honestly, no offense
> intended!) first (i.e., please provide a source
> for your
> information :-)).
> 
> > The process switch times are 2-4x slower than
> on
> > linux. Thats not 2-4%, thats 200-400% slower.
> 
> 
> Could you provide me with a source here (Not
> trying to be rude,
> but I'd be really interested in reading about
> this)?
> 
> > Simply enabling SMP on a single processor
> system
> > adds 20-25% overhead in freebsd 6.1.
> 
> Whilst I have trouble accepting these
> particular figures, I
> don't doubt that there is *some* overhead in
> dealing with
> multiple CPUs, from a kernel perspective.
> 
> > Again, readily admitted/accepted by the
> developers.
> > There is no way to recover that in
> efficiency, at
> > least not for a long time.
> > 
> > What's really frightening is that Dragonfly
> is
> > going to shed the giant lock before Freebsd,
> and
> > there's only one guy working on it.
> 
> Please see
> "http://www.dragonflybsd.org/about/team.cgi".
> My
> maths ain't great (alright, it's terrible!) but
> I count more
> than one committer. I'm probably just
> misunderstanding what
> you're trying to say here...
> 
> > Its prima facie evidence that IQ isn't
> cumulative.
> > 
> > DT
> 
> Sorry if this appears stand-off-ish - I don't
> mean it do be! I
> do have a bias in favour of what I see as the
> best OS ever,
> though (better that MacOS 7.5.3, even! :-))

Robert Watson's own test, on freebsd-performance:

"I'll run some more diverse tests today, such as
raw bandwidth tests, pps on 
UDP, and so on, and see where things sit.  The
reduced overhead should be 
measurable in cases where the test is CPU-bound
and there's no clear benefit 
to more accurate timing, such as with TCP, but it
would be good to confirm 
that. 

Robert N M Watson 
Computer Laboratory 
University of Cambridge 

peppercorn:~/tmp/netperf/hz> ministat *SMP 
x hz.SMP 
+ vendor.SMP 
+--------------------------------------------------------------------------+

|xx x xx   x       xx  x     +              +   +
 +   +    ++ +         ++| 
|  |_______A________|                    
|_____________A___M________|     | 
+--------------------------------------------------------------------------+

     N           Min           Max        Median 
         Avg        Stddev 
x  10         13715         13793         13750  
    13751.1     29.319883 
+  10         13813         13970         13921  
    13906.5     47.551726 
Difference at 95.0% confidence 
         155.4 +/- 37.1159 
         1.13009% +/- 0.269913% 
         (Student's t, pooled s = 39.502) 

peppercorn:~/tmp/netperf/hz> ministat *UP 
x hz.UP 
+ vendor.UP 
+--------------------------------------------------------------------------+

|x           x xx   x      xx+   ++x+   ++  * +  
 +                      +| 
|        
|_________M_A_______|___|______M_A____________|  
               | 
+--------------------------------------------------------------------------+

     N           Min           Max        Median 
         Avg        Stddev 
x  10         14067         14178         14116  
    14121.2     31.279386 
+  10         14141         14257         14170  
    14175.9     33.248058 
Difference at 95.0% confidence 
         54.7 +/- 30.329 
         0.387361% +/- 0.214776% 
         (Student's t, pooled s = 32.2787) "

As you can see, UP has a higher performance than
SMP.

Again, from FreeBSD-performance. I did misquote,
it seems there is a 50% overhead:

Kip Macy (freebsd-performance) on FreeBSD running
on Sun:

"To make it easier for future respondents to stay
on topic let me 
explain my situation. I have ported FreeBSD to
Sun's new UltraSPARC 
architecture, sun4v. The current implementation,
the T1, has 6-8 cores 
with 4 threads per core. Unlike HTT on x86, these
machines actually 
have ample memory bandwith ~26GB/s so threading
can actually be 
useful. On my 32-cpu system benchmarks like
supersmack max out at 9 
threads - i.e. one can't get the system below 70%
idle. Across the 
board context switches on solaris/T1 take 2x as
long as they do on 
linux/T1. Because of lock contention FreeBSD in
turn takes between 10% 
- 100% longer than Solaris to context switch. 

I would like to be able to tout FreeBSD as a
strong competitor on the 
sun4v architecture. At the moment I can't.
Perhaps this isn't the 
right forum for discussing my concerns - a
freebsd-scalability list 
might be in order. "

He's basically saying that the infrastructure of
freebsd, context switching, lock contention, etc,
just make the current state of FreeBSD impossible
to recommend. There's no reason to assume its
different on any platform.

"Q: Running a simple test with a traffic
generator (firing udp packets to a 
> blackhole), the system overhead with a single
processor goes up from 10% to 
> 15% when running a kernel with SMP enabled (and
nothing else different). I 
> have ITR set to 6000 interrupts per second.
That seems like an awful lot of 
> overhead. Is there some problem running an
SMP-enabled kernel when only 1 
> processor is present, or is there really 50%
extra overhead on an SMP 
> scheduler? I'll have a dual core in a few days
to test with. 

Robert watson writes:

I don't know about the particular number, but
there is a significant overhead 
to building in SMP support currently -- in
particular, you pick up a lot of 
atomic instructions which increases the cost of
locking operations even 
without contention.  Some of that overhead
reduces as the workload goes up, as 
there's coalescing of work under locked regions,
reduced context switch rates 
as work is performed in batches, etc.  There is
currently extremely active 
work in the area of reducing the overhead of
scheduling and context switching, 
being driven in part by the 32-processor support
in Sun4v.  I don't expect to 
see large portions of that merged to RELENG_6,
but it will be in RELENG_7. 
Again, not my area of expertise, but there is
work going on in this area. "

Again, he's basically saying "yes, we know
FreeBSD currently sucks, but it will be better in
v7" *yawn*

Its well-known that threading the kernel is going
to add overhead, so a UP threaded kernel is
always going to be slower than a non-threaded
kernel. The reasoning, of course, it that the
benefits of MP will make it worthwhile. NOT the
case in Freebsd, at least not at this point in
time.

DT

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 