%cpu in system - squid performance in FreeBSD 5.3

[Sean Chittenden]

>   PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
> COMMAND
>   474 squid     96    0 68276K 62480K select 0  53:38 16.80% 16.80%
> squid
>   311 bind      20    0 10628K  6016K kserel 0  12:28  0.00%  0.00%
> named
>
> It's actually so good that one machine can now handle all traffic
> (around 180 Mb/s) at < %50 cpu utilization.  Seems like something in 
> the
> network stack is responsible for the high %system cpu util...

OH!!!!  Wow, I should've noticed that earlier.  Your hint about someone 
on Linux having the same problem tipped me off to look at the process 
state again.  Anyway, you nearly answered your own question, save you 
probably aren't familiar with select(2)'s lack of scalability.  Read 
these:

http://www.kegel.com/c10k.html


Specifically the four methods mentioned here:

http://www.kegel.com/c10k.html#nb.select


Then look at the benchmarks done using libevent(3):

http://www.monkey.org/~provos/libevent/


Dime to dollar you're spending all of your time copying file descriptor 
arrays in and out of the kernel because squid uses select(2) instead of 
kqueue(2).  Might be an interesting project for someone to take up to 
convert that to kqueue(2).  Until then, any local TCP load balancer 
that uses kqueue(2) would also solve your problem (I'm not aware of any 
off the top of my head... pound(8) does, but it is only used for HTTP 
and is not a reverse proxy) and would likely prevent you from having 
your problems.   -sc

-- 
Sean Chittenden