buildworld releng7 exterme performance loss

[Frank Staals]

doug at safeport.com wrote:
> On Wed, 6 Feb 2008, Jerry McAllister wrote:
>> Sounds like the system is spending a lot of time waiting for some
>> resource - I don't know which one[s].
>>
>> Have you perused the system logs? Can you be running out of memory
>> or swap space? Is the system built with debugging turned on?
>>
>> I'm not sure what else to suggest.
>> ////jerry
>>
> if you have not already done so turn on all.log (see etc/syslog.conf)
>
<message snipped>

As suggested I turned on all.log and also captured the output from 
vmstat -i and top -S as 2 others suggested. From the all.log output I 
don't seem to see anything out of the ordinary if I compare it to a 
snapshot of all.log on my working environment but maybe I am looking for 
the wrong thing ? The same applies for the output of top -S. vmstat's 
output may be more interesting though. In both cases I the information 
was taken just after boot with a login on tty0 and 1 remote session. 
vmstat shows the amount of interrupts for my network interfaces are a 
lot higher with todays build compared to my working env from 3 weeks 
ago. There is one difference though: I did a rm -rf /usr/obj/* through 
ssh on the todays build. Could that be the reason for the difference or 
did I find the source of the problem ?

info :

all.log
todays build : http://fstaals.net/junk/rena/all.log.2008.07.02.1140
working env: http://fstaals.net/junk/rena/all.log.stable

top -S output (links so the output is still properly formatted )
todays build: http://fstaals.net/junk/rena/top
working env: http://fstaals.net/junk/rena/top_stable

vmstat -i on todays build:
frank at Rena# cat /root/vmstat
interrupt total rate
irq1: atkbd0 610 2
irq9: acpi0 1 0
irq12: psm0 6 0
irq14: ata0 58 0
irq17: wpi0 bge0+ 151862 706
irq19: cbb0+ 2 0
irq20: uhci0 uhci+ 1 0
irq21: pcm0 uhci1+ 8 0
irq22: ehci0 uhci4 5 0
cpu0: timer 179565 835
cpu1: timer 169770 789
Total 501888 2334

frank at Rena# cat /root/vmstat_stable
interrupt total rate
irq1: atkbd0 130 1
irq9: acpi0 1 0
irq14: ata0 58 0
irq17: wpi0 bge0+ 3544 44
irq19: cbb0+ 2 0
irq20: uhci0 uhci+ 1 0
irq21: pcm0 uhci1+ 7 0
irq22: ehci0 uhci4 5 0
cpu0: timer 155701 1946
cpu1: timer 145928 1824
Total 305377 3817



-- 
-Frank Staals