CAM disk I/O starvation
David Wolfskill
david at catwhisker.org
Tue Apr 17 19:50:33 UTC 2012
On Tue, Apr 17, 2012 at 09:15:58PM +0200, Gary Jennejohn wrote:
> ...
> I still have the old problem kernel around, but it's probably not
> instrumented for any meaningful diagnoses.
> ...
Several months ago, I was running a set of meaurements (to determine how
performance for a certain task varied when I changed the hardware
configuration of the machine in question).
While it turned out that disk I/O was (surprisingly) not very
significant, I did find that extending the mechanism I had been using to
graph (aggregate) CPU utilization to graph the utilization of each core
was enlightening.
I don't know whether I can release the code or not -- I'll ask -- but
the basic idea is to look at the CPU state counters (they are an ordered
quintuple, for user, nice, system, interrupt, and idle) -- use the
sysctl OIDs idle kern.cp_time for the aggregate of all CPUs; use idle
kern.cp_times for an array of them, one quintuple per core. I graphed
them using stacked barcharts; I used math/R to handle the graphing.
Since CPU (or any other) utilization only makes sense over an interval,
you also need to choose one; I used 10-second intervals by default.
In any case, even under 7.1, I noticed that one of the cores got the
vast bulk of the interrupt processing. (I also saw some of the cores go
quite a bit more idle than others, which was fairly curious.)
Anyway: the point of the above rambling is that it isn't necessary
to actually "instrument" the kernel itself: the work I did was
deliberately designed to be able to run on an unmodified FreeBSD
system with no ports, packages, or other 3rd-party software installed
except for lang/perl. I also tried comparing running under
/usr/bin/time vs. running under my Perl script (which invokes
/usr/bin/time to get the getrusage() info) several times, and found no
statistically significant difference in resource usage -- even when I
reduced the sampling interval down to 1/second.
(A sufficiently motivated & talented individual could probably replace
the Perl script with a shell script. As it is, the Perl script
fork/execs a shell script to do the interval-sampling.)
Peace,
david
--
David H. Wolfskill david at catwhisker.org
Depriving a girl or boy of an opportunity for education is evil.
See http://www.catwhisker.org/~david/publickey.gpg for my public key.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20120417/a5112f10/attachment.pgp
More information about the freebsd-hackers
mailing list