strange disk activity

[Dan Nelson]

In the last episode (Feb 20), Boris Samorodov said:
> On Sat, 20 Feb 2010 14:26:15 -0600 Dan Nelson wrote:
> > In the last episode (Feb 20), Boris Samorodov said:
> > > Hello List,
> > > 
> > > I've got a very strange disk activity:
> > > -----
> > > % iostat -xw60 da0
> > >                         extended device statistics  
> > > device     r/s   w/s    kr/s    kw/s wait svc_t  b  
> > > da0       43.2 204.4   971.9 10917.2    0  30.7  30 
> > >                         extended device statistics  
> > > device     r/s   w/s    kr/s    kw/s wait svc_t  b  
> > > da0        5.1 274.6    72.7 15206.2    0  50.4  27 
> > > 
> > > If I'm not mistaken it's approx. 15Mb/sec. Which means more than 1000 Tb a
> > > day!
> > > 
> > > How can I find which program is trashing the disk? The system is
> > > FreeBSD-7.0 with 15 jails.  Thanks for your help.
> 
> > Something like this would be a good start:
> 
> >  dtrace -n 'syscall::write:entry { @dist[pid,execname] = sum(arg2); }'
> 
> > Let it run a few seconds, then hit ^C, and it will print the total bytes
> > written by each process during that period.  You may also have to trace
> > writev and pwrite, since FreeBSD's dtrace doesn't include the sysinfo
> > provider that would let you count them all at once.
> 
> > Data written to swap or mmap'ped files won't register here, but most
> > heavy writes aren't done with mmap.
> 
> Thanks, Dan! Dtrace is really a way to go.
> 
> Unfortunately dtrace is not available at 7.0. That means that
> the system should be updated to 7-STABLE.
> 
> Are there other possibilities meanwhile?

  ktrace -d -i -p 0 ; sleep 10 ; ktrace -C 

, then run kdump -m64 and search for large numbers of writes in the output. 
"-d -i -p0" selects all current and future children of pid 0, which will
trace all processes.  "-m64" limits the I/O dump size to 64 bytes.

-- 
	Dan Nelson
	dnelson at allantgroup.com