netstat fails with memory allocation error and error in kvm_read

[Doug White]

On Fri, 10 Dec 2004, J. Martin Petersen wrote:

> We are trying to gather some debug information for a problem that is difficult
> to diagnose, as the machine always ends up hard frozen (does not do anything,
> can not break to debugger, does not respond to keyboard, etc.), so we are
> dumping output from netstat, vmstat, iostat etc. quite often.
>
> The cron jobs fail ever so often with error messages I do not quite understand,
> and I can not find anything relevant in the archives. Can anyone shed some light
> on this?
>
> Command:	netstat -r
> Error message:	netstat: kvm_read: Bad address
> Debug before:	http://www.aub.dk/~jmp/fw/debug-2004.12.09-21.34.41.gz
> Debug after:	http://www.aub.dk/~jmp/fw/debug-2004.12.09-21.35.06.gz
> # errors:	7
>
> Command:	netstat -an
> Error message:	netstat: sysctl: net.inet.udp.pcblist: Cannot allocate memory
> Debug before:	http://www.aub.dk/~jmp/fw/debug-2004.12.09-07.38.48.gz
> Debug after:	http://www.aub.dk/~jmp/fw/debug-2004.12.09-07.39.04.gz
> # errors:	3

You appear to be running out of kernel memory. Since you're capturing the
output of vmstat -m, you should check that for any bins that are growing
at a high rate of speed.

Seems possible that its in pf :)

>
> Command:	netstat -an
> Error message:	netstat: sysctl: net.inet.raw.pcblist: Cannot allocate memory
> Debug before:	http://www.aub.dk/~jmp/fw/debug-2004.12.10-22.19.50.gz
> Debug after:	http://www.aub.dk/~jmp/fw/debug-2004.12.10-22.20.10.gz
> # errors:	6
>
> The number of errors are in the last five days. Kernelconfig (FIREWALL), dmesg,
> pf.conf etc. can be found at http://www.aub.dk/~jmp/fw/
>
> The commands we are running are:
> 	netstat -i, netstat -m, netstat -s, netstat -an, netstat -r
> 	vmstat, vmstat -i, vmstat -f, vmstat -m, vmstat -z
> 	iostat
> 	ps waux
> every 15 seconds. We are running the same commands (plus pfctl -vvsa) from
> root's cron every minute.
>
> The hard lockups seems to be connected to our use of pf and altq, that is at
> least the problem we're trying to gather debug information for.
>
> Thanks in advance
> Martin
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>

-- 
Doug White                    |  FreeBSD: The Power to Serve
dwhite at gumbysoft.com          |  www.FreeBSD.org