Kernel memory leak in 8.2-PRERELEASE?

[Boris Kochergin]

On 04/04/11 18:43, Jeremy Chadwick wrote:
> On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote:
>> On 04/02/11 11:41, Boris Kochergin wrote:
>>> On 04/02/11 11:33, Kostik Belousov wrote:
>>>> On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote:
>>>>> Ahoy. This morning, I awoke to the following on one of my servers:
>>>>>
>>>>> pid 59630 (httpd), uid 80, was killed: out of swap space
>>>>> pid 59341 (find), uid 0, was killed: out of swap space
>>>>> pid 23134 (irssi), uid 1001, was killed: out of swap space
>>>>> pid 49332 (sshd), uid 1001, was killed: out of swap space
>>>>> pid 69074 (httpd), uid 0, was killed: out of swap space
>>>>> pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space
>>>>> ...
>>>>>
>>>>> And so on.
>>>>>
>>>>> The machine is:
>>>>>
>>>>> FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu
>>>>> Dec  2 11:39:21 EST 2010
>>>>> spawk at exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS  amd64
>>>>>
>>>>> 10:13AM  up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00
>>>>>
>>>>> The memory line from top intrigued me:
>>>>>
>>>>> Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf,
>>>>> 605M Free
>>>>>
>>>>> The machine has 8 gigs of memory, and I don't know what all that wired
>>>>> memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2
>>>>> on it which has had a disk in the UNAVAIL state for a few months:
>>>>>
>>>>> # zpool status
>>>>>    pool: home
>>>>>   state: DEGRADED
>>>>> status: One or more devices could not be used because the label is
>>>>> missing or
>>>>>          invalid.  Sufficient replicas exist for the pool to continue
>>>>>          functioning in a degraded state.
>>>>> action: Replace the device using 'zpool replace'.
>>>>>     see: http://www.sun.com/msg/ZFS-8000-4J
>>>>>   scrub: none requested
>>>>> config:
>>>>>
>>>>>          NAME        STATE     READ WRITE CKSUM
>>>>>          home        DEGRADED     0     0     0
>>>>>            raidz2    DEGRADED     0     0     0
>>>>>              ada0    ONLINE       0     0     0
>>>>>              ada1    ONLINE       0     0     0
>>>>>              ada2    ONLINE       0     0     0
>>>>>              ada3    ONLINE       0     0     0
>>>>>              ada4    ONLINE       0     0     0
>>>>>              ada5    UNAVAIL      0    85    11  experienced
>>>>> I/O failures
>>>>>
>>>>> errors: No known data errors
>>>>>
>>>>> "vmstat -m" and "vmstat -z" output:
>>>>>
>>>>> http://acm.poly.edu/~spawk/vmstat-m.txt
>>>>> http://acm.poly.edu/~spawk/vmstat-z.txt
>>>>>
>>>>> Anyone have a clue? I know it's just going to happen again if I reboot
>>>>> the machine. It is still up in case there are diagnostics for
>>>>> me to run.
>>>> Try r218795. Most likely, your issue is not leak.
>>> Thanks. Will update to today's 8-STABLE and report back.
>>>
>>> -Boris
>> The problem persists, I'm afraid, and seems to have crept up a lot
>> more quickly than before:
>>
>> # uname -a
>> FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr  2
>> 11:48:43 EDT 2011
>> spawk at exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS  amd64
>>
>> Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free
>>
>> Any ideas for a diagnostic recourse?
> Can you please provide the details I requested here?  Thanks.
>
> http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html
>

No swap, blank /boot/loader.conf, default /etc/sysctl.conf. I'm going to 
try this ARC tuning thing. I vaguely recall several claims that tuning 
wasn't necessary anymore on amd64 systems with the amount of memory mine 
has, but that's obviously not the case.

-Boris