vm.kmem_size_max and vm.kmem_size capped at 329853485875 (~307GB)

[Garrett Cooper]

On Thu, Aug 16, 2012 at 6:44 PM, Garrett Cooper <yanegomi at gmail.com> wrote:
> On Thu, Aug 16, 2012 at 5:46 PM, Gezeala M. Bacuño II <gezeala at gmail.com> wrote:
>> Hello fellow listers,
>>
>> On a server with 512GB RAM it appears that vm.kmem_size_max is not
>> being auto-tuned to use >329853485875 (~307GB).
>>
>> On this machine vm.kmem_size is equal to vm.kmem_size_max
>>
>> # from sysctl
>> vm.kmem_size_max: 329853485875
>> vm.kmem_size: 329853485875
>>
>> On a machine with 1GB of RAM, I have successfully set vm.kmem_size_max
>> to 330GB and vm.kmem_size automatically adjusts to 1GB even if I
>> manually set it in /boot/loader.conf.
>>
>> But on the machine with 512GB of RAM it just resets. For the machine
>> to boot, we need to go to the loader prompt and issue:
>>
>> OK set vm.kmem_size_max="300G"
>> OK boot
>>
>> On all PCBSD (8,9) or FreeBSD (8.1,8.2,9) machines we have,
>> vm.kmem_size_max is always set to 329853485875.
>>
>> How can I increase vm.kmem_size_max to use at least 500GB? And how is
>> 329853485875 determined (formula)? I need to increase vm.kmem_size_max
>> and vm.kmem_size so I can set vfs.zfs.arc_max (ZFS ARC) to use say
>> 490GB.
>>
>> I'm browsing thru the source code at
>> http://fxr.watson.org/fxr/search?v=FREEBSD9&string=vm.kmem_size_max
>> and I'm still trying to make sense of how vm.kmem_size_max is
>> computed.
>>
>> I have posted the same topic on forums.freebsd.org but I'm not getting
>> any recommendations.
>>
>> Please see the link for additional details:
>> http://forums.freebsd.org/showthread.php?t=33977
>
> Have you tried defining VM_KMEM_SIZE_MAX to your target value?
>
> Its architecture specific BTW... see
> sys/<architecture>/include/vmparam.h -- look for `VM_KMEM_SIZE_MAX`.

Also, it's a tunable, not a sysctl... so you need to set the value in
/boot/loader.conf .
-Garrett