[Bug 254333] [tcp] sysctl net.inet.tcp.hostcache.list hangs

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254333

--- Comment #13 from Richard Scheffenegger <rscheff at freebsd.org> ---
Thanks. I can not comment as to why the hostcache.count counts up and beyond
the limit.

But the other data confirms that what you are seeing is the (unsuccessful)
attempt in allocating a huge amount of kernel memory by the sbuf_new function.

When trying to dump all the entries of the hostcache, the hostcache.list
 tries to grab hostcache.cachelimit * 128 bytes, or  1966080 *  128 ~= 250 MB
of contingeous kernel memory - twice (!).

(Also, if the count really is > cachelimit, the hostcache.list may eventually
simply fail, due to insufficient memory...)

The following Diffs are under review, but should address these particular
issues:
o) immediately respond with buffer required without actually
allocating/preparing that 
o) allocating only one bucket's worth of output and moving the output
bucket-by-bucket to userspace (reducing the memory footprint temporarily
required from 2x 250 MB down to 1x 4kB).

See 
https://reviews.freebsd.org/D29471
https://reviews.freebsd.org/D29481
https://reviews.freebsd.org/D29483

Patching with only D29471 should mostly address the issue, although there
remain issues around keeping a lock for an extensive period of time, while
moving the output to userspace repeatedly. This may have undesired sideeffects,
so you want to probably go with all three.

To "unstuck" the system in this state - refrain from issuing hostcache.list
multiple times, and try to free up the above mentioned chunk of in-kernel
memory (at least temporarily). That should allow all the stalled malloc
processes to succeed eventually, and return properly.

But on systems with a high uptime or kernel memory churn, and a very large
hostcache.cachelimit, it really is only a question of time, until such a huge
malloc blocks "indefinitely" - thus the above patches try to be much more smart
in what kernel memory really is needed.

The downside is, that there is a higher chance, that hash buckets may change
more, than they would with that huge memory allocation (if successful).

But hostcache.list does not provide a "complete" snapshot of all the entries in
the hostcache at a very specific moment in time - they may change between the
evaluation of different buckets already (but the longer time for moving data to
userspace may allow more changes to happen).

-- 
You are receiving this mail because:
You are on the CC list for the bug.