12.0-BETA1 vnet with pf firewall
Rodney W. Grimes
freebsd-rwg at pdx.rh.CN85.dnsmgr.net
Tue Oct 30 16:46:15 UTC 2018
> On 30 Oct 2018, at 14:14, Rodney W. Grimes wrote:
>
> >> On 30 Oct 2018, at 14:29, Bjoern A. Zeeb wrote:
> >>> On 30 Oct 2018, at 12:23, Kristof Provost wrote:
> >>>> I?m not too familiar with this part of the vnet code, but it looks
> >>>> to me like we?ve got more per-vnet variables that was originally
> >>>> anticipated, so we may need to just increase the allocated space.
> >>>
> >>> Can you elfdump -a the two modules and see how big their set_vnet
> >>> section sizes are? I see:
> >>>
> >>> pf.ko: sh_size: 6664
> >>> ipl.ko: sh_size: 2992
> >>>
> >> I see exactly the same numbers.
> >>
> >>> VNET_MODMIN is two pages (8k). So yes, that would exceed the module
> >>> space.
> >>> Having 6.6k global variable space is a bit excessive? Where does
> >>> that
> >>> come from? multicast used to have a similar problem in the past
> >>> that
> >>> it could not be loaded as a module as it had a massive array there
> >>> and
> >>> we changed it to be malloced and that reduced it to a pointer.
> >>>
> >>> 0000000000000f38 l O set_vnet 0000000000000428
> >>> vnet_entry_pfr_nulltable
> >> That?s a default table. It?s large because it uses MAXPATHLEN for
> >> the pfrt_anchor string.
> >>
> >>> 0000000000000b10 l O set_vnet 00000000000003d0
> >>> vnet_entry_pf_default_rule
> >> Default rule. Rules potentially contain names, tag names, interface
> >> names, ? so it?s a large structure.
> >>
> >>> 0000000000001370 l O set_vnet 0000000000000690
> >>> vnet_entry_pf_main_anchor
> >> Anchors use MAXPATHLEN for the anchor path, so that?s 1024 bytes
> >> right
> >> away.
> >>
> >>> 0000000000000000 l O set_vnet 0000000000000120
> >>> vnet_entry_pf_status
> >>>
> >> pf status. Mostly counters.
> >>
> >> I?ll see about putting moving those into the heap on my todo list.
> >
> > Though that removes the current situation, it is a partial fix,
> > doesnt this static sized 2 page VNET_MODMIN needs to be fixed in the
> > longer term?
>
> I think about it the other way round: we might want to bump it to 4
> pages in short term for 12.0 maybe?
81MB in your 10000 vnet case, not horrible really as a quick and
slightly dirty fix.
> The problem is that whether or not you use modules these 2/4 pages will
> be allocated per-vnet, so if you run 50 vnet jails that?s 100/200
> pages. And while people might say memory is cheap, I?ve run 10.000
> vnet jails before on a single machine ? it adds up. I wonder if we
> could make it a tunable though.. Let me quickly think about it and come
> up with a patch.
A boot time tunable that defaulted to 2 pages would be good too.
We are faced with similiar issues in bhyve, where each VM created
gets an allocation based on MAXCPU, I have wip in process to alter
this so that the 4th (and currently hidden) cpu topology option
controls the size of these allocations.
>
> I?ll also go and see to get better error reporting into the
> link_elf*.c files for this case.
Great!
--
Rod Grimes rgrimes at freebsd.org
More information about the freebsd-current
mailing list