superpages in FreeBSD (netmap related) ?

[Marko Zec]

On Mon, 1 Jun 2015 12:11:12 +0200
Luigi Rizzo <rizzo at iet.unipi.it> wrote:

> On Monday, June 1, 2015, Marko Zec <zec at fer.hr> wrote:
> 
> > On Mon, 1 Jun 2015 11:34:00 +0200
> > Luigi Rizzo <rizzo at iet.unipi.it <javascript:;>> wrote:
> >
> > > Hi,
> > > i was wondering how we can improve the netmap memory allocator
> > > to make use of 2M pages (through the page promotion trick).
> > >
> > > in netmap, when we allocate packet buffers,
> > > we issue requests for 4k blocks to contigmalloc(),
> > > and i have no idea if there is a way to improve the
> > > chance that the memory is mapped to 2M pages ?
> >
> > In my (previous life) experience, when requested large enough
> > blocks, malloc() did a good job at automatically promoting those to
> > superpages, and in my applications this behavior was 100%
> > consistent, at least on amd64.  After the block is allocated one
> > can check whether it is superpage-mapped:
> >
> > pmap_t pmap = vmspace_pmap(curthread->td_proc->p_vmspace);
> >
> > if (pmap_mincore(pmap, (vm_offset_t) addr) & MINCORE_SUPER)
> >         /* you're good */
> > else
> >         /* bad luck */
> 
> 
> Thanks. Do you know if there is any way to run some equivalente test
> from user space ?

Sure, here's a quickie:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>

int 
main(int argc, char **argv)
{
        size_t size, flags, i, super = 0, ordinary = 0;
        void *addr;
        char *vec;

        if (argc != 2)
                return (0);

        size = atoi(argv[1]);
        addr = malloc(size);
        vec = malloc(size / 4096);

        memset(addr, 0, size);

        flags = mincore(addr, size, vec);

        printf("addr %p len %d:\n", addr, size);
        for (i = 0; i <= size / 4096; i++)
                if (vec[i] & MINCORE_SUPER)
                        super++;
                else
                        ordinary++;
        printf("%d 4K blocks super-mapped, %d ordinary 4K pages\n",
            super, ordinary);

        return (0);
}

x23% ./a.out 1000000
addr 0x801006000 len 1000000:
0 4K blocks super-mapped, 245 ordinary 4K pages

x23% ./a.out 10000000
addr 0x801000000 len 10000000:
2048 4K blocks super-mapped, 394 ordinary 4K pages

x23% ./a.out 100000000000
addr 0x801000000 len 1215752192:
296448 4K blocks super-mapped, 367 ordinary 4K pages


The key is that the pages must be touched to be considered for merging
in superpages!

Marko


> Cheers
> Luigi
> 
> 
> > OTOH I'm not aware of any mechanisms for forcing superpage
> > allocations at malloc() time.
> >
> > Marko
> >
> 
>