Re: CFT: snmalloc as libc malloc

On Thu, Feb 9, 2023, 12:37 PM David Chisnall <theraven@freebsd.org> wrote:

> On 9 Feb 2023, at 19:15, Mateusz Guzik <mjguzik@gmail.com> wrote:
> >
> > it fails to build for me:
> >
> > /usr/src/lib/libc/stdlib/snmalloc/malloc.cc:35:10: fatal error:
> > 'override/jemalloc_compat.cc' file not found
> > #include "override/jemalloc_compat.cc"
> >         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > 1 error generated.
> > --- malloc.o ---
> > *** [malloc.o] Error code 1
> >
> > make[4]: stopped in /usr/src/lib/libc
> > /usr/src/lib/libc/stdlib/snmalloc/memcpy.cc:25:10: fatal error:
> > 'global/memcpy.h' file not found
> > #include <global/memcpy.h>
> >         ^~~~~~~~~~~~~~~~~
> > 1 error generated.
> > --- memcpy.o ---
> > *** [memcpy.o] Error code 1
>
> This looks as if you haven’t got the submodule?  Is there anything in
> contrib/snmalloc?
>

And that is why we can't just start using submodules. They are not
automatically used. People have to do different things that need to be
publicized and well documented. And there are about a half a dozen
decisions that need to be made about the details of their use, many of
which have no clear obvious choice for the project. Without a good plan,
clear comms and good docs it will be a support nightmare.

Now please stop making these passive agressive comments about submodules.
All they do is demotivate me to work on the plans to adopt them. There are
a lot of details, many of which need to be play tested, before we can even
get a plan to adopt. The snarky comments are why I quit working on things a
year ago... they don't move the ball forward and just piss people off...

Warner

> this is a fresh world, top of snmalloc2 branch:
> > commit a5c83c69817d03943b8be982dd815c7e263d1a83
> > Author: David Chisnall <theraven@FreeBSD.org>
> > Date:   Fri Jan 21 15:13:09 2022 +0000
> >
> >    Initial commit of snmalloc2 in libc.
> >
> > anyway, I wanted to say I find the memcpy thing incredibly suspicious.
> > I found one article in
> >
> https://github.com/microsoft/snmalloc/blob/main/docs/security/GuardedMemcpy.md
> > which benches it and that made it even more suspicious. How did the
> > benched memcpy look like inside?
>
> Perhaps you could share what you are suspicious about?  I don’t really
> know how to respond to something so vague.  The document you linked to has
> the benchmark that we used (though the graphs in it appear to be based on
> an older version of the memcpy).  The PR that added PowerPC tuning has some
> additional graphs of measurements.
>
> If you compile the memcpy file, you can see the assembly.  The C++
> provides a set of building blocks for producing efficient memcpy
> implementations.  The fastest on x86 is roughly:
>
>  - A jump table of power for small sizes that do power-of-two-sized small
> copies (double-word, word, half-word, and byte) to perform the copy.
>  - A vectorised copy for medium-sized copies using a loop of SSE copies.
>  - rep movsb for large copies.
>
> The compiler does some quite complex layout for the jump table.
>
> David
>
>
>