Re: libc/libsys split coming soon

On Sat, Feb 03, 2024 at 12:12:35PM +0100, Mateusz Guzik wrote:
> On 2/3/24, David Chisnall <theraven@freebsd.org> wrote:
> > On 3 Feb 2024, at 09:15, Mateusz Guzik <mjguzik@gmail.com> wrote:
> >>
> >> Binary startup is very slow, for example execve of a hello world
> >> binary in a Linux-based chroot on FreeBSD is faster by a factor of 2
> >> compared to a native one. As such perf-wise this looks like a step in
> >> the wrong direction.
It is the right change to improve modularity and the structure of the code.

> >
> > Have you profiled this?  Is the Linux version using BIND_NOW (which comes
> > with a load of problems, but it often the default for Linux systems and
> > reduces the number of slow-path entries into rtld)?  Do they trigger the
> > same number of CoW faults?  Is there a path in rtld that’s slower than the
> > equivalent ld-linux.so path?
Linux version probably benefits from pre-linking, which might have the
side-effect of breaking semantic into as if BIND_NOW is activated.

> >
> 
> I only profiled FreeBSD, it was 4 years ago. I have neither time nor
> interest in working on this.
> 
> Relevant excerpts from profiling an fexecve loop:
> 
> Sampling what syscalls was being executed when in kernel mode
> (or trap):
> 
> syscalls:
>            pread              108
>            fstat              162
>        issetugid              250
>      sigprocmask              303
>             read              310
>         mprotect              341
>             open              380
>            close             1547
>             mmap             2787
>             trap             5421
> [snip]
> In userspace most of the time is spent here:
>                       ld-elf.so.1`memset              406
>               ld-elf.so.1`matched_symbol              431
>                       ld-elf.so.1`strcmp             1078
>                ld-elf.so.1`reloc_non_plt             1102
>                  ld-elf.so.1`symlook_obj             1102
>                  ld-elf.so.1`find_symdef             1439
> 
> find_symdef iterates a linked list, which I presume induces strcmp calls
> due to unwanted entries.
> [snip]
So strcmp() is almost 1:1 with reloc_non_plt and/or symlook_obj.  It
demonstrates that the ELF hash (perhaps GNU hash, but I do not remember
how long do we have it) provides very good distribution.