Re: Why is main's system clang (12.0.1-rc2) using /usr/local/bin/aarch64-unknown-freebsd14.0-ld ? (such breaks things)

On 19 Jul 2021, at 18:00, Mark Millard via toolchain <toolchain@freebsd.org> wrote:
> 
> [Mostly this fills in some detail rather than going in
> a different direction from your notes.]

Thanks.

>> The clang driver will provide a *default* value for the -target argument if one is not provided.
> 
> And it happens to be aarch64-unknown-freebsd14.0 in my context.

I suspect that having the base system and ports use different vendor IDs here would prevent any related problems.

> QUOTE from https://llvm.org/docs/GoldPlugin.html
> Building with link time optimization requires cooperation from the
> system linker. LTO support on Linux systems is available via the
> gold linker which supports LTO via plugins. This is the same
> mechanism used by the GCC LTO project.
> 
> . . . The same plugin can also be used by other tools such as ar
> and nm. Note that ld.bfd from binutils version 2.21.51.0.2 and
> above also supports LTO via plugins. However, usage of the LLVM
> gold plugin with ld.bfd is not tested and therefore not officially
> supported or recommended.
> END QUOTE
> 
> So if it is not using its own linker, clang presumes the plugin
> is appropriate, even when it was not built/installed? Possibly
> an "oddity" for a FreeBSD context but possibly a messy thing
> to track.
> 
> (But the wording only allows for the "system linker" as well.)

The wording here predates LLD and probably needs updating.  It looks as if the last time that file was touched was in 2018 and most of the text is from 2013, before lld was useable: https://github.com/llvm/llvm-project/commits/main/llvm/docs/GoldPlugin.rst

It’s still mostly true on non-FreeBSD platforms: On Linux, the system linker is typically provided by GNU binutils and requires the plugin.  On macOS, the system linker is ld64 and does not require a plugin.  In both cases, the linker is maintained outside of the LLVM project and is distributed separately.

FreeBSD is quite unusual in taking the compiler and linker from the LLVM project (though I believe there are plans for macOS to migrate from ld64 to lld at some point).  The other places that ship both tend to be cross-compile toolchains (e.g. the Playstation SDK) and so do not use the system linker.

> For reference:
> 
> QUOTE
> You should produce bitcode files from clang with the option
> -flto. This flag will also cause clang to look for the gold
> plugin in the lib directory under its prefix and pass the
> -plugin option to ld. It will not look for an alternate
> linker without -fuse-ld=gold, which is why you otherwise
> need gold to be the installed system linker in your path.
> 
> ar and nm also accept the -plugin option and it’s possible
> to to install LLVMgold.so to /usr/lib/bfd-plugins for a
> seamless setup. If you built your own gold, be sure to
> install the ar and nm-new you built to /usr/bin.
> . . .
> --- command lines ---
> $ clang -flto a.c -c -o a.o      # <-- a.o is LLVM bitcode file
> $ ar q a.a a.o                   # <-- a.a is an archive with LLVM bitcode
> $ clang b.c -c -o b.o            # <-- b.o is native object file
> $ clang -flto a.a b.o -o main    # <-- link with LLVMgold plugin
> END QUOTE
> 
>> but we don't install it.  It should probably check for the existence of LLVMgold.so and not pass it to the linker if it doesn't exist.
> 
> Could well be. But does that get back into looking in a
> bunch of places to be sure, and to know what would be
> found based on which linker was used? May be too much
> of a mess to be practical.

I was a bit surprised that it’s passed, because the default for FreeBSD should be that the linker is lld-flavoured and so clang shouldn’t need to pass it.  It turns out that, for compatibility, lld silently ignores being told to load the Gold plugin and clang passes this argument even if it knows that the linker is LLD.

> 
>>> There is another message from me where I note that I'd figured
>>> out the -fuse-ld=lld hack, although I omitted "ld." in the
>>> example: should have been aarch64-unknown-freebsd14.0-ld.lld .
>>> I'm actively using this technique because it seems to have the
>>> minimal other potential consequences.
>> 
>> If you do anything that depends on a specific linker or linker behaviour (e.g. LTO) then specifying the linker to use is good practice, not a 'hack'.
> 
> As I understand, that other linker is supposed to be able
> to support LLVM's lto via the plugin. "This is the same
> mechanism used by the GCC LTO project.”

Note that there is no guarantee that the linker in your path is lld or a bfd ld, especially for Arm targets.  The Arm linker, for example, does not support LTO (I think - if it does, it doesn’t, neither do some other proprietary linkers).

> But:
> 
> QUOTE
> You should produce bitcode files from clang with the
> option -flto. This flag will also cause clang to look
> for the gold plugin in the lib directory under its
> prefix and pass the -plugin option to ld. It will not
> look for an alternate linker without -fuse-ld=gold,
> which is why you otherwise need gold to be the
> installed system linker in your path.
> END QUOTE
> 
> But FreeBSD with -fuse-ld=lld works instead of the
> non-existent linker that would go with -fuse-ld=gold .
> This seems to be a FreeBSD mismatch with the
> documentation at https://llvm.org/docs/GoldPlugin.html .

-fuse-ld=gold will cause LLVM to search for a linker whose name ends ld.gold.

The more I look at the docs, the more I realise that most of the docs I’ve read about this are actually comments in the code.  The user facing docs are somewhat less than ideal.

David