undefined reference to `memset'

[Bruce Evans]

On Wed, 23 Mar 2005, Vinod Kashyap wrote:

> If any kernel module has the following, or a similar line in it:
> -----
> char x[100] = {0};
> -----

I think you mean:
-----
auto char x[100] = {0};
-----
or after fixing some style bugs:
-----
 	char x[100] = { 0 };
-----

> building of the GENERIC kernel on FreeBSD 5 -STABLE for amd64
> as of 03/19/05, fails with the following message at the time of linking:
> "undefined reference to `memset'".
>
> The same problem is not seen on i386.
>
> The problem goes away if the above line is changed to:
> -----
> char x[100];
> memset(x, 0, 100);
> -----

This version makes the pessimizations and potential bugs clear:
- clearing 100 bytes on every entry to the function is wasteful.  C90's
   auto initializers hide pessimizations like this.  They should be
   used very rarely, especially in kernels.  But they are often misused,
   even in kernels, even for read-only data that should be static.  gcc
   doesn't optimize even "auto const x[100] = { 0 };" to a static
   initialization -- the programmer must declare the object as static to
   prevent gcc laboriously clearing it on every entry to the function.
- 100 bytes may be too much to put on the kernel stack.  Objects just
   a little larger than this must be dynamically allocated unless they
   can be read-only.

> Adding CFLAGS+=-fbuiltin, or CFLAGS+=-fno-builtin to /sys/conf/Makefile.amd64
> does not help.

-fno-builtin is already in CFLAGS, and if it has any effect on this then
it should be to cause gcc to generate a call to memset() instead of doing
the memory clearing inline.  I think gcc has a builtin memset() which is
turned off by -fno-builtin, but -fno-builtin doesn't affect cases where
memset() is not referenced in the source code.

-ffreestanding should prevent gcc generating calls to library functions
like memset().  However, -ffreestanding is already in CFLAGS too, and
there is a problem: certain initializations like the one in your example
need to use an interface like memset(), and struct copies need to use and
interface like memcpy(), so what is gcc to do when -fno-builtin tells it
to turn off its builtins and -ffreestanding tells it that the relevant
interfaces might not exist in the library?

> Anyone knows what's happening?

gcc is expecting that memset() is in the library, but the FreeBSD kernel
is freestanding and happens not to have memset() in its library.

Related bugs:
- the FreeBSD kernel shouldn't have memset() at all.  The kernel interface
   for clearing memory is bzero().  A few files misspelled bzero() as
   memset() and provided a macro to convert from memset() to bzero(), and
   instead of fixing them a low-quality memset() was added to <sys/libkern.h>.
   This gives an inline memset() so it doesn't help here.  memset() is of
   some use for setting to nonzero, but this is rarely needed and can
   easily be repeated as necessary.  The support for the nonzero case in
   <sys/libkern.h> is of particularly low quality -- e.g., it crashes if
   asked to set a length of 0.
- memset() to zero and/or gcc methods for initialization to 0 might be
   much slower than the library's methods for clearing memory.  This is
   not a problem in practice, although bzero() is much faster than gcc's
   methods in some cases, because:
   (a) -fno-builtin turns off builtin memset().
   (b) the inline memset() just uses bzero() in the fill_byte = 0 case,
       so using it instead of bzero() is only a tiny pessimization.
   (c) large copies that bzero() can handle better than gcc's inline
       method (which is stosl on i386's for your example) cannot because
       the data would be too large to fit on the kernel statck.
- there are slightly different problems for memcpy():
   (a) memcpy() is in the library and is not inline, so there is no
       linkage problem if gcc generates a call to memcpy() for a struct
       copy.
   (b) the library memcpy() never uses bcopy(), so it is much slower than
       bcopy() in much cases.
   (c) the reason that memcpy() is in the library is to let gcc inline
       memcpy() for efficiency, but this reason was turned into nonsense
       by adding -fno-builtin to CFLAGS, and all calls to memcpy() are
       style bugs and ask for inefficiency.  (The inefficiency is small
       or negative in practice because bzero() has optimizations for
       large copies that are small pessimizations for non-large copies.)
- the FreeBSD kernel shouldn't have memcmp().  It has an inline one that
   has even lower quality than the inline memset().  memcmp() cannot be
   implemented using bcmp() since memcmp() is tri-state but bcmp() is
   boolean, but the inline memcmp() just calls bcmp().  This works, if
   at all, because nothing actually needs memcmp() and memcmp() is just
   a misspelling of bcmp().

Bruce