RFC: cpuid_t

[Bruce Evans]

On Sat, 9 Aug 2014, Adrian Chadd wrote:

> How's this look?

Not  good.

> Index: sys/sys/_types.h
> ===================================================================
> --- sys/sys/_types.h    (revision 269480)
> +++ sys/sys/_types.h    (working copy)
> @@ -52,6 +52,7 @@
> typedef    __uint16_t    __nlink_t;    /* link count */
> typedef    __int64_t    __off_t;    /* file offset */
> typedef    __int32_t    __pid_t;    /* process [group] */
> +typedef    __uint32_t    __cpuid_t;    /* CPU ID */
> typedef    __int64_t    __rlim_t;    /* resource limit - intentionally */
>                     /* signed, because of legacy code */
>                     /* that uses -1 for RLIM_INFINITY */

Unsorting.  In the English alphabet, c is not between p and r.

The whitespace seems to be inconsistent, but the mail corrupted all the
tabs so it is hard to tell.

> Index: sys/sys/types.h
> ===================================================================
> --- sys/sys/types.h    (revision 269480)
> +++ sys/sys/types.h    (working copy)
> @@ -154,6 +154,11 @@
> #define    _LWPID_T_DECLARED
> #endif
>
> +#ifndef    _CPUID_T_DECLARED
> +typedef    __cpuid_t    cpuid_t;    /* CPU ID */
> +#define    _CPUID_T_DECLARED
> +#endif
> +
> #ifndef _MODE_T_DECLARED
> typedef    __mode_t    mode_t;        /* permissions */
> #define    _MODE_T_DECLARED

Unsorting.  c is also not between l and m.

The comment is banal, like many nearby ones.  It does less than echo the
code.  Most for ids at least echo the code by repeating "id" in lower case.

In private mail, I said that this typedef is less needed than one for
file descriptors.  Since the latter need is negative, the need for this
one is very negative.  Ids should be small integers so that they can be
used directly as array indexes, unless they need to cover a large sparse
namespace so that they need to be hash numbers.

POSIX has far too many typedefed types.  Similar mistakes were made for
pid_t.  It should have been plain int like the file descriptor type,
but some implementations made it short and some implementations made it
longer.  (4.4BSD-Lite1 even made it long on all arches, but that was a
larger mistake since it gives a very long type on 64-bit arches and
using the same underlying type on all arches loses the only advantage
of the typedef -- that its ABI doesn't depend on the underlying types.
It was changed to int32_t in NetBSD. Lite2 picked up this change, and
FreeBSD picked up the change from Lite2.  Changing it back and forth
mainly exposed brokenness in printf formats for it (many new ones are
broken again, since they assume that pid_t promotes to int).  Changing
it would have broken the ABI for 64-bit arches, but FreeBSD had only
tier {PID_MAX} ones at the time.  NetBSD had higher(numerically lower)
ones, so it had to be more careful with ABIs.  It changed many other
longs back down to ints spelled as int32_t's so that the sizes were the
same as they used to be but the types were not hard-coded as int so as
to defend agains future expansion of ints (hasn't happened yet, and
might never happen since API and ABI assumptions that types are plain
int (even when they are typedefed) are much more pervasive than similar
assumptions for longs (like the one that off_t == long).

POSIX had to make pid_t a typedef to support historical misimplementations
where it wasn't the same.  The problem never affected file descriptors
since mot or all implementations always used plain int.

Plain int being right for file descriptors is related to syscalls returning
int.  open() is a very basic syscall and it always returned int.  pid_t's
are used in not so basic syscalls, so types error from using a more
complicated or different type for pids would have been less obvious.

For CPU ids, the only problem seems to be that u_char is used for them.
This limits their number to 255 real ones and 1 dummy one (not 128
real ones as claimed in this thread).  This saves a whole 3 bytes per
id in struct thread and perhaps in a few other data structures where
packing is not unimportant.  This is like misimplementing pid_t as
u_char (except pid_t needs more than 1 dummy value since it abuses the
top bit for process group ids).  Most systems don't have more than 255
or even 128 CPUs or users, but some do.  If int had been used from the
start like it was for file descriptors, then no one would have noticed
the problem of larger systems.  Using anything except plain int gives
minor type promotion and printing problems.  Using a typedef gives
more problems in theory, since if someone actually uses its for its
only useful use of changing it, then code breaks unless it takes
excessive care with promotion and printing.

A variable type also causes ABI problems.  The current u_char type is
hard-coded twice in struct kinfo.  So to just expand it to int32_t, you
have to add new fields to struct kinfo, and maintain the old fields
for old programs, and put something harmless in these fields instead
of blindly overflowing for unrepresentable values.  kinfo defines an
ABI so it shouldn't have any typedef types in it except ones for
fixed-width integers.  It has many now.  cpuid_t would be a new one.
Then you can't simply change cpuid_t, but have to go through the
expansion or contraction process for kinfo every time you change it.

CPU ids seem to be used mainly as indexes in cpuid_to_cpuid[].  See
pcpu_find().  If they were cookies, then they could be struct pcpu *'s
and not need looking up.  But that wouldn't work in user APIs like
kinfo.  The current use is similar to looking up file descriptors
in a table.  The descriptor must be a small index to work for that.
Management of file descriptor tables has become very complex with
bloat in the number of fds.  The current maximum number of CPUs is
similar to the maximum number of file descriptors on old systems.
I doubt that the number of CPUs will grow as fast as the number of
file descriptors.  If it does, then managing the pcpu tables would
be the least of the problems.  Who knows how to schedule a few million
CPUs?

Bruce