endianess of /etc/pwd.db

[Jacques A. Vidrine]

On Wed, Apr 09, 2003 at 07:26:20AM -0500, Jacques A. Vidrine wrote:
> On Wed, Apr 09, 2003 at 10:17:41AM +0200, Poul-Henning Kamp wrote:
> > 
> > Kris ran into this problem:  copying a /etc/pwd.db from one endianess
> > to another gave him really weird uid/gid numbers.
> > 
> > The DB code itself is endianess-agnostic, so the first warning one
> > gets is the weird UID/GID.
> > 
> > Should we make the endianess of this file explicit to prevent this
> > pit-fall for our users ?  The cost would be less than epsilon.
> 
> I am working in this area recently, and the exact same thought
> occurred to me.  I think we should do it.  I would add a file format
> version flag, the absence of which indicates the current MD format,
> and adjust getpwent/pwd_mkdb accordingly.  Are there any other tools
> that would need to be touched?

In case you didn't otherwise notice, this is done.

The `.db' files now have versioned entries.  It is necessary for the
`old MD formatted' and `new MI formatted' entries to co-exist within
the .db files so that old binaries still work.

A `version key' within the database indicates to consumers what
version they should use for lookups.  The key is "\xFF" "VERSION" (8
bytes) and the value is a single byte representation of the integer
version.  The `old MD formatted' version is version 3; the `new MI
formatted' version is version 4.

All entries have historically had a single-byte tag to seperate the
namespace for lookups by name, by user-ID, and so forth.  The tags
have had values that happened to be ASCII-encoded digits (i.e.
'1' -> '\x31', '2' -> 'x32', ... ). These tags are now interpreted
to contain the version in the upper four bits, and the namespace in
the lower four bits.  (Now you see why I picked version 3 for old
entries.)  For example, the keys for the entry `nectar:*:1001:...'
on i386 would be
    "\x31" "nectar"         MD (version 3) by name
    "\x32" "\xe9\x03\0\0"   MD (version 3) by user-ID
    "\x41" "nectar"         MI (version 4) by name
    "\x42" "\0\0\x03\xe9"   MI (version 4) by user-ID

Version 4 (MI formatted) entries have all integers encoded as 32-bit
values in network byte order, but are otherwise the same as the old MD
formatted entries.

libc will read version 4 entries if the version key is present, or
else attempt to read version 3 entries.  pwd_mkdb creates both version
3 and version 4 entries.  Thus, you may safely move your versioned
'.db' files between architectures (as long as the DB format itself
doesn't change!), and binaries using a new libc will return the right
results for getpwent() et. al.

Cheers,
-- 
Jacques Vidrine   . NTT/Verio SME      . FreeBSD UNIX       . Heimdal
nectar at celabo.org . jvidrine at verio.net . nectar at freebsd.org . nectar at kth.se