Speeding up the loader(8).

[John Baldwin]

On Tuesday, January 24, 2012 1:22:57 pm Artem Belevich wrote:
> 2012/1/23 Edward Tomasz Napierała <trasz at freebsd.org>:
> > Some time ago I've spent some time on trying to speed up loading
> > modules by the loader(8).  Result can be found at:
> >
> > http://people.freebsd.org/~trasz/fast-loader-3.diff
> >
> > This patch solves three issues:
> >
> > 1. As it is now, the code in biosdisk.c tries very hard to split
> >   reasonably sized (up to 64kB, IIRC) requests into smaller ones.

It is more that the I/O's cannot cross a 64kb boundary.  This is due to a 
limitation in old disk controllers.  Newer versions of EDD provide flags
that indicate whether a device needs this to be honored or not.  You could use 
these flags from EDD to determine if the splitting should be used or not which 
might help with your case while still being safe for older devices (and for 
some more limited devices such as flash).  EDD3 can also let you specify the 
raw 64-bit physical address to write the bits into rather than always using a 
bounce buffer in the low 1MB.  This would also be a good thing to take 
advantage of.

> > 2. The code in biosdisk.c rereads the partition table and probably
> >   some filesystem metadata every time a file gets opened, i.e.
> >   for every module.  These reads bypass the bcache.
> >
> > 3. The code in bcache.c doesn't really implement an LRU - it implements
> >   'least recently added' algorithm, i.e. a kind of queue.  Not that
> >   it matters much, since it flushes the elements two seconds after
> >   caching them anyway.  I replaced it with Least Frequently Used.
> >   LRU didn't behave well, as it tended to replace metadata with data
> >   used only once.

These sound reasonable, though I suspect they are in part due to dealing with 
floppies where the user can swap out of the disk and we have no way of 
noticing otherwise.  However, we could possibly adjust some behavior to cache 
the bits if the disk is not a floppy drive.

> 4. it flushes cache on access to a different drive which means that
> cache does not help on multi-disk ZFS setups.

I believe this is also necessary to deal with floppies and the fact that you 
don't have a reliable way of knowing if a floppy has changed.

-- 
John Baldwin