Unmapped buffers: to be merged in several days

[Konstantin Belousov]

The latest version of the unmapped buffers patch is available at
http://people.freebsd.org/~kib/misc/unmapped.17.patch
The patch makes the user data buffers, as well as the page-ins, for
UFS, the swap-in/out, clustering use unmapped buffers, removing the TLB
shootdown overhead and buffer map contention and fragmentation.
The ahci(4) and md(4) is converted to accept unmapped BIO requests.

Other drivers and geom classes get the compat mapped BIOs, the
transient mapping is established by the geom down thread. The KVA
for the transient maping the carved from the buffer map, up to 10%
of which is repurposed to the transient bio KVA. The hope is that
the rest of drivers and geom classes will be converted to accept
unmapped i/o shortly, making the transient map unused.

The patch was tested by Peter Holm using the whole stress2 suite,
on both i386 and amd64, on ahci(4) and ad(4) attached disks. ad(4)
uses the transient remapping for unmapped requests, so the testing
should cover both new and old i/o pathes. The previous version of the
patch is already used on some high-load machines by Scott Long, on
ahci(4), isci(4) and mps(4). Brendan Fabeny did useful testing in his
environment.

The biggest change comparing to the previous mail, is the prevention of
the deadlocks due to the bugs in the bufspace limit code. In the HEAD,
bufspace is equal to the size of the buffer map, which effectively
makes the code which limits the total space allocated to buffers, by
maxbufspace, a nop, due to the buffer map fragmentation.

In the patch, filesystem metadata is not the subject to maxbufspace
limit anymore. Since the metadata buffers are always mapped, the buffers
still have to fit into the buffer map, which provides a reasonable
(but practically unreachable) upper bound on it. The non-metadata buffer
allocations, both mapped and unmapped, is accounted against maxbufspace,
as before. Effectively, this means that the maxbufspace is forced on
mapped and unmapped buffers separately.

I intend to commit the change as is, with the following modifications:
- the pmap_copy_pages() will be a stub for all architectures where
  it was not tested. The only tested arches are i386, amd64 and powerpc64.
- For all architectures where pmap_copy_pages() is a stub, the GB_UNMAPPED
  flag for the buffer allocators will be nop.

FYI.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-arch/attachments/20130311/0f731cd2/attachment.sig>