cvs commit: src/sys/amd64/include vmparam.h src/sys/arm/include vmparam.h src/sys/i386/include vmparam.h src/sys/ia64/include vmparam.h src/sys/kern kern_exec.c vfs_bio.c src/sys/powerpc/include vmparam.h src/sys/sparc64/include ...

[Jeff Roberson]

On Tue, 25 Sep 2007, Alan Cox wrote:

> alc         2007-09-25 06:25:07 UTC
>
>  FreeBSD src repository
>
>  Modified files:
>    sys/amd64/include    vmparam.h
>    sys/arm/include      vmparam.h
>    sys/i386/include     vmparam.h
>    sys/ia64/include     vmparam.h
>    sys/kern             kern_exec.c vfs_bio.c
>    sys/powerpc/include  vmparam.h
>    sys/sparc64/include  vmparam.h
>    sys/sun4v/include    vmparam.h
>    sys/sys              vmmeter.h
>    sys/vm               vm_contig.c vm_fault.c vm_map.c
>                         vm_object.c vm_object.h vm_page.c
>                         vm_page.h vm_pageout.c vm_pageq.c
>                         vm_phys.c vm_phys.h
>  Log:
>  Change the management of cached pages (PQ_CACHE) in two fundamental
>  ways:

Excellent work Alan!  I'm glad this is making it in.

Thanks,
Jeff

>
>  (1) Cached pages are no longer kept in the object's resident page
>  splay tree and memq.  Instead, they are kept in a separate per-object
>  splay tree of cached pages.  However, access to this new per-object
>  splay tree is synchronized by the _free_ page queues lock, not to be
>  confused with the heavily contended page queues lock.  Consequently, a
>  cached page can be reclaimed by vm_page_alloc(9) without acquiring the
>  object's lock or the page queues lock.
>
>  This solves a problem independently reported by tegge@ and Isilon.
>  Specifically, they observed the page daemon consuming a great deal of
>  CPU time because of pages bouncing back and forth between the cache
>  queue (PQ_CACHE) and the inactive queue (PQ_INACTIVE).  The source of
>  this problem turned out to be a deadlock avoidance strategy employed
>  when selecting a cached page to reclaim in vm_page_select_cache().
>  However, the root cause was really that reclaiming a cached page
>  required the acquisition of an object lock while the page queues lock
>  was already held.  Thus, this change addresses the problem at its
>  root, by eliminating the need to acquire the object's lock.
>
>  Moreover, keeping cached pages in the object's primary splay tree and
>  memq was, in effect, optimizing for the uncommon case.  Cached pages
>  are reclaimed far, far more often than they are reactivated.  Instead,
>  this change makes reclamation cheaper, especially in terms of
>  synchronization overhead, and reactivation more expensive, because
>  reactivated pages will have to be reentered into the object's primary
>  splay tree and memq.
>
>  (2) Cached pages are now stored alongside free pages in the physical
>  memory allocator's buddy queues, increasing the likelihood that large
>  allocations of contiguous physical memory (i.e., superpages) will
>  succeed.
>
>  Finally, as a result of this change long-standing restrictions on when
>  and where a cached page can be reclaimed and returned by
>  vm_page_alloc(9) are eliminated.  Specifically, calls to
>  vm_page_alloc(9) specifying VM_ALLOC_INTERRUPT can now reclaim and
>  return a formerly cached page.  Consequently, a call to malloc(9)
>  specifying M_NOWAIT is less likely to fail.
>
>  Discussed with: many over the course of the summer, including jeff@,
>     Justin Husted @ Isilon, peter@, tegge@
>  Tested by: an earlier version by kris@
>  Approved by: re (kensmith)
>
>  Revision  Changes    Path
>  1.49      +3 -2      src/sys/amd64/include/vmparam.h
>  1.11      +3 -2      src/sys/arm/include/vmparam.h
>  1.45      +3 -2      src/sys/i386/include/vmparam.h
>  1.17      +3 -2      src/sys/ia64/include/vmparam.h
>  1.308     +1 -1      src/sys/kern/kern_exec.c
>  1.527     +4 -10     src/sys/kern/vfs_bio.c
>  1.10      +3 -2      src/sys/powerpc/include/vmparam.h
>  1.18      +3 -2      src/sys/sparc64/include/vmparam.h
>  1.5       +3 -2      src/sys/sun4v/include/vmparam.h
>  1.34      +2 -2      src/sys/sys/vmmeter.h
>  1.63      +2 -11     src/sys/vm/vm_contig.c
>  1.236     +8 -21     src/sys/vm/vm_fault.c
>  1.388     +7 -11     src/sys/vm/vm_map.c
>  1.383     +30 -14    src/sys/vm/vm_object.c
>  1.114     +1 -0      src/sys/vm/vm_object.h
>  1.354     +271 -86   src/sys/vm/vm_page.c
>  1.151     +11 -15    src/sys/vm/vm_page.h
>  1.292     +2 -35     src/sys/vm/vm_pageout.c
>  1.35      +0 -27     src/sys/vm/vm_pageq.c
>  1.4       +115 -33   src/sys/vm/vm_phys.c
>  1.3       +3 -0      src/sys/vm/vm_phys.h
>