cvs commit: src/sys/vm uma_core.c uma_int.h

[Jeff Roberson]

On Fri, 29 Apr 2005, Robert Watson wrote:

> rwatson     2005-04-29 18:56:36 UTC
>
>   FreeBSD src repository
>
>   Modified files:
>     sys/vm               uma_core.c uma_int.h
>   Log:
>   Modify UMA to use critical sections to protect per-CPU caches, rather than
>   mutexes, which offers lower overhead on both UP and SMP.  When allocating
>   from or freeing to the per-cpu cache, without INVARIANTS enabled, we now
>   no longer perform any mutex operations, which offers a 1%-3% performance
>   improvement in a variety of micro-benchmarks.  We rely on critical
>   sections to prevent (a) preemption resulting in reentrant access to UMA on
>   a single CPU, and (b) migration of the thread during access.  In the event
>   we need to go back to the zone for a new bucket, we release the critical
>   section to acquire the global zone mutex, and must re-acquire the critical
>   section and re-evaluate which cache we are accessing in case migration has
>   occured, or circumstances have changed in the current cache.

Excellent work.  thanks.  You could also use sched_pin() in uma_zalloc to
prevent migration so you can be certain that you're still accessing the
same cache.  You wont be able to trust the state of that cache.  I'm not
sure whether or not this would make a difference, but it could be
beneificial if we decide to do per-cpu slab lists for locality on NUMA
machines.

>
>   Per-CPU cache statistics are now gathered lock-free by the sysctl, which
>   can result in small races in statistics reporting for caches.
>
>   Reviewed by:    bmilekic, jeff (somewhat)
>   Tested by:      rwatson, kris, gnn, scottl, mike at sentex dot net, others
>
>   Revision  Changes    Path
>   1.119     +120 -103  src/sys/vm/uma_core.c
>   1.30      +0 -10     src/sys/vm/uma_int.h
>