Memory reserves or lack thereof

[Dieter BSD]

Alan writes:
> In conclusion, I think it's time that we change M_NOWAIT so that it doesn't
> dig any deeper into the cache/free page queues than M_WAITOK does and
> reintroduce a M_USE_RESERVE-like flag that says dig deep into the
> cache/free page queues.  The trouble is that we then need to identify all
> of those places that are implicitly depending on the current behavior of
> M_NOWAIT also digging deep into the cache/free page queues so that we can
> add an explicit M_USE_RESERVE.

find /usr/src/sys | xargs grep M_NOWAIT | wc -l
2101

Sounds like a lot of work that would need to happen atomically.
Would this work:

M_NO_WAIT       do not sleep, do not dig deep unless M_USE_RESERVE also set
M_USE_RESERVE   dig deep
M_NOWAIT        M_NO_WAIT | M_USE_RESERVE (deprecated)

New code avoids using M_NOWAIT. Existing code continues working the same way.
As time permits, old code is converted to new flags. Eventually M_NOWAIT
goes away.

Pro: the amount of code that needs to change atomically is much smaller.

Con: (1) Have to remember (or look up) difference between M_NOWAIT
and M_NO_WAIT. Maybe calling the new flag M_NO_SLEEP would help?
(2) Would M_NOWAIT really ever go away? The spl() calls haven't,
even after some cage rattling.