Attention pf/ipfw users with uid/gid/jail rules (Re: Reminder: NET_NEEDS_GIANT, debug.mpsafenet going away in 7.0)

[Robert Watson]

On Fri, 20 Jul 2007, Paul Allen wrote:

> From Julian Elischer <julian at elischer.org>, Fri, Jul 20, 2007 at 11:36:50AM -0700:
>> Robert Watson wrote:
>>> On Tue, 17 Jul 2007, Max Laier wrote:
>>>
>>> So far I have had 0 (zero) reports of problems since this thread began. 
>>> Could people using uid/gid/jail rules with ipfw or pf on 7.x *please* try 
>>> running their firewalls without debug.mpsafenet -- ignore the witness 
>>> warnings and/or disable witness, and let us know if you experience 
>>> deadlocks.  We're reaching the very end of the merge cycle for 7.0, and I 
>>> would really like to remove the Giant crutches (now effectively unused) 
>>> from the network stack so it's not part of the ABI/API, the code is 
>>> simplified and cleaned up, etc.
>
> Wasn't there a a clear solution to the uid/gid problem involving flip-pages: 
> eliminate the pf lock by forcing reconfigurations to build a parallel 
> data-structure and then perform an atomic operation to exchange the 
> pointers.

I think there are a few potential solutions and areas for work here, the trick 
is figuring out the best approach to get 7.0 out the door.  I think any long 
term structural changes to the firewalls should be avoided at this point, and 
targeted at 7.1 or 8.0.

FYI, my feeling is that the current approach taken, using a pcb lookup in the 
firewall, is not really an appropriate solution to the problem.  Among other 
things, there are (small) race conditions such that the lookup could return 
one pcb in the input path and use that for the check, but another pcb during 
TCP-layer delivery.  The lock order reversal warning is a symptom of reaching 
across layers in fairly ugly (and atomicity-unsafe) ways.

One idea that I'd been pondering was having the inpcb code in the 
TCP/UDP/SCTP/etc layers invoke event handlers as bindings/connections are 
made, making credentials and other information available to firewall packages, 
which could then cache information under their own locks.

> AFAIK, Max's patch was just an ugly hack and it isn't really suitable for 
> performance reasons.
>
> What's the state of MAC for the networking stack?  Are we able to restrict 
> particular uid's to listening only on particular ports?

See mac_portacl(4), which is a functional but not particularly elegant 
implementation of this idea.  In Mac OS X Leopard, many of the traditional 
"firewall" sorts of checks are now performed at the socket layer using this 
sort of approach -- this provides greater application context, allows control 
of things like binding/listening, not just packet transmission and receipt, 
and provides access to the data as received at the application layer rather 
than at the datagram layer, avoiding the need for normalization.  The MAC 
Framework will not be enabled by default in 7.0, but one of my goals for 8.0 
is to ship the framework enabled in GENERIC by default.  This will require a 
significant amount of performance optimization to do.

Robert N M Watson
Computer Laboratory
University of Cambridge