Transitioning if_addr_lock to an rwlock

[Robert N. M. Watson]

On 22 Dec 2011, at 16:30, John Baldwin wrote:

> Phase 1 should definitely be MFC'able.  Phase 2 may or may not be.  Robert had 
> the foresight to change drivers to use explicit function wrappers around 
> IF_ADDR_LOCK, and sizeof(struct mtx) == sizeof(struct rwlock), so if we 
> changed the lock type the KBI for existing device drivers would all be fine.  
> Most of the remaining uses of the locking macros are in parts of the kernel 
> that aren't loadable (such as inet and inet6).  We can look at the places that 
> to do change and if any of them are in kld's then it would be up to re@ to 
> decide if 2) was actually safe to merge.  However, even if Phase 2 cannot be 
> MFC'd, having phase 1 makes it easier for downstream consumers to apply Phase 
> 2 locally if they need it.
> 
> You can find the patch for 8.x at 
> http://www.freebsd.org/~jhb/patches/if_addr_rwlock.patch

I think this is a reasonable approach, although to repeat a point I made privately to you, I had anticipated migrating the address lock model to an rmlock in 10.x (and hoped to do it in 9.x, but missed the boat). The reason this improves performance has to do with the synchronous nature of card reconfiguration in the presence of (for example) multicast filter changes, which otherwise stall other consumers of the address list. I didn't encounter this in my earlier measurement, where the migration to rwlocks offered only a small (perhaps no) win due to continued high line contention, which an rmlock would mitigate. I'm quite happy to see the rwlock transition occur, especially in as much as it is mergeable without damaging KPI/KBI. However, in 10.x a significant revisiting of both network configuration locking *and* inter-layer synchronisation during hardware reprogramming is probably called for.

Robert