uma_zalloc_arg complaining about non-sleepable locks

[Marius Strobl]

On Mon, Feb 01, 2010 at 09:46:57AM -0500, John Baldwin wrote:
> On Sunday 31 January 2010 11:28:54 am Marius Strobl wrote:
> > On Sun, Jan 31, 2010 at 12:06:18PM +1100, Peter Jeremy wrote:
> > > Sorry for the delay, I was trying to avoid rebooting my server.
> > > I've setup a similar environment in VirtualBox to test it.
> > > 
> > > On 2010-Jan-27 12:52:29 +0100, Marius Strobl <marius at alchemy.franken.de> 
> wrote:
> > > >Ah, I forgot that using nfsm_aligned() causes nfs_realign() to
> > > >be a NOP on architectures without strict alignment requirements
> > > >for performance reasons. That's generally fine but unfortunately
> > > >that way you don't actually exercise the code which caused the
> > > >problem before (unfortunately I still don't manage to hit the
> > > >unaligned case myself).
> > > 
> > > >Could you please test with #ifdef __NO_STRICT_ALIGNMENT replaced
> > > >with #if 0 in sys/nfs/nfs_common.h? The vfs.nfs.realign_count
> > > >counter should also increase then.
> > > 
> > > I'm not sure what triggers the unaligned case either - I tried
> > > roughly "tar -cf - -C /mnt/usr src | tar -xf - -C /mnt/tmp" and
> > > that caused some unaligned accesses (but also completely wedged
> > > the VBox host).  I also tried copying a pile of files off my
> > > NFS client (FreeBSD-8.x/i386) and that also triggered some
> > > unaligned accesses without any errors being reported.
> > > 
> > > Currently, I have:
> > > vfs.nfs.realign_count: 12
> > > vfs.nfs.realign_test: 188817
> > > 
> > > I'd say that your patch works.
> > 
> > John, are you okay with that patch?
> > http://people.freebsd.org/~marius/fha_extract_info_realign2.diff
> > 
> > It's intention is to:
> > - Move nfs_realign() from the NFS client to the shared NFS code and
> >   remove the NFS server version in order to reduce code duplication.
> >   The shared version now uses a second parameter how, which is passed
> >   on to m_get(9) and m_getcl(9) as the server used M_WAIT while the
> >   client requires M_DONTWAIT, and replaces the the previously unused
> >   parameter hsiz.
> > - Change nfs_realign() to use nfsm_aligned() so as with other NFS code
> >   the alignment check isn't actually performed on platforms without
> >   strict alignment requirements for performance reasons because as the
> >   comment suggests only occasionally occurs with TCP.
> > - Change fha_extract_info() to use nfs_realign() with M_NOWAIT rather
> >   than M_DONTWAIT because it's called with the RPC sp_lock held.
> > 
> > The only downside of the shared nfs_realign() are the combined
> > SYSCTL counters but the fact we incremented them non-atomically
> > so far I think already indicates that their intention only is a
> > rough indication rather than exact values for client and server.
> 
> This all sounds good to me, but isn't M_NOWAIT == M_DONTWAIT?  Hmm, reading 
> the code more closely, it looks like fha_extract_info() was using M_WAIT 
> rather than M_DONTWAIT previously.  Also, I think you should be careful to use 
> M_DONTWAIT instead of M_NOWAIT for mbufs, so I would fix that in 
> fha_extract_info().
> 

Ah, of course you're right, the above description should have read:
Change fha_extract_info() to use nfs_realign() with M_DONTWAIT rather
than M_WAIT because it's called with the RPC sp_lock held.

and I had also managed to copy&paste M_NOWAIT instead of M_DONTWAIT
into fha_extract_info() despite having read mbuf(9) regarding this
topic. Thanks for pointing these out.

Marius