Dead vnode locking against itself

[Alexander Kabaev]

On Mon, Dec 06, 2004 at 11:07:46PM +0100, Thomas Nystrom wrote:
> Alexander Kabaev wrote:
> > 
> > On Fri, Dec 03, 2004 at 09:12:09PM +0100, Thomas Nystrom wrote:
> > >
> > > If I look at the source I can see that in sys/kern/vfs_subr.c:2696
> > > (in vgonel) the vnode is locked by VI_XLOCK. Later on (via vclean)
> > > dead_lock is called and founds that the vnode is locked. It then
> > > waits for it to be unlocked and the results is a hanged process.
> > >
> > > What have I missed or have I found a bug?
> > >
> > > /thn
> > >
> > I wonder why vclean was called on a vnode with dead_vnodeops in the
> > first place.
> 
> I think that is because the vnode is returned to the free list by arla vith
> a call to vgonel(). That would run vclean() on the vnode and setting it to
> deadfs. When the vnode then is tried to be recycled vgonel() is called once
> more on the same vnode. Vgonel() sets the VI_XLOCK on the node, calls vclean(),
> which calls dead_lock() and finally chkvnlock() which finds VI_XLOCK set and
> tries to wait for it to be released.

Unless I am reading sources wrong, the scenario you describe should not
happen. vgonel resets the vnode type to VBAD just before releasing vnode
locks for the last time and vtryrecycle avoids calling vgonel on vnodes
with that type:

        if (vp->v_type != VBAD) {
                VOP_UNLOCK(vp, 0, td);  
                vgonel(vp, td);
                VI_LOCK(vp);
        } else
                VOP_UNLOCK(vp, 0, td);

The vnode data below shows that vnode has a type of VDIR and evidently its
v_op is set to dead_vnodeops. At the same time, v_tag is "none", which
suggests that vnode has been processed by vclean already, but vgonel had
not yet have a chance to update vnode's type. Just out of curiosity, can you
try your code with VI_UNLOCK(vp);/VI_LOCK(vp); pair removed from the
vgonel function around line 294? This is the only place where vnode interlock
is released after vclean completes and before vgonel sets the vnode type.
There is no point in unlocking the interlock just to lock it again
immediately anyway.

> Since my first posting I have changed the loop in chkvnlock() to:
> 
> while ((vp->v_iflag & VI_XLOCK) && (vp->v_vxthread != curthread)) {
>                 vp->v_iflag |= VI_XWANT;
>                 (void) msleep((caddr_t)vp, VI_MTX(vp), PINOD, "ckvnlk", 0);
>                 locked = 1;
>         }
> 
> That seems to fix the lockup problem. There are still problems in arla
> but that is another problem.
> 
> > Could you please post an output of 'print *vp' for
> > the vnode being cleaned?
> 
> (kgdb) print *vp
> $1 = {v_interlock = {mtx_object = {lo_class = 0xc066f25c, 
>       lo_name = 0xc06471bc "vnode interlock", 
>       lo_type = 0xc06471bc "vnode interlock", lo_flags = 196608, lo_list = {
>         tqe_next = 0xc239d318, tqe_prev = 0xc2399328}, lo_witness = 0x0}, 
>     mtx_lock = 3250837056, mtx_recurse = 0}, v_iflag = 129, v_usecount = 0, 
>   v_numoutput = 0, v_vxthread = 0xc1c3d640, v_holdcnt = 0, v_cleanblkhd = {
>     tqh_first = 0x0, tqh_last = 0xc239d458}, v_cleanblkroot = 0x0, 
>   v_cleanbufcnt = 0, v_dirtyblkhd = {tqh_first = 0x0, tqh_last = 0xc239d468}, 
>   v_dirtyblkroot = 0x0, v_dirtybufcnt = 0, v_vflag = 0, v_writecount = 0, 
>   v_object = 0x0, v_lastw = 0, v_cstart = 0, v_lasta = 0, v_clen = 0, v_un = {
>     vu_mountedhere = 0x0, vu_socket = 0x0, vu_spec = {vu_cdev = 0x0, vu_specnext = {
>         sle_next = 0x0}}, vu_fifoinfo = 0x0}, v_freelist = {tqe_next = 0x0, 
>     tqe_prev = 0xc1e063a0}, v_nmntvnodes = {tqe_next = 0xc241da50, 
>     tqe_prev = 0xc21c9cf0}, v_synclist = {le_next = 0x0, le_prev = 0x0}, 
>   v_type = VDIR, v_tag = 0xc063729d "none", v_data = 0x0, v_lock = {
>     lk_interlock = 0xc0697808, lk_flags = 16777216, lk_sharecount = 0, 
>     lk_waitcount = 0, lk_exclusivecount = 0, lk_prio = 80, 
>     lk_wmesg = 0xc1d892b5 "afs", lk_timo = 6, lk_lockholder = 0xffffffff, 
>     lk_newlock = 0x0}, v_vnlock = 0xc239d4cc, v_op = 0xc1928800, v_mount = 0x0, 
>   v_cache_src = {lh_first = 0x0}, v_cache_dst = {tqh_first = 0x0, 
>     tqh_last = 0xc239d500}, v_id = 88207, v_dd = 0xc2488b58, v_ddid = 87557, 
>   v_pollinfo = 0x0, v_label = 0x0, v_cachedfs = 4294967295, v_cachedid = 389548441, 
>   v_bsize = 512}
> 
> 
> --
> ---------------------------------------------------------------
> Svensk Aktuell Elektronik AB                     Thomas Nystr?m
> Box 10                                    Phone: +46 8 35 92 85
> S-191 21  Sollentuna                        Fax: +46 8 35 92 86
> Sweden                                      Email: thn at saeab.se
> ---------------------------------------------------------------
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"