FreeBSD 7.1-PRERELEASE-p1, panic: dqget: free dquot isn't

[Kostik Belousov]

On Thu, Dec 04, 2008 at 08:12:45AM +0100, Frode Nordahl wrote:
> On 3. des.. 2008, at 20.43, Kostik Belousov wrote:
> 
> >On Wed, Dec 03, 2008 at 08:12:22PM +0100, Frode Nordahl wrote:
> >>(moved to freebsd-stable)
> >>
> >>On 3. des.. 2008, at 18.39, Kostik Belousov wrote:
> >>
> >>>>db> where
> >>>>Tracing pid 41111 tid 100199 td 0xffffff0056f1f370
> >>>>kdb_enter_why() at kdb_enter_why+0x3d
> >>>>panic() at panic+0x17b
> >>>>dqget() at dqget+0xaa4
> >>>>getinoquota() at getinoquota+0x5b
> >>>>ufs_access() at ufs_access+0x28c
> >>>>ufs_lookup() at ufs_lookup+0x9fe
> >>>>vfs_cache_lookup() at vfs_cache_lookup+0xf8
> >>>>VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x40
> >>>>lookup() at lookup+0x531
> >>>>namei() at namei+0x35d
> >>>>kern_rmdir() at kern_rmdir+0xbd
> >>>>syscall() at syscall+0x256
> >>>>Xfast_syscall() at Xfast_syscall+0xab
> >>>
> >>>For the start, I want to see the content of the *dq in the dqget()
> >>>frame.
> >>
> >>
> >>I am unable to get to *dq :-(
> >>
> >>(kgdb) frame 10
> >>#10 0xffffffff806dbc54 in dqget (vp=0xffffff011e0767e0, id=419444,
> >>   ump=0xffffff00038a9000, type=0, dqp=0xffffff0122e47268)
> >>   at /usr/src/sys/ufs/ufs/ufs_quota.c:1210
> >>1210				panic("dqget: free dquot isn't");
> >If this is repeatable, change panic to
> >	panic("dqget: free dquot isn't %p", dq);
> >
> >and then in kgdb p/x *(struct dquot *)<printed value>
> 
> Got it!
> 
> panic: dqget: free dquot isn't dq=0xffffff00b3d27000
> (kgdb) p/x *(struct dquot *)0xffffff00b3d27000
> $1 = {dq_hash = {le_next = 0x0, le_prev = 0xffffff010b9da700},  
> dq_freelist = {
>     tqe_next = 0xffffff0055e2de00, tqe_prev = 0xffffffff80b03710},  
> dq_lock = {
>     lock_object = {lo_name = 0xffffffff8088c21c, lo_type =  
> 0xffffffff8088c21c,
>       lo_flags = 0x1030000, lo_witness_data = {lod_list = {stqe_next  
> = 0x0},
>         lod_witness = 0x0}}, mtx_lock = 0x4, mtx_recurse = 0x0},
>   dq_flags = 0xc, dq_type = 0x0, dq_cnt = 0x0, dq_id = 0x16a37d,
>   dq_ump = 0xffffff00038e1600, dq_dqb = {dqb_bhardlimit = 0x0,
>     dqb_bsoftlimit = 0x0, dqb_curblocks = 0x0, dqb_ihardlimit = 0x0,
>     dqb_isoftlimit = 0x0, dqb_curinodes = 0x1, dqb_btime = 0x493ef2b5,
>     dqb_itime = 0x493ef2b5}}
> 
> Let me know if I can provide any more information!
> 
> Updated the crashinfo archive you got an URL to yesterday if you want  
> to poke around yourself.

Please, try the patch below, it should fix a race I see. Quite possible,
this is what you tripped over.

On the other hand, dq_id == 1483645 looks very suspicious. Do you
actually have such large uid in the system ? Could it be that you have
file on fs owned by such uid ?

diff --git a/sys/ufs/ufs/ufs_quota.c b/sys/ufs/ufs/ufs_quota.c
index e5cb956..0c8209a 100644
--- a/sys/ufs/ufs/ufs_quota.c
+++ b/sys/ufs/ufs/ufs_quota.c
@@ -1262,7 +1262,7 @@ dqrele(struct vnode *vp, struct dquot *dq)
 		return;
 	}
 	DQH_UNLOCK();
-
+sync:
 	(void) dqsync(vp, dq);
 
 	DQH_LOCK();
@@ -1271,6 +1271,18 @@ dqrele(struct vnode *vp, struct dquot *dq)
 		DQH_UNLOCK();
 		return;
 	}
+
+	/*
+	 * The dq may become dirty after it is synced but before it is
+	 * put to the free list. Checking the DQ_MOD there without
+	 * locking dq should be safe since no other references to the
+	 * dq exist.
+	 */
+	if ((dq->dq_flags & DQ_MOD) != 0) {
+		dq->dq_cnt++;
+		DQH_UNLOCK();
+		goto sync;
+	}
 	TAILQ_INSERT_TAIL(&dqfreelist, dq, dq_freelist);
 	DQH_UNLOCK();
 }
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20081204/f1ef145d/attachment.pgp