Namecache lock contention?
Dan Nelson
dnelson at allantgroup.com
Fri Jan 28 16:04:34 UTC 2011
In the last episode (Jan 28), Ivan Voras said:
> I have this situation on a PHP server:
>
> 36623 www 1 76 0 237M 30600K *Name 6 0:14 47.27% php-cgi
> 36638 www 1 76 0 237M 30600K *Name 3 0:14 46.97% php-cgi
> 36628 www 1 105 0 237M 30600K *Name 2 0:14 46.88% php-cgi
> 36627 www 1 105 0 237M 30600K *Name 0 0:14 46.78% php-cgi
> 36639 www 1 105 0 237M 30600K *Name 5 0:14 46.58% php-cgi
> 36643 www 1 105 0 237M 30600K *Name 7 0:14 46.39% php-cgi
> 36629 www 1 76 0 237M 30600K *Name 1 0:14 46.39% php-cgi
> 36642 www 1 105 0 237M 30600K *Name 2 0:14 46.39% php-cgi
> 36626 www 1 105 0 237M 30600K *Name 5 0:14 46.19% php-cgi
> 36654 www 1 105 0 237M 30600K *Name 7 0:13 46.19% php-cgi
> 36645 www 1 105 0 237M 30600K *Name 1 0:14 45.75% php-cgi
> 36625 www 1 105 0 237M 30600K *Name 0 0:14 45.56% php-cgi
> 36624 www 1 105 0 237M 30600K *Name 6 0:14 45.56% php-cgi
> 36630 www 1 76 0 237M 30600K *Name 7 0:14 45.17% php-cgi
> 36631 www 1 105 0 237M 30600K RUN 4 0:14 45.17% php-cgi
> 36636 www 1 105 0 237M 30600K *Name 3 0:14 44.87% php-cgi
>
> It looks like periodically most or all of the php-cgi processes are
> blocked in "*Name" for long enough that "top" notices, then continue,
> probably in a "thundering herd" way. From grepping inside /sys the most
> likely suspect seems to be something in the namecache, but I can't find
> exactly a symbol named "Name" or string beginning with "Name" that would
> be connected to a lock.
My guess would be:
kern/vfs_cache.c:151 static struct rwlock cache_lock;
kern/vfs_cache.c:152 RW_SYSINIT(vfscache, &cache_lock, "Name Cache");
The CACHE_*LOCK() macros.c in vfs_cache use cache_lock, so you've got lots
of possible contention points. procstat -ka and/or dtrace might help you
determine exactly where.
--
Dan Nelson
dnelson at allantgroup.com
More information about the freebsd-hackers
mailing list