Bad performance when accessing a lot of small files

[dima]

>>>The behavior that I'm observing and that want your help is when the
>>>system is accessing some directory with many small files ( directories
>>>with ~ 1 million of ~30kb files), the performance is very poor.
>>
>> Hi,
> 
> I'm using zfs, I think this change the things.. no ?
> 
> >          Have you adjusted the dirhash value ? What does
> >
> >   sysctl -a vfs.ufs | grep dirhash
> 
> # sysctl -a vfs.ufs | grep dirhash
> vfs.ufs.dirhash_docheck: 0
> vfs.ufs.dirhash_mem: 1410338
> vfs.ufs.dirhash_maxmem: 2097152
> vfs.ufs.dirhash_minsize: 2560

I agree with Mike.

Try to run this command while listing your large directory. You'll see that dirhash_mem will reach dirhash_maxmem. So, the value of dirhash_maxmem should be increased.

One more hint. Mounting filesystems with -noatime option greatly improves read performance. Apply this also, if you haven't done this yet.