slow directory operation on huge dirs
Miroslav Lachman
000.fbsd at quip.cz
Thu Jan 23 13:35:16 UTC 2020
Wojciech Puchar wrote on 2020/01/23 14:12:
>>>>
>>> nobody have idea what's the problem?
>>
>> Is it "just slow" or is your disk too busy? (iostat)
>
> no. disk is idle, system CPU load takes half of system (8 cores)!
>
>>
>> Can it be that pre-cacheing on huge directory makes disk too busy
>> because there is too much to read?
> this part works fine. The problem begins where it's already cached.
>
> To get the problem you need BOTH of:
>
> - large maxvnodes like half million
> - large directory (in order of 100000 files)
I remember very similar problem with slow access of directories with 80
000+ files. It was always slow like frozen hell. There is some
recommendation to avoid more than 10 000 of file entries in a directory.
That is why filesystem caches or sessions in PHP must create more levels
of subdirectories.
I know you cannot control users of maildirs.
I am not sure if this situation can be improved or not.
Miroslav Lachman
More information about the freebsd-hackers
mailing list