clamav problems with 6-STABLE, with libthr and libpthread

[Attila Nagy]

Hello,

I have:
ldd /usr/local/sbin/clamd
/usr/local/sbin/clamd:
        libclamav.so.2 => /usr/local/lib/libclamav.so.2 (0x80063e000)
        libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x8007a8000)
        libz.so.3 => /lib/libz.so.3 (0x800999000)
        libbz2.so.2 => /usr/lib/libbz2.so.2 (0x800aad000)
        libgmp.so.7 => /usr/local/lib/libgmp.so.7 (0x800bbc000)
        libthr.so.2 => /usr/lib/libthr.so.2 (0x800cf6000)
        libc.so.6 => /lib/libc.so.6 (0x800e0d000)
clamd -V
ClamAV 0.90.1/2873/Mon Mar 19 18:46:10 2007
uname -a
FreeBSD mx 6.2-STABLE FreeBSD 6.2-STABLE #1: Tue Jan 30 11:22:54 CET 
2007     root at compile:/usr/obj/usr/src/sys/MX  amd64

And this in top:
  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 2879 vscan     258  96    0   272M   240M umtx   2 222:07 87.55% 
clamav-milter

A ktrace for the process shows:
kdump -EH | less
  2879 100579 clamav-milter 0.000000 RET   _umtx_op 0
  2879 100469 clamav-milter 0.000009 CALL  
_umtx_op(0x801226ac0,0,0x18875,0,0)
  2879 100579 clamav-milter 0.000041 CALL  
_umtx_op(0x801226ac0,0x1,0x188e3,0,0)
  2879 100579 clamav-milter 0.000078 RET   _umtx_op 0
  2879 100469 clamav-milter 0.000082 RET   _umtx_op 0
  2879 100579 clamav-milter 0.000084 CALL  
_umtx_op(0x801226ac0,0,0x188e3,0,0)
  2879 100469 clamav-milter 0.000091 CALL  
_umtx_op(0x801226ac0,0x1,0x18875,0,0)
  2879 100469 clamav-milter 0.000122 RET   _umtx_op 0
  2879 100579 clamav-milter 0.000127 RET   _umtx_op 0
  2879 100579 clamav-milter 0.000141 CALL  
_umtx_op(0x801226ac0,0x1,0x188e3,0,0)
  2879 100469 clamav-milter 0.000131 CALL  
_umtx_op(0x801226ac0,0,0x18875,0,0)
  2879 100744 clamav-milter 0.000161 RET   _umtx_op 0
  2879 100744 clamav-milter 0.000177 CALL  
_umtx_op(0x801226ac0,0x1,0x18988,0,0)
  2879 100579 clamav-milter 0.000155 RET   _umtx_op 0
  2879 100469 clamav-milter 0.000198 RET   _umtx_op 0
  2879 100469 clamav-milter 0.000213 CALL  
_umtx_op(0x801226ac0,0x1,0x18875,0,0)
  2879 100579 clamav-milter 0.000202 CALL  
_umtx_op(0x801226ac0,0,0x188e3,0,0)
  2879 100782 clamav-milter 0.000233 RET   _umtx_op 0
  2879 100744 clamav-milter 0.000191 RET   _umtx_op 0
  2879 100782 clamav-milter 0.000251 CALL  
_umtx_op(0x801226ac0,0x1,0x189ae,0,0)
  2879 100744 clamav-milter 0.000266 CALL  
_umtx_op(0x801226ac0,0,0x18988,0,0)
  2879 100782 clamav-milter 0.000283 RET   _umtx_op 0
  2879 100782 clamav-milter 0.000308 CALL  
_umtx_op(0x801226ac0,0,0x189ae,0,0)
[13210 similar lines skipped]
 2879 100553 clamav-milter 0.125179 CALL  read(0x38f,0xb83d000,0x1000)
  2879 100579 clamav-milter 0.125180 CALL  
_umtx_op(0x801226ac0,0x1,0x188e3,0,0)
  2879 100553 clamav-milter 0.125198 GIO   fd 911 read 4096 bytes

Although clamav does some work occasionally, it really just spins in the 
above loops and postfix (which delivers mail to it via milter) just gets 
connection timeouts.

I've seen Martin's e-mails in some places about this issue and there he 
wrote that this problem occurs only with libpthread. It seems it's not.

The previous version of the port worked fine^Wbetter.
It seems that the problem has a straight relationship with the number of 
processors, because those machines which has less of them (2 cores) 
timeout and get stuck far less (or don't timeout at all) than the 
others, which has 4 (two dual core xeon CPUs).
The problem was even worse with a dual quad core system, where the 
previous port (both with libthr and libpthreads) was useless with 
clamav, because of the above problem.

Do you have any idea about the solution?

ps: I run clamav in foreground mode, most of the machines have non-HTT 
processors (Xeon 51XX for the most problematic ones, which has two 
cores, but no HTT)

Thanks,

-- 
Attila Nagy                                   e-mail: Attila.Nagy at fsn.hu
Free Software Network (FSN.HU)                 phone: +3630 306 6758
http://www.fsn.hu/