"Request Requeued/Retrying Command" with twa card on FreeBSD 6.0

[Mike Andrews]

I've got a weird but apparently minor issue with a 3Ware 9500S-4LP in a 
FreeBSD 6.0-RELEASE system...

When the system is run with boot.verbose="YES" in /boot/loader.conf, I get 
these messages on the console:

(da0:twa0:0:0:0): Request Requeued
(da0:twa0:0:0:0): Retrying Command
(da0:twa0:0:0:0): Request Requeued
(da0:twa0:0:0:0): Retrying Command
(repeat a few dozen times each time)

...during some heavy disk activity, like, for example, qpopper rewriting a 
50 MB mailbox.  (I know qpopper's a bit of a slug.)  Not all heavy disk IO 
triggers it, though; I can do something like "dd if=/dev/zero 
of=/tmp/testfile bs=10485760 count=128" followed by "dd if=/tmp/testfile 
of=/dev/null bs=10485760" and have it complete without a single SCSI 
timeout message.

Turn boot.verbose="NO" off and the messages go away, but IO still lags 
badly.

"camcontrol tags" says it has 254 tag openings, which seems like a bit 
much.  Dropping down to 16 seems to make the messages go away.

Do the twa cards even do TCQ between the card and the OS?  From what I 
understand, they won't do NCQ between the card and the SATA disks, even if 
you have something like Barracudas that support it -- you need the new 
3Ware 9550 card for that.

If they don't, why does it advertise 254 tags, and if it does, why does it 
seem to choke on the default setting of 254?

Other than laggy IO (shell prompts hang, MP3's skip for 5 - 30 seconds) 
there aren't any actual IO errors or anything else wrong, which for this 
particular system is just minor annoyance instead of serious problem. 
5.4-RELEASE did the same thing.  I have an apparent workaround, I'm just 
trying to sanity-check it and its necessity.

This is an Asus P4C800-E Deluxe board, so I'm running the twa card in a 
32-bit slot.  The system is otherwise totally idle when this happens, no 
interrupt storms, USB and Firewire disabled in the BIOS so we don't have 
IRQs shared with Giant-locked drivers...

dmesg, vmstat -i, other stuff available on request if needed.  Kernel 
config is the stock 'SMP' one.



Mike Andrews  *  mandrews at bit0.com  *  http://www.bit0.com
It's not news, it's Fark.com.                  Carpe cavy!