iscsi initiator speed very low in FreeBSD 7

[Thomas Steen Rasmussen]

Gentlemen,

I am experiencing some serious speed problems with the FreeBSD 7 iscsi initiator.

Introduction:
We have recently purchased a SAN (EqualLogic PS5000E, I'll just refer to this as "SAN") to use for logfiles among other things. The SAN is accessible through the iscsi protocol only.
Windows Vista / Windows Server 2008 get impressive speeds from this SAN (2008 got close to 800 megabits in one test). FreeBSD is not even close to that speed. I have tried a bunch of different things, but I always seem to end up with a speed close to 700 kilobits. This is all on gigabit links.

Test setup:
The server (an IBM i386 blade server, EM nics, I'll call this one "blade1") that needs to access the SAN is a production server.

I have therefore configured a test-server (also some i386 blade machine, I'll call this one "blade2") for the sole purpose of troubleshooting this issue.

I do not have the resources to purchase a test SAN :), so a laptop with iscsi-target will have to do (I'll call this one "laptop-target").

The test initiator "blade2" is running FreeBSD 7.0-PRERELEASE from last week, basic install with no additional stuff installed to screw it up. Iscsi-initiator configured with the most basic configuration, just an target IP and target name.

The "laptop-target" is configured with PC-BSD 1.4.1 (~FreeBSD 6.3-PRERELASE) and the iscsi-target port. I've been told that the iscsi-target port isn't built for performance, and I am keeping that in mind. Also, since it is a laptop, the disk is unable to deliver significant amounts of data. But I believe this test still remains valid.

For the test there is a crossover cable between "blade2" and "laptop-target", they are both configured with private static IP addresses and the connectivity between them is fine. Iperf got to around 690 megabits in a quick test. A real world test with SCP also achieved OK speed. I mount the iscsi-target and try to copy something to the iscsi mounted directory. It works but the speed is still around 700 kilobits.

To make sure this isn't a problem where my test "laptop-target" is unable to deliver enough data for some reason, I tried mounting the same target from Windows Vista, formatted it and tested the speed, it got to around 130 megabit which I believe is around the max limit for the poor 5400 rpm laptop drive. So no problems with the laptop, no problems with iscsi-target. The problem really seems to be with the FreeBSD iscsi initiator.

I've reproduced this same result again and again and I fail to see any common points between the two separate cases where I have the problem. My network or the SAN certainly can't be blamed after seeing the results of this test, using a crossover cable and alternate iscsi target.

After mailing a bit with Danny Braniss I learned that he got his initiator to deliver in excess of 80 megabytes/sec when the target is a dedicated appliance, and 50 megabytes/sec when the target is the iscsi-target port. Nice to hear that the initiator code is solid enough to deliver speeds like that, but why on earth doesn't it work for me ?

dmesg, configs etc. will be forwarded immediately upon request if necessary :)

Thank you in advance!


Med venlig hilsen/Kind regards

Thomas Rasmussen
Sikkerheds- og Unix-Administrator
Ventelo
Direkte: + 45 35 25 47 46| Mobil: + 45 27 80 80 90
Tlf.: + 45 70 26 23 00 | Fax: + 45 70 26 23 01
Århusgade 88, 2100 København Ø
thomas.rasmussen at ventelo.dk | www.ventelo.dk