Performance question - istgt with dual 10g data links to linux client

John jwd at FreeBSD.ORG
Tue Sep 25 03:09:53 UTC 2012


Hi Folks,

   I have a bsd 9.1 zfs server running the latest istgt connected to a
RHEL 6.1 system.  Regardless of how I configure the systems, I cannot seem
to exceed 1GB throughput. If I create a 25G /dev/md0 and export it via
istgt (no mpio here), format it with default xfs values, place a
20G file on it, I get the following:

dd if=/usr2/20g of=/dev/null bs=512K
40960+0 records in
40960+0 records out
21474836480 bytes (21 GB) copied, 21.4256 s, 1.0 GB/s

   Running the above /dev/md0 with mpio, dual paths on 10G cards, with
rr_minio set anywhere from 1 to 100 on the linux side:

[PortalGroup2]
   Comment "Two networks - one port"
   Portal DA1 10.59.6.14:5020          # 10G mtu 9000
   Portal DA2 10.60.6.14:5020          # 10G mtu 9000
   Comment "END: PortalGroup2"

mpatha (33000000051ed39a4) dm-0 FreeBSD,USE136EXHF_iSCSI
size=25G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  |- 11:0:0:0 sdd 8:48 active ready  running
  `- 12:0:0:0 sde 8:64 active ready  running


dd if=/usr2/20g of=/dev/null bs=1M
20480+0 records in
20480+0 records out
21474836480 bytes (21 GB) copied, 20.0076 s, 1.1 GB/s

   I can see the traffic evenly across both interfaces. I simply
can't seem to get the parallelization factor up. Higher levels
of mpio have no effect.

   I realize I haven't included the entire configuration. I'm hoping
someone can give some high-level thoughts. I do need to maximize
single process large file i/o..

Thanks,
John

ps: My next thought is to setup a non-unix box and see if I get the
    same results - and point at either client or server side issues.




More information about the freebsd-scsi mailing list