9.1-PRERELEASE Performance slowdown with gmultipath Active/Active

[John]

Hi Folks,

   A colleague was setting up a system and noticed some performance
differences. After poking around for a while, we found that gmultipath
in active/active mode is slower than active/passive:

#gmultipath configure -A Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (ACTIVE)
                         da372 (ACTIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.690743 secs (151804079 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681540 secs (153853916 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 1.531437 secs (68470064 bytes/sec)

  Above, note the multipath device is 2x slower.

#gmultipath configure -P Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (ACTIVE)
                         da372 (PASSIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.691087 secs (151728507 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681589 secs (153842830 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681605 secs (153839278 bytes/sec)

   In A/P mode, the performance is equal.


#gmultipath rotate Z181
#gmultipath status Z181
          Name   Status  Components
multipath/Z181  OPTIMAL  da180 (PASSIVE)
                         da372 (ACTIVE)
#dd if=/dev/da180 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.685385 secs (152990803 bytes/sec)
#dd if=/dev/da372 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681613 secs (153837395 bytes/sec)
#dd if=/dev/multipath/Z181 of=/dev/null bs=1m count=100
100+0 records in
100+0 records out
104857600 bytes transferred in 0.681614 secs (153837233 bytes/sec)

Using the other path the performance is also the same.


The drives:

# camcontrol inquiry da180
pass190: <HP EG0600FBDSR HPD4> Fixed Direct Access SCSI-5 device 
pass190: Serial Number EA01PB70SKG41128
pass190: 600.000MB/s transfers, Command Queueing Enabled
# camcontrol inquiry da372
pass390: <HP EG0600FBDSR HPD4> Fixed Direct Access SCSI-5 device 
pass390: Serial Number EA01PB70SKG41128
pass390: 600.000MB/s transfers, Command Queueing Enabled


A complete dmesg from the system - a 9.1-prerelease

FreeBSD 9.1-PRERELEASE #0 r239482M: Tue Aug 21 22:49:16 EDT 2012


here: http://people.freebsd.org/~jwd/dmesg.boot.txt



Any thoughts on what this might be or the best way to debug it?

Thanks,
John