HAST slowness ?

[Mike Tancsa]

On 8/1/2014 2:30 PM, Mikolaj Golub wrote:
> On Thu, Jul 31, 2014 at 05:20:34PM -0400, Mike Tancsa wrote:
>> I am trying to setup my first HAST setup and for some reason, traffic
>> sync seems to be really slow.  If I do something simple like
>>
>>
>> #dd if=/dev/zero of=/hastvirt/test count=2000 bs=1024k
>>
>> on the primary box, traffic on the secondary never gets much beyond a
>> Mb/s ?!?!
>>
>
> It looks you are writing to a file-system, not directly to HAST
> device? What is this FS? I see two HAST devices? Are they combined in
> one FS? How? What if you write directly to a HAST device instead?

Hi Mikolaj,
	Thank you for responding. I have 2 servers, both with two disks.  Each 
have 2 zfs partitions The first zfs partition is the operating system 
ada0p4 and ada1p4 are for hast on each of the boxes

# zpool status
   pool: hastvirt
  state: ONLINE
   scan: scrub repaired 0 in 0h5m with 0 errors on Thu Jul 31 17:34:49 2014
config:

         NAME            STATE     READ WRITE CKSUM
         hastvirt        ONLINE       0     0     0
           mirror-0      ONLINE       0     0     0
             hast/disk1  ONLINE       0     0     0
             hast/disk2  ONLINE       0     0     0


The idea if one box dies, the other can take over and mount hastvirt

>
> Also note, "hole" compression is enabled by default, so your test with
> copying zero blocks is not very good. You might be limited here by
> RTT, because in memsync mode, on every "compressed" block sent HAST
> waits for response before proceeding with the next block.

Perhaps this indeed was the issue. I tried importing a windows VM and it 
seemed to go much faster.  I am now seeing traffic across the HAST 
dedicated NIC at close to 200Mb

% ifstat -b -i vlan5
       vlan5
  Kbps in  Kbps out
262365.4   6407.79
235609.5   5757.97
217813.4   5380.81
144054.8   3606.39
150899.8   3882.17
240806.7   5908.37
238943.7   5718.61

Looking at gstat on the master, it currently has maxed out the disk 
writing so I guess the network is keeping up.

# hastctl status
Name    Status   Role           Components
disk1   complete primary        /dev/ada0p4     1.1.1.1
disk2   complete primary        /dev/ada1p4     1.1.1.1

  hastctl list
disk1:
   role: primary
   provname: disk1
   localpath: /dev/ada0p4
   extentsize: 2097152 (2.0MB)
   keepdirty: 64
   remoteaddr: 1.1.1.1
   replication: memsync
   status: complete
   workerpid: 1021
   dirty: 4194304 (4.0MB)
   statistics:
     reads: 387530
     writes: 309272
     deletes: 11
     flushes: 480
     activemap updates: 19121
     local errors: read: 0, write: 0, delete: 11, flush: 0
     queues: local: 0, send: 0, recv: 0, done: 0, idle: 255
disk2:
   role: primary
   provname: disk2
   localpath: /dev/ada1p4
   extentsize: 2097152 (2.0MB)
   keepdirty: 64
   remoteaddr: 1.1.1.1
   replication: memsync
   status: complete
   workerpid: 1023
   dirty: 4194304 (4.0MB)
   statistics:
     reads: 387008
     writes: 309251
     deletes: 11
     flushes: 480
     activemap updates: 19118
     local errors: read: 0, write: 0, delete: 11, flush: 0
     queues: local: 0, send: 0, recv: 0, done: 0, idle: 255

Are there any good hast tuning guides beyond the wiki you recommend ? 
Should anything be tuned beyond the defaults ?

Thanks again

	---Mike


-- 
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike at sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/