hast vs ggate+gmirror sychrnoisation speed

[Pawel Jakub Dawidek]

On Fri, Oct 22, 2010 at 05:51:03PM +0300, Mikolaj Golub wrote:
> 
> On Thu, 21 Oct 2010 13:25:34 +0100 Pete French wrote:
> 
>  PF> Well, I bit the bullet and moved to using hast - all went beautifully,
>  PF> and I migrated the pool with no downtime. The one thing I do notice,
>  PF> however, is that the synchronisation with hast is much slower
>  PF> than the older ggate+gmirror combination. It's about half the
>  PF> speed in fact.

What speed do you expect? IIRC from my tests, I was able to saturate
1Gbit link with initial synchronization. Also note, that hast
synchronize only differences, and not the entire thing after crash or
power failure.

>  PF> When I orginaly setup my ggate configuration I did a lot of tweaks to
>  PF> get the speed good - these copnsisted of expanding the send and
>  PF> receive space for the sockets using sysctl.conf, and then providing
>  PF> large buffers to ggate. Is there a way to control this with hast ?
>  PF> I still have the sysctls set (as the machines have not rebooted)
>  PF> but I cant see any options in hast.conf which are equivalent to the
>  PF> "-S 262144 -R 262144" which I use with ggate
> 
>  PF> Any advice, or am I barking up the wrong tree here ?
> 
> Currently there are no options in hast.conf to change send and receive buffer
> size. They are hardcoded in sbin/hastd/proto_tcp4.c:
> 
>         val = 131072;
>         if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_SNDBUF, &val,
>             sizeof(val)) == -1) {
>                 pjdlog_warning("Unable to set send buffer size on %s", addr);
>         }
>         val = 131072;
>         if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_RCVBUF, &val,
>             sizeof(val)) == -1) {
>                 pjdlog_warning("Unable to set receive buffer size on %s", addr);
>         }
> 
> You could change the values and recompile hastd :-). It would be interesting
> to know about the results of your experiment (if you do).

I'd also like to know if it improves anything.

> Also note there is another hardcoded value in sbin/hastd/proto_common.c
> 
>  /* Maximum size of packet we want to use when sending data. */
> #define MAX_SEND_SIZE   32768
> 
> that looks like might affect synchronization speed too. Previously we had 128kB
> here but this has been changed to 32Kb because it was reported about slow
> synchronization with MAX_SEND_SIZE=128kB.

The same restriction exists in ggate.

-- 
Pawel Jakub Dawidek                       http://www.wheelsystems.com
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20101022/d4082fa0/attachment.pgp