zfs send/receive: is this slow?

[Dan Langille]

On Fri, October 1, 2010 11:45 am, Dan Langille wrote:
>
> On Wed, September 29, 2010 3:57 pm, Artem Belevich wrote:
>> On Wed, Sep 29, 2010 at 11:04 AM, Dan Langille <dan at langille.org> wrote:
>>> It's taken about 15 hours to copy 800GB.  I'm sure there's some tuning
>>> I
>>> can do.
>>>
>>> The system is now running:
>>>
>>> # zfs send storage/bacula at transfer | zfs receive
>>> storage/compressed/bacula
>>
>> Try piping zfs data through mbuffer (misc/mbuffer in ports). I've
>> found that it does help a lot to smooth out data flow and increase
>> send/receive throughput even when send/receive happens on the same
>> host. Run it with a buffer large enough to accommodate few seconds
>> worth of write throughput for your target disks.
>>
>> Here's an example:
>> http://blogs.everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/
>
> I'm failing.  In one session:
>
> # mbuffer -s 128k -m 1G -I 9090 | zfs receive
> storage/compressed/bacula-mbuffer
> Assertion failed: ((err == 0) && (bsize == sizeof(rcvsize))), function
> openNetworkInput, file mbuffer.c, line 1358.
> cannot receive: failed to read from stream
>
>
> In the other session:
>
> # time zfs send storage/bacula at transfer | mbuffer -s 128k -m 1G -O
> 10.55.0.44:9090
> Assertion failed: ((err == 0) && (bsize == sizeof(sndsize))), function
> openNetworkOutput, file mbuffer.c, line 897.
> warning: cannot send 'storage/bacula at transfer': Broken pipe
> Abort trap: 6 (core dumped)
>
> real    0m17.709s
> user    0m0.000s
> sys     0m2.502s

My installed mbuffer was out of date.  After an upgrade:

# mbuffer -s 128k -m 1G -I 9090 | zfs receive
storage/compressed/bacula-mbuffer
mbuffer: warning: unable to set socket buffer size: No buffer space available
in @  0.0 kB/s, out @  0.0 kB/s, 1897 MB total, buffer 100% full


# time zfs send storage/bacula at transfer | mbuffer -s 128k -m 1G -O ::1:9090
mbuffer: warning: unable to set socket buffer size: No buffer space available
in @ 4343 kB/s, out @ 2299 kB/s, 3104 MB total, buffer  85% full


-- 
Dan Langille -- http://langille.org/