TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux)

[Mike Tancsa]

On 9/10/2013 6:42 PM, Barney Cordoba wrote:
> NFS has been broken since Day 1, so lets not come to conclusions about
> anything
> as it relates to NFS.

iSCSI is NFS ?

	---Mike

> 
> BC
> 
> ------------------------------------------------------------------------
> *From:* Mike Tancsa <mike at sentex.net>
> *To:* Rick Macklem <rmacklem at uoguelph.ca>
> *Cc:* FreeBSD Net <net at freebsd.org>; David Wolfskill <david at catwhisker.org>
> *Sent:* Wednesday, September 4, 2013 11:26 AM
> *Subject:* TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux)
> 
> On 9/4/2013 8:50 AM, Rick Macklem wrote:
>> David Wolfskill wrote:
>>>
>>>
>>> I noticed that when I tried to write files to NFS, I could write
>>> small
>>> files OK, but larger ones seemed to ... hang.
>>> * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on.
>>> * "sysctl net.inet.tcp.tso" showed "1" -- enabled.
>>>
>>> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked
>>> without
>>> a hitch or a whine.  And I was able to copy all 117709618 bytes, not
>>> just
>>> 2097152 (2^21).
>>>
>>> Is the above expected?  It came rather as a surprise to me.
>>>
>> Not surprising to me, I'm afraid. When there are serious NFS problems
>> like this, it is often caused by a network fabric issue and broken
>> TSO is at the top of the list w.r.t. cause.
> 
> 
> I was just experimenting a bit with iSCSI via FreeNAS and was a little
> disappointed at the speeds I was getting. So, I tried disabling tso on
> both boxes and it did seem to speed things up a bit.  Data and testing
> methods attached in a txt file.
> 
> I did 3 cases.
> 
> Just boot up FreeNAS and the initiator without tweaks.  That had the
> worst performance.
> disable tso on the nic as well as via sysctl on both boxes. That had the
> best performance.
> re-enable tso on both boxes. That had better performance than the first
> case, but still not as good as totally disabling it.  I am guessing
> something is not quite being re-enabled properly ? But its different
> than the other two cases ?!?
> 
> tgt is FreeNAS-9.1.1-RELEASE-x64 (a752d35) and initiator is r254328 9.2
> AMD64
> 
> The FreeNAS box has 16G of RAM, so the file is being served out of cache
> as gstat shows no activity when sending out the file
> 
> 
> 
>     ---Mike
> 
> 
> -- 
> -------------------
> Mike Tancsa, tel +1 519 651 3400
> Sentex Communications, mike at sentex.net <mailto:mike at sentex.net>
> Providing Internet services since 1994 www.sentex.net
> Cambridge, Ontario Canada  http://www.tancsa.com/
> 
> _______________________________________________
> freebsd-net at freebsd.org <mailto:freebsd-net at freebsd.org> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org
> <mailto:freebsd-net-unsubscribe at freebsd.org>"
> 


-- 
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike at sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/