Poor performance with natd/ipfw and TSO enabled on bce(4) card and 8.1-PRERELEASE

[Garrett Cooper]

On Thu, Jul 1, 2010 at 10:32 PM, Garrett Cooper <yanefbsd at gmail.com> wrote:
> On Thu, Jul 1, 2010 at 7:20 PM, Aleksandr A Babaylov <. at babolo.ru> wrote:
>> On Thu, Jul 01, 2010 at 04:54:47PM -0700, Pyun YongHyeon wrote:
>>> On Wed, Jun 30, 2010 at 07:00:53PM -0700, Garrett Cooper wrote:
>>> > Hi,
>>> >     Just an observation I made while transferring a file:
>>> >
>>> > # time scp floppy.img somehost:
>>> > Password:
>>> > floppy.img                                    100% 1440KB  13.7KB/s   01:45
>>> >
>>> > real        1m59.400s
>>> > user        0m0.031s
>>> > sys 0m0.028s
>>> > # sysctl net.inet.tcp.tso=0
>>> > net.inet.tcp.tso: 1 -> 0
>>> > # time scp floppy.img somehost:
>>> > floppy.img                                    100% 1440KB   1.4MB/s   00:00
>>> >
>>> > real        0m0.712s
>>> > user        0m0.018s
>>> > sys 0m0.018s
>>> >
>>> >     Going ISDN speeds transferring a 1.44MB file is sad when you have
>>> > a gigabit uplink :(... natd seems to be doing a LOT of spinning when
>>> > TSO is enabled (it's going up to 73% CPU on a dual-proc quad-core
>>> > machine).
>>> I would use pf(4) if I have to handle lots of NAT rules.
>> Or ipfw nat.
>> man ipfw | grep nat
>
>    That uses the kernel module though, and that's horribly broken on
> my machine with 8-STABLE/9-CURRENT (see:
> http://www.mail-archive.com/freebsd-net@freebsd.org/msg33518.html ). I
> wonder if that's related to the TSO issue.

    As Vitezslav pointed out to me off-list, this is a known issue
with libalias and IPFW:

     Due to the architecture of libalias(3), ipfw nat is not compatible with
     the TCP segmentation offloading (TSO).  Thus, to reliably nat your net-
     work traffic, please disable TSO on your NICs using ifconfig(8).

    Both the ipfw kernel based nat'ing and natd use libalias, so both
will be affected by this issue. This needs to be fixed (I'll see what
I can do to help expedite this).
Thanks,
-Garrett