FreeBSD 10-R, Xen 4.1 guest, pf/NAT performance question

[Mark Felder]

On 2014-04-07 07:57, seanrees at gmail.com wrote:
> Hi there freebsd-questions,
> 
> I've been batting my head against this problem for a few days now and 
> not
> having much progress, so I'm hoping to get pointers at what to look at 
> next.
> 
> I've got a FreeBSD 10-R guest in Xen 4.1 (I am just a customer of the 
> Xen
> provider; I don't run the Xen hypervisor myself). I use this instance 
> to
> terminate a VPN, for which I also NAT VPN clients with PF. I am seeing
> unusually slow packet forwarding performance: 0.5mbit internet -> vpn
> client, 2.0 mbit vpn client -> internet. (the numbers should be closer 
> to
> 10mbit/5mbit).
> 
> This guest is a duplicate of another Xen instance I have in another 
> data
> centre. I manage the configurations and packages centrally and aside 
> from
> IP address differences, the machines are configured identically. The
> differences: it's 30ms closer to me and runs in Xen 3.4. I see 
> performance
> from this machine in the 10mbps range.
> 
> I've eliminated the obvious:
>   - The problem VPS is fine network wise; can download tarballs from 
> the
> Internet at 100mbps.
>   - VPS -> Home is fine; can download at ~10mbps; the problem is 
> isolated
> to forwarding Home -> VPS -> Internet and back.
>   - I excluded OpenVPN as the cause by replicating the setup with ssh 
> -w;
> same performance.
>   - SSH port forwarding (ssh -L) is fast; indicating to me the issue is
> somewhere in the PF/kernel.
>   - I checked TCP options by capturing traffic at varying points; these
> seem fine. I see a good deal of TCP retransmits but the window sizes 
> stay
> the same.
> 
> Any thoughts on what to check next?
> 

Have you turned off TSO?

ifconfig xn0 -tso

or

sysctl net.inet.tcp.tso=0