performance issue within VNET jail

[Eugene Grosbein]

23.12.2017 2:11, Michael Grimm wrote:

> Kristof Provost <kristof at sigsegv.be> wrote:
> 
>> I run a very similar setup (although on CURRENT), and see no performance issues from my jails.
> 
> In utter despair I did upgrade one server to CURRENT (#327076) today, but that hasn't been successful :-(
> 
> Ok, right now I do know:
> 
> (#) there is *no* performance loss (TCP) when:
> 
> 	(-) fetching files from outside through PF/extIF to host
> 	(-) fetching files from partner server host via IPSEC tunnel bound to extIF (ESP) to host
> 	(-) fetching files from partner server host via IPSEC tunnel bound to extIF (ESP) to jail via bridge
> 	(-) fetching files from partner server jail via bridge and then via IPSEC tunnel bound to extIF (ESP) to host
> 	(-) fetching files from partner server jail via bridge and then via IPSEC tunnel bound to extIF (ESP) and then via bridge to jail
> 
> (#) there is a *dramatic* performance loss (TCP) when:
> 
> 	(-) fetching files from outside through PF/extIF via bridge to jail
> 
> (#) I did try to tweak the following settings *without* success:
> 
> 	(-) sysctl net.inet.tcp.tso=0 
> 	(-) sysctl net.link.bridge.pfil_onlyip=0
> 	(-) sysctl net.link.bridge.pfil_bridge=0
> 	(-) sysctl net.link.bridge.pfil_member=0 
> 	(-) reducing mtu to 1400 (1490 before) on all interfaces extIF, bridge, epairXs
> 	(-) deactivating "scrub in all" and "scrub out on $extIF all random-id" in /etc/pf.conf
> 	(-) setting "set require-order yes" and "set require-order no" in /etc/pf.conf [1]
> 
> [1] I do see more a lot of out-of-order packages within a jail "netstat -s -p tcp" after those slow downloads, but not after downloads via IPSEC tunnel from partner host.
> 
> That leads me to the conclusions:
> 
> 	(#) the bridge is not to blame
> 	(#) it's either the PF/NATing or something else, right?
> 
> Thanks for your suggestions so far, but I am lost here. Any ideas?

It seems to me some kind of bug in the PF.
I personally never tried it, I use ipfw and it works just fine.

Maybe, you should try to switch to it too, at least for a test.