REPOST: Performance problems with FTP

[Bill Moran]

Charles Swiger <cswiger at mac.com> wrote:
> On Jun 29, 2004, at 11:08 PM, Bill Moran wrote:
> > Charles Swiger <cswiger at mac.com> wrote:
> >> Well, that does tend to rule out a bunch of issues.  Have you tried
> >> changing the MTU of the FreeBSD box down to 1400 or so (or even 512),
> >> just to see whether that does anything?
> >
> > OK.  I've had a bit of success here ...
> >
> > By setting the MTU down to ~650, I get the best performance I've seen
> > with this setup (about 27k/sec ... which isn't too bad)  I set
> > "SocketOptions maxseg 650" in the proftpd.conf file for now, which 
> > seems
> > to help a good bit.
> 
> Bingo, found something!  All should be easy from here on out...  :-)
> 
> > But I'm really confused.  Why does reducing the MTU improve 
> > performance?
> > I would have thought it would hurt performance by increasing the # of
> > packets, thus increasing overhead.
> 
> Using a smaller MTU normally does hurt performance, as you have to send 
> more packets (as you've said) and because the overhead (ratio of packet 
> header size to data size) becomes larger.  However, using an MTU which 
> is too big means packets have to be fragmented and reassembled, which 
> slows things down a lot, too.
> 
> Anyway, it is likely that one of the networks involved in the 
> connection has a smaller MTU than normal, which means large data 
> packets get fragmented, resulting in delays.  FTP data connections tend 
> to show this more than scp does, as the latter seems to vary the packet 
> size more: perhaps a result of using compression/encryption within the 
> SSH protocol.
> 
> > I did some captures using Ethereal, and I'm seeing a weird pause (with
> > the MTU at the default) where the client will send three packets, 
> > there'll
> > be a pause, then the ack comes back, then three packets, pause, ack ...
> 
> That rings a bell.  Are you using path MTU discovery?  Is some firewall 
> in place that might be blocking ICMP_UNREACH_NEEDFRAG messages (ICMP 
> type 3, subtype 4)?
> 
> You should try doing a traceroute which can do pMTU testing; I'm not 
> sure whether the stock FreeBSD traceroute can do this, but a search 
> ought to dig up something.

Well ... this has turned up the weirdest stuff I've seen in a while!

Seems like you're right about PMTU ... but the problem is pretty complex
overall.

The max PMTU seems to change periodically.  It was around 800bytes early
yesterday when I checked it.  Then, late yesterday, when I was going to
track down exactly where the problem was, the PMTU went up to the same
MTU my cable connection uses, thus I couldn't use this connection for
testing as I couldn't get packets out of it without them fragmenting.
(on the flip side, it seems to be _my_ ISP that's rejecting ICMP packets,
so I'll have to complain to them)

Anyway ... your advice, in addition to a number of resource I found on
the web by searching for PMTU, has isolated the problem.  Figuring out
how to fix it will be another thing altogether.

Thanks.

-- 
Bill Moran
Potential Technologies
http://www.potentialtech.com