Multilink PPP Download Speeds With Round-Robin Packets

[Julian Elischer]

Michael MacLeod wrote:
> On Jan 16, 2008 2:17 AM, Julian Elischer <julian at elischer.org> wrote:
>> 1/ when downloading, does the load on each incoming interface
>> (I assume you have one ethernet to each modem) match?
> 
> This is a pretty rough way to estimate it, but starting/stopping
> tcpdump on each of the interfaces at the same time suggests the load
> is balanced down each link pretty evenly. There's a lot of inbound
> traffic right now, as a friend is sending me a file right now. Most of
> the other tools I'm used to working with to watch traffic only want to
> work on the tun0 interface created by ppp.
> 
>> 2/ do the IP stats show a lot of out-of order packets?
>> (netstat -s -ptcp I think)
>> in fact ip reassembly problems might be of interest.
>> (netstat -s -pip (?))
>> 3/are there many retransmissions?
> 
> # netstat -s -ptcp
> tcp:
>         144824 packets sent
>                 88543 data packets (117925386 bytes)
>                 34 data packets (45103 bytes) retransmitted
>                 6 data packets unnecessarily retransmitted
>                 0 resends initiated by MTU discovery
>                 49084 ack-only packets (1022 delayed)
>                 0 URG only packets
>                 0 window probe packets
>                 7156 window update packets
>                 42 control packets
>         140315 packets received
>                 42211 acks (for 117742680 bytes)
>                 7009 duplicate acks
>                 0 acks for unsent data
>                 90610 packets (117590676 bytes) received in-sequence
>                 5665 completely duplicate packets (0 bytes)
>                 0 old duplicate packets
>                 0 packets with some dup. data (0 bytes duped)
>                 45 out-of-order packets (64530 bytes)
>                 0 packets (0 bytes) of data after window
>                 0 window probes
>                 640 window update packets
>                 0 packets received after close
>                 0 discarded for bad checksums
>                 0 discarded for bad header offset fields
>                 0 discarded because packet too short

it would be interesting to see how fast the dup packet and dup ack 
numbers are changing..

> < snip >
> 
> # netstat -s -pip
> ip:
>         2143677 total packets received
>         0 bad header checksums
>         0 with size smaller than minimum
>         0 with data size < data length
>         0 with ip length > max ip packet size
>         0 with header length < data size
>         0 with data length < header length
>         0 with bad options
>         0 with incorrect version number
>         0 fragments received
>         0 fragments dropped (dup or out of space)
>         0 fragments dropped after timeout
>         0 packets reassembled ok
>         149764 packets for this host
>         0 packets for unknown/unsupported protocol
>         1986854 packets forwarded (0 packets fast forwarded)
>         651 packets not forwardable
>         0 packets received for unknown multicast group
>         0 redirects sent
>         156851 packets sent from this host
>         0 packets sent with fabricated ip header
>         0 output packets dropped due to no bufs, etc.
>         0 output packets discarded due to no route
>         0 output datagrams fragmented
>         0 fragments created
>         0 datagrams that can't be fragmented
>         0 tunneling packets that can't find gif
>         0 datagrams with bad address in header
> 
> So it looks like the connection is pretty decent, though admittedly I
> haven't seen what these stats should look like on a non-multilink ppp
> connection. However, empirically my VoIP traffic doesn't suffer from
> jitter or choppiness, so I'm going to go out on a limb and say the
> connection is pretty decent.
> 
>> 4/ what is the outgoing data bandwidth when you are downloading?
> 
> I've been tuning my pf firewall to do outbound traffic shaping, so I'm
> not swamping the ack packets in either direction. I can basically run
> the pipe at 5000kb down and 1350kb up consistently.
> 
>> 5/ have you tried mpd? (in ports (multilink ppp daemon).)
>> it may have different operating characteristics.. (it does it all
>> in the kernel using netgraph). Sounds like you need to fix it
>> at the other end but mpd might trigger different behaviour
>> in the router.
> 
> I looked into mpd first. I tried several of the ports (mpd, mpd4, and
> mpd5) all without much success. There isn't much documentation for mpd
> (at least compared to the standard userland ppp). Also, both of the
> individuals who I know have successfully used multilink ppp and
> TekSavvy are using userland ppp. I'd be happy to use mpd, except that
> I'm *almost* there with userland ppp, and I'm guessing it's something
> I've overlooked or misconfigered somewhere and once I find and correct
> it, it'll work like a charm.
> 
> A bit more info about the connection: I'm using pf to do my
> firewalling. My pf.conf includes some packet reassembly bits that one
> of the other successful guys at dslreports sent me.
> 
> The wan nic's are 3com (xl) cards, and my lan and dmz cards are both
> Intel gigabit (em) cards. The router itself is an older dual PIII
> 550Mhz rackmount server I got from my place of employment with plenty
> of ram. Previously I'd used an older PIII 933Mhz computer I had lying
> around, with two realtek (rl) cards for the wan nic's, and I ran into
> the same problems, so I can assume it's not the hardware either.

hmm I 'm not seeing much.. What happens with multiple downloads at 
once vs one big one..