The vim port needs a refresh

[Michael Gmelin]

On Tue, 28 May 2013 15:16:00 +0100
RW <rwmaillists at googlemail.com> wrote:

> On Tue, 28 May 2013 15:14:52 +0200
> John Marino wrote:
> 
> > All
> > patches only take 74 seconds to download[2] so there is no sympathy
> > for your obviously single data point anecdote, 
> 
> Well at the point you provided one data-point there was only one data
> point. And it was like pulling teeth to get you to eliminate the
> alternative explanations. Was it really too much to ask that you
> provided some actual evidence. 
> 
> > you're clearly doing
> > something wrong.  You need to stop complaining and start think about
> > folks with slow connections[3] who also rebuild Vim frequently. 
> 
> Don't make things up. I never said anything about frequent rebuilds,
> the patches all get redownloaded on the next rebuild. 

The real issue is not the number of patches, but the fact that every
single patch is downloaded by invoking the fetch(1) command, creating
lots of overhead not limited to the fetch command itself. The ports
system wasn't designed for such an amount of distfiles in a single
port I guess.

I just timed fetching the patches through ports vs. fetching over
HTTP/1.1 using ftp/curl vs calling fetch directly. The VIM tarball was
already downloaded, so this is really just the patches (plus
downloading 6mb is barely noticeable on a fast line). It's a slow
machine on a fast line.

Fetch:
[user at server /usr/ports/editors/vim]$ time sudo make fetch
....
real    4m57.327s
user    0m17.010s
sys     0m39.588s

Curl:
[user at server /tmp]$ longcurlcommandline 
....
real    0m15.291s
user    0m0.026s
sys     0m0.272s

Fetch on the command line (after initial make fetch, so this is only
measuring transmission of the files):
cd /usr/ports/editors/distfiles
time for name in 7.3.*; do
  fetch http://artfiles.org/vim.org/patches/7.3/$name
done
....
real    1m25.329s
user    0m0.660s
sys     0m3.174s

So just the fact we're invoking fetch for every file costs us about one
minute - I assume the time lost is much bigger on a slow line with
long latency. The remaining 3.5 minutes are spent somewhere in the
ports infrastructure and clearly depend on the performance of the
machine used. For comparison I timed "make fetch" on a reasonably fast
server (good IO, fast datacenter connection), make fetch still took
about 120 seconds(!).

So the bottomline is:
- Using HTTP/1.1 and keepalive could safe a lot of time
- The ports infrastructure creates a lot of overhead per patch file

Maybe there's something we can do to improve the situation.

Cheers,
Michael

PS: I don't use vim myself and have no stake in this discussion
whatsoever.

-- 
Michael Gmelin