editors/vim -- long fetch times

[Jeremy Chadwick]

This is a subject I've wanted to bring up for a few years now.  Why I
decided to bring it up now, I don't know; I guess I just felt it was
time.

Simply put: the time it takes for "make fetch" to complete in
editors/vim gets worse and worse as more patches are released:

icarus# cd /usr/ports/editors/vim-lite && make distclean
icarus# time make fetch
...
10.801u 3.297s 6:16.33 3.7%     568+1347k 0+52io 0pf+0w

That's over 6 minutes to fetch 116 patches and the base vim tarball,
totalling about 7MBytes of data:

icarus# du -sk /usr/ports/distfiles/vim
7248    /usr/ports/distfiles/vim

True, it's only excessive when you have none of the patches in
DISTFILES/vim (that is to say, if you already have 115 patches and
there's 116 of them, the wait time isn't very long).

The "wasted" time is caused by a few things.  This opens somewhat of a
can of worms (in regards to how to enhance fetch(1)), but the main two:

* FTP protocol is used for each and every patch.  FTP server may be
  doing things like forward and reverse DNS look-ups on client, as well
  as an RFC931/1413 ident check.
* fetch(1) does not support the ability to fetch multiple files during
  the same FTP session.  The idea would be to modify fetch(1) to support
  doing multiple RETRs during a single FTP session.

Solutions as I see them:

* Change the first site in MASTER_SITE_VIM to the HTTP version of
  the main distribution site: http://ftp.vim.org/pub/vim/unix/.  This
  is already in the list, but the HTTP mirrors are *last*, not first.
* Provide "snapshots" of the vim patches as a tgz-ball somewhere.
  Have "make fetch" download that and untar it into DISTFILES/vim/.
  Snapshots would need to be updated every time a new patch was added.
* Modify fetch(1) as described above.
* Get Bram to stop releasing hundreds of individual patches between
  minor releases.  :-) 

Using the HTTP fetch method, the time drastically decreases to 1.5
minutes for me:

icarus# cd /usr/ports/editors/vim-lite && make distclean
icarus# time make MASTER_SITE_VIM=http://ftp.vim.org/pub/vim/unix/ fetch
9.282u 2.747s 1:28.83 13.5%     628+1444k 1+52io 0pf+0w

Thoughts?

-- 
| Jeremy Chadwick                                    jdc at parodius.com |
| Parodius Networking                           http://www.parodius.com/ |
| UNIX Systems Administrator                      Mountain View, CA, USA |
| Making life hard for others since 1977.                  PGP: 4BD6C0CB |