package distribution crisis - CDN needed

[Pav Lucistnik]

Eberhard Moenkeberg píše v út 08. 04. 2008 v 01:54 +0200:

> > > I do propose a creation of a CDN (Content Delivery Network), having these
> > > features:
> > 
> > The success of a CDN is premised on the working set of active files being
> > [significantly] smaller than the set of available files.  While I don't doubt
> > that only a few packages are of interest to most users, for a CDN to provide a
> > real benefit it needs to be the case that no users require most files.  The
> > reason being that, if you have to transfer all or even most files every time,
> > a CDN may well be a significantly less efficient way to do it than rsync. Has
> > a working set analysis for our main mirrors been done that demonstrates that
> > the actual working set is sufficiently smaller than the total set that a CDN
> > would be more, rather than less, efficient?

I don't have these statistics, but I'd like to see them very much.
Can I ask some of the mirror admins to compile me statistics on
downloads of files from /pub/FreeBSD/ports/*packages*, say for March
2008 ?

I'd also like to have these numbers from ftp.FreeBSD.org so I have a
comparision of volume.

My suspicion is that 99% of users get their packages from
ftp.FreeBSD.org, because that's what pkg_add -r uses without PACKAGEROOT
specification.

> Maybe it is interesting for you to have a look at 
> 
> http://www.poeml.de/~poeml/talks/redirector/
> 
> The SUSE team has a redirection mechanism at work on single file basis, 
> and they also have experienced that it would be "enough" for the users if 
> there are lots of mirrors with the "most requested" files plus some 
> mirrors with all files.

Now this looks so much better and easier than my CDN idea! We could use
it practically as-is.

We will need to patch pkg_install to use http://redirector.FreeBSD.org/
instead of ftp://ftp.FreeBSD.org/


-- 
Pav Lucistnik <pav at oook.cz>
              <pav at FreeBSD.org>

Logitech is infamous for providing 30 MB compressed driver software
packages that install a bunch of crap applications on the system and
round up the number of icons in the tray to the nearest power of two.
  -- Michael Stromberg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: Toto je =?UTF-8?Q?digit=C3=A1ln=C4=9B?=
	=?ISO-8859-1?Q?_podepsan=E1?= =?UTF-8?Q?_=C4=8D=C3=A1st?=
	=?ISO-8859-1?Q?_zpr=E1vy?=
Url : http://lists.freebsd.org/pipermail/freebsd-hubs/attachments/20080408/bfe62618/attachment.pgp