Who herds the ftp mirror felines?

[Doug Barton]

On 09/11/2011 22:48, Peter Pentchev wrote:
> On Sun, Sep 11, 2011 at 08:06:48PM -0700, Doug Barton wrote:
>> So I'm in the process of updating the code in portmaster that takes the
>> user's uname and converts that into the proper string for fetching
>> packages based on version, arch, etc. and I was looking through the ftp
>> mirrors to see what is what, and noticed some odd stuff (in more or less
>> increasing order of importance):
>>
>> 1. All the mirrors (which AFAICS are ftp[1-14].freebsd.org) have
>> packages for 6-stable
> 
> Actually, there are a lot more mirrors than that :)  Just take a look
> at the list in the Handbook (appendix A.2) or even on the very first
> page of the website :) (although the website only lists the WWW mirrors,
> for the FTP ones you'd have to go to the Handbook).

Excellent, thanks!

I notice on that list that in some cases there are contacts for the
CC.freebsd.org domain listed, but no contacts are listed for the admins
of the servers themselves. Does $someone have the ability to contact them?

>> 2. ftp3 has packages for 6.4-RELEASE
>> 3. ftp8 and ftp9 have packages for 5-current
> 
> AFAIU, obsolete package collections are first removed from ftp-master,
> then this is propagated to ftp-master.eu, then the mirrors themselves
> decide whether to remove files no longer present on their uptsream
> server.  Sometimes the individual mirror administrators may decide to
> not automatically remove files (if they judge that there is enough space
> available on their particular mirror) that have disappeared upstream,
> sometimes they may.

That sounds fine, except in the cases of truly old stuff like packages
for 5-current (and arguably for 6.x stuff as well, but I'm less
concerned about that than I am 7 year old stuff for an unsupported
branch). Has there ever been an attempt to bring some kind of order to
the chaos?

>> 4. ftp10 and ftp14 have DNS records, but the servers either don't
>> respond, or don't have a useful response.
>>
>> So who manages the package distribution, mirroring, etc. (either
>> generally, or for packages)?
> 
> Generally, mirror-admin@ does, with discussions happening on the -hubs
> list.

Ok, I've cc'ed both, anyone who replies should feel free to take
developers@ off the cc list.

>> Given that we're about to have a whole new
>> release version one could make the argument that removing stale stuff
>> from the mirrors now would help make space available for the new, useful
>> stuff.
> 
> Removing stale stuff from ftp-master actually happens often enough,
> with heads-up mails from the mirror-admin@ members to at least the -hubs
> list, sometimes other lists, too.
> 
>> Not to mention either fixing or removing DNS for the sites that
>> aren't answering being a benefit to our users.
> 
> This is indeed something best raised with mirror-admin@, as described in
> the "Mirroring FreeBSD" article -
> http://www.FreeBSD.org/doc/en/articles/hubs/

Ok, done.

On a more long term basis one wonders what kind of adult supervision is
in place for this stuff. In 5 minutes testing I found 2 broken ftp
servers out of 14. Isn't this something that we should be monitoring?


Doug

-- 

	Nothin' ever doesn't change, but nothin' changes much.
			-- OK Go

	Breadth of IT experience, and depth of knowledge in the DNS.
	Yours for the right price.  :)  http://SupersetSolutions.com/