Re: When are the git servers available to obtain the ports tree?

On Tue, 3 May 2022, at 04:04, Philip Paeps wrote:
>>>> I'm a maintainer for well over 100 ports. But more often than
>>>> not, I am not permitted to obtain the ports tree from any of
>>>> the FreeBSD git servers:
>>>>
>>>> # git clone -o freebsd --config 
>>>> remote.freebsd.fetch='+refs/notes/*:refs/notes/*' 
>>>> https://git.freebsd.org/ports.git PORTS-20220502

The initial full clone from github, of ports and/or src,
often fails with gateway errors. I've bugged them in the
past and they have amended their "limits" once, and given
up the next time.

I'm curious about the "shallow-info" appearing in your logs
though.

https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables

There's extensive debugging for git, and for HTTP, you
could try `GIT_CURL_VERBOSE=1` and see if that helps
identify problems.

I've previously seen gateway errors when the remote git
server takes too long to prepare whatever packfile it
needs to send me. I'm murky on git internals so I don't
really understand what it's failing on.

Generally, I only cloned a single branch, and fetch main,
which seems to have removed issues for me, but I do this
at least 1x/week. My freebsd remote is called upstream.

# fetch a single branch instead of all branches

git fetch upstream main

# my snippet from /usr/ports/.git/config

[remote "upstream"]
	url = https://git.freebsd.org/ports.git
	fetch = +refs/heads/main:refs/remotes/upstream/main
	fetch = +refs/notes/*:refs/notes/*
	pushurl = ssh://git@gitrepo.freebsd.org/ports.git

[branch "upstream"]
	remote = upstream
	merge = refs/heads/main

If, for some reason, you really need to do a from-scratch
clone every time, it might be simpler to have a personal
mirror of just the /main/ branch on github or similar, and
avoid pulling the whole lot from the mirrors each time.

You can also put `git fetch <remote> <branch>` into a
crontab and let it catch up daily and with less to fetch.

A+
Dave