Re: Impossible to push my ports directory to my CI/CD pipeline

Reply: Brooks Davis : "Re: Impossible to push my ports directory to my CI/CD pipeline"
In reply to: Brooks Davis : "Re: Impossible to push my ports directory to my CI/CD pipeline"
Go to: [ bottom of page ] [ top of archives ] [ this month ]
From: Warner Losh <imp_at_bsdimp.com>
Date: Wed, 02 Oct 2024 15:29:00 UTC
On Wed, Oct 2, 2024, 9:26 AM Brooks Davis <brooks@freebsd.org> wrote:

> On Tue, Oct 01, 2024 at 07:37:00PM +0000, Shawn Webb wrote:
> > On Mon, Sep 30, 2024 at 01:09:32PM UTC, Matthias Fechner wrote:
> > > Dear all,
> > >
> > > I already tried to address that problem on the all-developer list, but
> it is
> > > maybe better to put it here.
> > > I think by accident someone pushed LFS object with a commit of
> java/eclipse.
> > >
> > > git remote -v
> > > freebsd git@gitrepo.freebsd.org:ports.git (fetch)
> > > freebsd git@gitrepo.freebsd.org:ports.git (push)
> > > freebsd-https?? https://git.freebsd.org/ports.git (fetch)
> > > freebsd-https?? https://git.freebsd.org/ports.git (push)
> > > githubfreebsd?? https://github.com/freebsd/freebsd-ports.git (fetch)
> > > githubfreebsd?? https://github.com/freebsd/freebsd-ports.git (push)
> > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (fetch)
> > > origin? git@gitlab.fechner.net:mfechner/Gitlab.git (push)
> > >
> > > git lfs fetch --all freebsd
> > > fetch: 22 object(s) found, done.
> > > fetch: Fetching all references...
> > > [99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [c89e491472c8af1d153e2e2801836fad0e38f428d0250ed982e16baa5effbe4f]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [f7db6794c6c3e7a03a3987537b8a0cdb1bde2996ed7c7eec50f2e5df1e7a30c2]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [e8e2d306fd11358a07b4ee10bfef19de3b518793568fc498cdd989170d31712b]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [5dbe70b5c83520df09c100815bb164c8903da5eb8bffbd094aaaed54b212d73a]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [8813907b0f7b001080a346ed2d0307a1c749b85688fdfa1d270fe59a07aa3909]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [1d41b04bdd4115c52aac8177e1b899403a3d7d9f6d147cd864463029648a4714]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [c0155fc8e7c4b9736d81db638fed95bc9ff297453bfd0bbd7dd5d882ebca6e89]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [6d0aef4ba9e96d4c63baae8db9cdce7c634140a2bda7da670e698e38e0df57ce]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [a928176ec92ad347404b2454e5f4f321aaef8ba862a60b0c28adc3bdcc619a4a]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [42fa73302624dc511e6d8958c66c82f31b8d9155f2980343b6bcdc25345e1dcd]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [ed324ddc16ccb8b9fab3041da394b844166b6778c39cd12708bcade05b41c0da]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [9ef847d4e699b5dbe49ac74ee35596fdddc22c6335bc96affee19a99ecd077f5]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [f5417c277c6ca509334010bb6999c56c3ab5d3751204c74dc00c2fa25d93dfae]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [a1ffa591430df806983c8798113264f12ea679bce4b785d70d2868dc91cec883]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [b2e525c38598dac6885a75e7eaffa346c8aca3dbdda73cb0275f50103ec3d245]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [5db9424884396cf5bfc46a6b22dacbe245f43a7e1eda6aeac831905dc921802e]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [b5e05ef77effb4048e72e38605700151fdf4eae214cfa358e9e81a48e124cb52]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [ca2c21bf21e1fa2f53c72653ea57e76506c305bdd44c6416542ebbf2ab8b1b83]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [4fc786126dd64f936554a12c4752787c4989453654fc5d6fd119b6e5eeafec6f]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > [f56db5e81304e441260b84d6c2805ae7050ea56760d763d2b32e6be928fa1cae]
> Object
> > > does not exist on the server: [404] Object does not exist on the server
> > > error: failed to fetch some objects from
> > > 'https://github.com/freebsd/freebsd-ports.git/info/lfs'
> > >
> > > I have my own testing pipeline in place to automatically test my
> commits
> > > against changes I did to ports.
> > >
> > > But as LFS is now in the FreeBSD repository, but the files cannot be
> > > retrieved from the FreeBSD repository I'm now locked out for my CI/CD
> > > pipeline and my automated tests.
> > >
> > > If I now try to push to my local repo I get:
> > >
> > > git push origin
> > > Warning: untrusted X11 forwarding setup failed: xauth key data not
> generated
> > > Locking support detected on remote "origin". Consider enabling it with:
> > > ? $ git config
> > > lfs.
> https://gitlab.fechner.net/mfechner/Gitlab.git/info/lfs.locksverify true
> > > Unable to find source for object
> > > 99a670c083e000008ee29dd972b60cbad6ade1d6ec024b30ff45f41da3d19e1d (try
> > > running git lfs fetch --all)
> > > Uploading LFS objects:?? 0% (0/21), 0 B | 0 B/s, done.
> > > error: failed to push some refs to 'gitlab.fechner.net:
> mfechner/Gitlab.git'
> > >
> > > Does anyone have an idea how that can be fixed?
> >
> > Hey Matthias,
> >
> > With commit c94e4ced329298efb06189d1c6db26e3d6ff3eef[1], FreeBSD
> > introduced LFS objects into the ports git history. This commit was
> > reverted with commit 3ec21e417cd5752c96b88587a5a605567ba5cd19[2].
> >
> > Even though the offending commit was reverted, the git commit history
> > still contains LFS references. This caused HardenedBSD's self-hosted
> > GitLab to become unhappy in exactly the same way yours did.
> >
> > I ended up disabling LFS for the ports repo in the GitLab side. This
> > caused GitLab to become happy again. Problem is, now everyone
> > downstream from us (and FreeBSD, too) will need to disable LFS on
> > their repos/forks, too.
>
> I'm curious if you can turn LFS back on once the broken change is past?
> That that ports should ever use LFS, but it seems somewhat better if
> GitLab configs only need to be non-default during bootstrapping.
>
> At this point there's probably nothing to be done other than workarounds
> as rewriting history to remove this stuff would be quite disruptive
> after several weeks of commits.
>
> We could really use some hooks to block both things that look like lfs
> metadata and submodule bits.  (The latter is somewhat challenging in that
> we need to allow them on the vendor side to support OpenZFS.)
>

I didn't think we imported the submodules of OpenZFS to the vendor branch,
except to the extent that data was plain files in that import not otherwise
used...

Warner

>
>