Re: HELP! fetch: stuck forever OR error: RPC failed: curl 56 recv failure: Operation timed out

On Wed, 04 Dec 2024 17:20:39 +0000
"Dave Cottlehuber" <dch@skunkwerks.at> wrote:

Thank you very much for responding!

> On Tue, 3 Dec 2024, at 19:46, FreeBSD User wrote:
> > On most recent CURRENT (on some boxes of ours, not all) fetch/git seem 
> > to be stuck
> > forever fetching tarballs from ports, fetching Emails via claws-mail 
> > (TLS), opening
> > websites via librewolf and firefox or pulling repositories via git.
> >
> > CURRENT: FreeBSD 15.0-CURRENT #1 main-n273978-b5a8abe9502e: Mon Dec  2 
> > 23:11:07 CET 2024
> > amd64
> >
> > When performing "git pull" und /usr/ports, I received after roughly 5-7 minutes:
> >
> > error: RPC failed: curl 56 recv failure: Operation timed out  
> 
> Generally it would be worth seeing if the HTTP(S) layers are doing the right thing or
> not, and then working down from there, to tcpdump / wireshark and then if necessary
> into kernel itself.

My skills are limited, according to packet analysis utilizing tcpdum/wireshark (and
theory,of course). I tried due to "a feeling" my used older Intel based NIC could have
some checksum issues like in the past (I saw e1000 driver updates recently flowing into
FreeBSD CURRENT).
> 
> If fetch fails reliably in ports distfile fetching, then isolate a suitable tarball,
> and try it again in curl, with tcpdump already prepared to capture traffic to the
> remote host.
> 
> tcpdump -w /tmp/curl.pcap -i ... host ...
> 
> env SSLKEYLOGFILE=/tmp/ssl.keys curl -vsSLo /dev/null --trace
> /tmp/curl.log https://what.ev/er
> 
> I would guess that between the two something useful should pop up.
> 
> I like opening the pcap in wireshark, it often has angry red and black highlighted
> lines already giving me a hint.
> 
> The SSLKEYLOGFILE can be imported into wireshark, and allows decrypting the TLS traffic
> as well in case there are issues further in. Very handy,
> see https://everything.curl.dev/usingcurl/tls/sslkeylogfile.html for how to do that.
> 
> If your issues only occur with git pull, its also curl inside and supports similar
> debugging. Ferreting
> through https://stackoverflow.com/questions/6178401/how-can-i-debug-git-git-shell-related-problems/56094711#56094711 should get you similar info.
> 
> A+
> Dave
> 

Thanks for the hints and precious tips! I'll digg deeper into the matter.

In the meanwhile, I updated some other machines running CURRENT since approx. two weeks
with an older CURRENT to the most recent one - and face similar but not identical
problems!
Updating exiting FreeBSD repositories, like src.git and ports.git, show no problems
except they take longer to accomplish than expected.
Cloning a repo is impossible, after 10 or 15 minutes I receive a timeout.

On aCURRENT recently updated and worked flawlessly before (CURRENT now: FreeBSD
15.0-CURRENT #5 main-n274014-b2bde8a6d39: Wed Dec  4 22:22:22 CET 2024 amd64), cloning
attempts for 14.2-RELENG ends up in this mess:

# git clone --branch releng/14.2 https://git.freebsd.org/src.git 14.2-RELENG/src/
Cloning into '14.2-RELENG/src'...
error: RPC failed; curl 56 Recv failure: Operation timed out
fatal: expected 'packfile'

This is nasty. The host now in question has an i350 based dual-port NIC - the host's
kernel is very similar to the box I reported the issue first time, both do have
customized kernels (in most cases, I compile several modules like ZFS and
several NETGRAPH modules statically into the kernel - a habit inherited from a small FBSD
project I configured (I wouldn't say developed) which does not allow loadable kernel
modules due to regulations.

I hoped others would stumble over this tripwire in recent CURRENT sources, since the
phenomena and its distribution over a bunch of CURRENT boxes with different OS states
seemingly show different behviour.

And for the record: I also build my ports via poudriere and mostly via make. I also
rebuilt in a two day's marathon all packages via "make -f" - for librewolf, curl and so
on to ensure having latest sources/packages.

(I repeat myself here again, sorry, its for the record).

Will report in on further development and "investigations" 

Kind regards and thanks,

oh