Re: ftp-master rsync client issue?

This problem (transfer timeout) was solved after I switched back to
'freebsd' tcp functions.

  net.inet.tcp.functions_default=freebsd

Previously this server was set to

  net.inet.tcp.functions_default=rack

Looks like 'rack' or 'bbr' new tcp functions are not stable yet?

Best,

On Wed, Nov 8, 2023 at 10:13 PM Junho Choi <junho.choi@gmail.com> wrote:
>
> Philip,
>
> Thanks for your help. Now rsync is connecting but getting slow and
> almost stopped after a few minutes:
>
> $ /usr/local/bin/rsync -vaHz --delete --progress --stats
> ftp-master.freebsd.org::FreeBSD/ <ftp-pub>/FreeBSD/
> ==============================================================================
> This is the FreeBSD master archive for the ftp area /pub/FreeBSD
> It is available for primary mirrors in order to ensure rapid archive
> updates and distribution.
>
> rsync -vaHz --delete rsync://ftp-master.freebsd.org/FreeBSD/ ~ftp/pub/FreeBSD/
>
> Contact information etc: http://www.FreeBSD.org/doc/en/articles/hubs/
>
> ==============================================================================
> ********    MINIMUM RSYNC VERSION REQUIRED = 3.00 (prefer 3.1.0+)    *********
> ==============================================================================
>
> receiving incremental file list
> ./
> TIMESTAMP
>              35 100%   34.18kB/s    0:00:00 (xfr#1, ir-chk=1006/1013)
> dir.sizes
>           3,110 100%  337.46kB/s    0:00:00 (xfr#2, ir-chk=1005/1013)
> development/
> development/TIMESTAMP
>              35 100%    3.42kB/s    0:00:00 (xfr#3, ir-chk=1027/1041)
> development/tarballs/
> development/tarballs/doc_main.tar.gz
>         851,895   0%  392.42kB/s    0:04:33  ^C^C
>
> Latency is high (expected) and no obvious loss.
>
> > ping -c 10 ftp-master.freebsd.org
> PING ftp-master.freebsd.org (96.47.72.39): 56 data bytes
> 64 bytes from 96.47.72.39: icmp_seq=0 ttl=50 time=206.468 ms
> 64 bytes from 96.47.72.39: icmp_seq=1 ttl=50 time=205.864 ms
> 64 bytes from 96.47.72.39: icmp_seq=2 ttl=50 time=205.773 ms
> 64 bytes from 96.47.72.39: icmp_seq=3 ttl=50 time=205.814 ms
> 64 bytes from 96.47.72.39: icmp_seq=4 ttl=50 time=205.780 ms
> 64 bytes from 96.47.72.39: icmp_seq=5 ttl=50 time=205.958 ms
> 64 bytes from 96.47.72.39: icmp_seq=6 ttl=50 time=205.775 ms
> 64 bytes from 96.47.72.39: icmp_seq=7 ttl=50 time=205.817 ms
> 64 bytes from 96.47.72.39: icmp_seq=8 ttl=50 time=206.643 ms
> 64 bytes from 96.47.72.39: icmp_seq=9 ttl=50 time=205.772 ms
>
> --- ftp-master.freebsd.org ping statistics ---
> 10 packets transmitted, 10 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 205.772/205.966/206.643/0.302 ms
>
> I used to sync with ftp-master all the time before this connection issue.
> I can sync with other mirrors (e.g. ftp.jp) so probably not a network
> issue here.
>
> Any help appreciated.
>
> Best,
>
> On Wed, Nov 8, 2023 at 9:49 AM Philip Paeps <philip@freebsd.org> wrote:
> >
> > On 2023-11-08 05:44:22 (+0800), Junho Choi wrote:
> > > On Tue, Nov 7, 2023 at 4:05 PM Philip Paeps <philip@freebsd.org>
> > > wrote:
> > >> On 2023-11-07 05:40:20 (+0800), Junho Choi wrote:
> > >>> On Tue, Nov 7, 2023 at 1:58 AM Philip Paeps wrote:
> > >>>> On 2023-11-06 19:34:59 (+0800), Junho Choi wrote:
> > >>>>> I am running ftp/rsync.kr.freebsd.org and recently (not sure when
> > >>>>> it
> > >>>>> starts, but at least a few weeks) starting to see it's always
> > >>>>> full:
> > >>>>
> > >>>> Could you let me know a src address I can grep for in the
> > >>>> ftp-master
> > >>>> logs?
> > >>>
> > >>> Source address should be 182.252.182.30 (ftp.kr.freebsd.org)
> > >>
> > >> I believe I've fixed your problem.  You were hitting the wrong
> > >> section
> > >> of the rsyncd.conf with tighter limits.  Please let me know if you
> > >> still
> > >> run into limits.
> > >
> > > Yes it started working again. Thanks!
> >
> > Hooray!
> >
> > > For a full mirror, what's the requirement? Last time I checked it's
> > > too big for me. (I have one server running, not easy to expand)
> >
> > Still too big for you in that case, I think. ;-)
> >
> > We can do single-machine installations, but the machine has to be very
> > beefy.  pkg alone currently needs about 10TB on disk on average, and a
> > bit more during times we have to support three stable/X branches plus
> > main.  The workload also wants 100+GB memory for ZFS to keep up.
> >
> > But no worries.  As I wrote earlier: you're well connected to Japan,
> > where we have two full multi-machine installations (AS9597 and AS7530).
> >
> > Thanks again for keeping ftp.kr.freebsd.org up!
> >
> > Best wishes.
> > Philip
> >
> > --
> > Philip Paeps
> > Senior Reality Engineer
> > Alternative Enterprises
>
>
>
> --
> Junho Choi <junho dot choi at gmail.com> | https://saturnsoft.net



-- 
Junho Choi <junho dot choi at gmail.com> | https://saturnsoft.net