Re: Portscout is stuck

> On Jun 19, 2023, at 2:10 AM, Danilo G. Baio <dbaio@FreeBSD.org> wrote:
> 
> 
> 
> On Sat, Jun 17, 2023, at 04:11, Michael Gmelin wrote:
>>> On 17. Jun 2023, at 02:27, Danilo G. Baio <dbaio@freebsd.org> wrote:
>>> 
>>> 
>>> 
>>>> On Fri, Jun 16, 2023, at 17:55, Yuri wrote:
>>>> It looks like portscout is stuck again.
>>>> 
>>>> The numbers changed once 27+ hours ago, and didn't change again.
>>>> 
>>>> Normally portscount updates numbers at least several times a day.
>>> 
>>> 
>>> 
>>> The check command was stuck, running for more than a day.
>>> 
>>> I got a few errors, but we need to increase the debug.
>>> 
>>> + /usr/bin/lockf -t 1800 /tmp/.portscout.build.lock /usr/local/bin/portscout build
>>> Insufficient data for port java/bootstrap-openjdk8: missing version
>>> Bad limitw value provided by portconfig variable in port textproc/gspell
>>> 
>>> + /usr/bin/lockf -t 1800 /tmp/.portscout.build.lock /usr/local/bin/portscout rebuild
>>> + /usr/bin/lockf -t 1800 /tmp/.portsout.check.lock /usr/local/bin/portscout check
>>>   no element found at line 762, column 99, byte 76060:
>>>       <item>
>>>         <title><![CDATA[/pptpd/pptpd-1.2.2/pptpd-1.2.2-0.src.rpm]]></title>
>>>         <link>https://sourceforge.net/projects/poptop/files/pptpd/pptpd-1.2.2/pptpd-1.2.2-0.src.rpm/d
>>>   ==================================================================================================^
>>>    at /usr/local/lib/perl5/site_perl/mach/5.32/XML/Parser.pm line 187.
>>> 
>>>   unclosed token at line 328, column 6, byte 30972:
>>>         <files:sf-file-id xmlns:files="https://sourceforge.net/api/files.rdf#">31999936</files:sf-file-id>
>>>         <files:extra-info xmlns:files="https://sourceforge.net/api/files.rdf#">empty (Zip archive data)</files:extra-info>
>>>         <media:content xmlns:media="http://video.search.yahoo.com/mrss/" type="application/zip;
>>>   =====^
>>>    at /usr/local/lib/perl5/site_perl/mach/5.32/XML/Parser.pm line 187.
>>> 
>>> 
>> 
>> Maybe we’re rate limited by one of the big sites (e.g., github). Is/are
>> auth token(s) valid?
>> 
>> In case it’s really hanging, you might be able to tell how far it got
>> based on the db.
> 
> 
> I've decreased the timeout of HTTP and FTP requests.
> 
> Yesterday I cleaned up some MASTER_SITES that don't resolve DNS.
> 
> And the last portscout check I manually did today was faster.
> 
> Many entries still need to be fixed/removed; here is the current list so everyone can help.
> 
> https://people.freebsd.org/~dbaio/portscout_host_does_not_resolve-2023-06-18.txt
Thanks for the detailed debugging. I will try to see how many I can axe.


> Regards.
> --
> Danilo G. Baio
>