Re: Portscout is stuck

On Sat, Jun 17, 2023, at 04:11, Michael Gmelin wrote:
>> On 17. Jun 2023, at 02:27, Danilo G. Baio <dbaio@freebsd.org> wrote:
>> 
>> 
>> 
>>> On Fri, Jun 16, 2023, at 17:55, Yuri wrote:
>>> It looks like portscout is stuck again.
>>> 
>>> The numbers changed once 27+ hours ago, and didn't change again.
>>> 
>>> Normally portscount updates numbers at least several times a day.
>> 
>> 
>> 
>> The check command was stuck, running for more than a day.
>> 
>> I got a few errors, but we need to increase the debug.
>> 
>> + /usr/bin/lockf -t 1800 /tmp/.portscout.build.lock /usr/local/bin/portscout build
>> Insufficient data for port java/bootstrap-openjdk8: missing version
>> Bad limitw value provided by portconfig variable in port textproc/gspell
>> 
>> + /usr/bin/lockf -t 1800 /tmp/.portscout.build.lock /usr/local/bin/portscout rebuild
>> + /usr/bin/lockf -t 1800 /tmp/.portsout.check.lock /usr/local/bin/portscout check
>>    no element found at line 762, column 99, byte 76060:
>>        <item>
>>          <title><![CDATA[/pptpd/pptpd-1.2.2/pptpd-1.2.2-0.src.rpm]]></title>
>>          <link>https://sourceforge.net/projects/poptop/files/pptpd/pptpd-1.2.2/pptpd-1.2.2-0.src.rpm/d
>>    ==================================================================================================^
>>     at /usr/local/lib/perl5/site_perl/mach/5.32/XML/Parser.pm line 187.
>> 
>>    unclosed token at line 328, column 6, byte 30972:
>>          <files:sf-file-id xmlns:files="https://sourceforge.net/api/files.rdf#">31999936</files:sf-file-id>
>>          <files:extra-info xmlns:files="https://sourceforge.net/api/files.rdf#">empty (Zip archive data)</files:extra-info>
>>          <media:content xmlns:media="http://video.search.yahoo.com/mrss/" type="application/zip;
>>    =====^
>>     at /usr/local/lib/perl5/site_perl/mach/5.32/XML/Parser.pm line 187.
>> 
>> 
>
> Maybe we’re rate limited by one of the big sites (e.g., github). Is/are 
> auth token(s) valid?
>
> In case it’s really hanging, you might be able to tell how far it got 
> based on the db.


I've decreased the timeout of HTTP and FTP requests.

Yesterday I cleaned up some MASTER_SITES that don't resolve DNS.

And the last portscout check I manually did today was faster.

Many entries still need to be fixed/removed; here is the current list so everyone can help.

https://people.freebsd.org/~dbaio/portscout_host_does_not_resolve-2023-06-18.txt

Regards.
-- 
Danilo G. Baio