Re: periodic daily takes a very long time to run (14-stable)

Van: void <void@f-m.fm>
Datum: vrijdag, 27 oktober 2023 18:38
Aan: freebsd-stable@freebsd.org
Onderwerp: Re: periodic daily takes a very long time to run (14-stable)
> 
> On Fri, Oct 27, 2023 at 05:23:49PM +0200, Ronald Klop wrote:
> 
> >Well. You could remove daily_clean_disks_enable="YES" from /etc/periodic.conf. >That saves you the "find". I have never used it before. The default is "off".
> 
> Yes, I'll try that, but it's a very recent addition. The periodic daily problem
> is something thats been happening since even before the machine went from 13-stable to 14-stable. The addition of daily_clean_disks has made
> a bad problem worse, because rather than the problem happening for ~1 hour
> it happens for 3.
> 
> periodic daily should be finished soon, it's moved onto /usr/local/etc/periodic/security/460.pkg-checksum
> 
> edit: it finished, took exactly 3 hrs:
> # date && periodic daily && date
> Fri Oct 27 13:40:15 BST 2023
> Fri Oct 27 16:40:14 BST 2023
> 
> >The list of files it checks for doesn't look very useful to me in 2023. >This does do a full find over *all* directories and files. *every day* ???
> 
> yeah that was sort of my reaction. I've not looked yet for a monthly-clean_disks where I could define an exclude pattern for things like
> ccache. That to me would be useful.
> 
> The reason it was enabled was no more than a "sounds like a good idea"
> variable to enable on a machine that is presently used mainly
> for poudriere, which can tend to generate a lot of core consequential to
> pkg build failures etc.
> 
> >If you have a lot of *.core files you are better of putting this in >sysctl.conf: kern.corefile=/var/tmp/%U.%N.%I.%P.core . >So you know where to look to delete them.
> 
> >Actually my RPI3 has this in cron: @daily  find /var/tmp/ -name "*.core" -mtime +7 -ls -delete .
> 
> thanks for these, have implemented both
> 
> >That is pretty heavy for you setup if you manage to run things in parallel as the RPI4 >has 4CPUs.
> PARALLEL_JOBS=1
> TMPFS=ALL # with excludes for things like llvm & rust & gcc
> 
> >It doesn't help to run daily_cleanup together. ;-)
> 
> Before I started asking about the issue, what I was trying to address/work around was that sometimes (if for example the poudriere run went over 24 hrs) it would run into periodic daily. This caused problems before I made it worse by adding
> daily cleanup LOL
> 
> Now to test periodic daily without daily cleanup...
> 
> How long does yours take?
> -- 
>  
> 
> 
> 


Mine takes:

[root@rpi4 ~]# date && periodic daily && date
Mon Oct 30 14:35:53 CET 2023
Mon Oct 30 14:54:18 CET 2023

=========================================
[root@rpi4 ~]# cat /etc/periodic.conf
daily_output="/var/log/daily.log"
daily_status_security_output="/var/log/daily.log"

weekly_output="/var/log/weekly.log"
weekly_status_security_output="/var/log/weekly.log"

monthly_output="/var/log/monthly.log"
monthly_status_security_output="/var/log/monthly.log"

# 223.backup-zfs
daily_backup_zfs_enable="YES"                           # Backup output from zpool/zfs list
daily_backup_zfs_props_enable="YES"                     # Backup zpool/zfs filesystem properties
daily_backup_zfs_verbose="YES"                          # Report diff between the old and new backups.
# 404.status-zfs
daily_status_zfs_enable="YES"                           # Check ZFS
# 800.scrub-zfs
daily_scrub_zfs_enable="YES"

=========================================

So comparable to your "fixed" daily.

I can probably gain some speed by setting exec/setuid off on some ZFS volumes too.

Regards,
Ronald.