periodic daily takes a very long time to run (14-stable)
- Reply: Ronald Klop : "Re: periodic daily takes a very long time to run (14-stable)"
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Fri, 27 Oct 2023 10:15:25 UTC
Hello list, I'm asking this in order to determine whether I've got something misconfigured or maybe my expectations need recalibrating, or maybe both. context is rpi4b with 8GB and usb3-connected 1tb zfs hard disk. /etc/daily.local has been moved out of the way to try to fix the issue. When periodic daily runs, it seems to take a long time. It's running now and has been running for overan hour so far. top shows PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 21649 root 1 20 0 18M 6668K zio->i 1 0:40 0.53% find There's plenty of resources 93 processes: 1 running, 92 sleeping CPU: 0.1% user, 0.0% nice, 0.5% system, 0.0% interrupt, 99.4% idle Mem: 202M Active, 688M Inact, 69M Laundry, 3079M Wired, 216K Buf, 3780M Free ARC: 975M Total, 552M MFU, 244M MRU, 808K Anon, 21M Header, 158M Other 229M Compressed, 905M Uncompressed, 3.95:1 Ratio Swap: 12G Total, 12G Free but the 'find' thing does heavy disk i/o and everything else reading from or writing to disk slows to a crawl. # cat /etc/periodic.conf daily_status_smart_devices="da0" daily_status_include_submit_mailq="NO" daily_clean_hoststat_enable="NO" daily_queuerun_enable="NO" daily_submit_queuerun="NO" daily_scrub_zfs_enable="YES" daily_scrub_zfs_pools="" daily_scrub_zfs_default_threshold="7" daily_status_ntpd_enable="YES" daily_clean_disks_enable="YES" zpool last scrubbed on the 21st; it's not happening right now there's the following in /etc/sysctl.conf relevant to the context: vfs.zfs.min_auto_ashift=12 # # filesystem vm.pageout_oom_seq=120 vm.pfault_oom_attempts=-1 vm.pageout_update_period=0 # vfs.zfs.resilver_min_time_ms=5000 vfs.zfs.arc.min=536870912 vfs.zfs.arc_max=1073741824 The OOM settings are there for poudriere. # zdb | grep ashift ashift: 12 # ps xx | grep periodic 68824 11 S+ 0:00.00 grep periodic 16115 14 I+ 0:00.00 /bin/sh - /usr/sbin/periodic daily 17425 14 I+ 0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily 17747 14 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily <=== why are there two of these? 19460 14 I+ 0:00.00 /bin/sh /usr/sbin/periodic LOCKED daily <=== 19921 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== also here 21312 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== I'm not sure if it's relevant, but the disk, when the zroot/zfs was initialised, was set to enforce 4k blocks, geli-encrypted swap, geli-encrypted fs. Because it's an rpi4, there's no aes-ni equivalent hardware. But the slowdown doesn't show much in other areas, only a couple, like this problem and when git updates src or ports. smartctl shows no remapped or pending blocks. What else can I do to debug the issue? --