sync command runs for hours after upgrade to 11.2
Miroslav Lachman
000.fbsd at quip.cz
Mon Oct 29 12:34:51 UTC 2018
I upgraded about 10 machines from 10.4 to 11.2 this month. The last
machines were upgrade few days ago and one of them exhibits strange
behaviour with sync(8) command.
We have sync in one script for MRTG which is called each 5 minutes.
Every day between 8 - 15h the sync calls are not finished in few seconds
(or fraction of second) as usual but they run for hours and cannot be
killed.
# ps auxww | grep sync
mrtg 22897 23.9 0.0 4232 2048 - R 09:20 69:30.86 sync
mrtg 22186 23.8 0.0 4232 2048 - R 09:15 72:14.36 sync
mrtg 29535 23.2 0.0 4232 2048 - R 10:05 48:05.80 sync
mrtg 46040 22.5 0.0 4232 2048 - R 11:25 22:24.87 sync
mrtg 59472 22.4 0.0 4232 2048 - R 12:40 4:30.85 sync
mrtg 25683 22.2 0.0 4232 2048 - R 09:40 59:20.61 sync
mrtg 51972 22.1 0.0 4232 2048 - R 12:05 12:19.05 sync
mrtg 27983 22.0 0.0 4232 2048 - R 09:55 52:25.62 sync
mrtg 6534 21.8 0.0 4232 2048 - R 08:00 126:28.14 sync
mrtg 49674 21.7 0.0 4232 2048 - R 11:50 15:57.96 sync
mrtg 58720 21.6 0.0 4232 2048 - R 12:35 5:33.81 sync
mrtg 7419 21.5 0.0 4232 2048 - R 08:05 122:03.06 sync
mrtg 16371 21.5 0.0 4232 2048 - R 08:50 88:07.14 sync
mrtg 26454 21.5 0.0 4232 2048 - R 09:45 56:50.32 sync
mrtg 36149 21.5 0.0 4232 2048 - R 10:35 37:25.89 sync
mrtg 37607 21.5 0.0 4232 2048 - R 10:45 34:19.37 sync
mrtg 6588 21.3 0.0 4232 2048 - R 08:00 126:31.50 sync
mrtg 30283 21.1 0.0 4232 2048 - R 10:10 46:04.96 sync
mrtg 11030 21.0 0.0 4232 2048 - R 08:15 114:01.56 sync
mrtg 52724 21.0 0.0 4232 2048 - R 12:10 11:07.51 sync
mrtg 61965 21.0 0.0 4232 2048 - R 12:55 1:23.08 sync
mrtg 35360 20.9 0.0 4232 2048 - R 10:30 38:59.89 sync
mrtg 40764 20.9 0.0 4232 2048 - R 11:05 27:59.73 sync
mrtg 28855 20.8 0.0 4232 2048 - R 10:00 50:07.67 sync
mrtg 33199 20.8 0.0 4232 2048 - R 10:15 44:08.61 sync
mrtg 57984 20.8 0.0 4232 2048 - R 12:30 6:39.88 sync
mrtg 17866 20.7 0.0 4232 2048 - R 09:00 81:18.44 sync
mrtg 57226 20.7 0.0 4232 2048 - R 12:25 7:49.24 sync
mrtg 34610 20.6 0.0 4232 2048 - R 10:25 40:34.89 sync
mrtg 27242 20.5 0.0 4232 2048 - R 09:50 54:37.43 sync
mrtg 8180 20.4 0.0 4232 2048 - R 08:10 117:47.97 sync
mrtg 12422 20.4 0.0 4232 2048 - R 08:25 106:02.85 sync
mrtg 17071 20.4 0.0 4232 2048 - R 08:55 84:41.97 sync
mrtg 48920 20.4 0.0 4232 2048 - R 11:45 17:14.73 sync
mrtg 23521 20.3 0.0 4232 2048 - R 09:25 66:46.53 sync
mrtg 39911 20.3 0.0 4232 2048 - R 11:00 29:28.89 sync
mrtg 56437 20.2 0.0 4232 2048 - R 12:20 8:56.43 sync
mrtg 61234 20.2 0.0 4232 2048 - R 12:50 2:23.54 sync
mrtg 14981 20.1 0.0 4232 2048 - R 08:40 95:00.02 sync
mrtg 46832 20.1 0.0 4232 2048 - R 11:30 21:03.80 sync
mrtg 50447 20.1 0.0 4232 2048 - R 11:55 14:42.42 sync
mrtg 51273 20.1 0.0 4232 2048 - R 12:00 13:33.55 sync
mrtg 14088 20.0 0.0 4232 2048 - R 08:35 98:16.55 sync
mrtg 39126 20.0 0.0 4232 2048 - R 10:55 31:12.26 sync
mrtg 62764 20.0 0.0 4232 2048 - R 13:00 0:21.80 sync
mrtg 15716 19.9 0.0 4232 2048 - R 08:45 91:27.30 sync
mrtg 55646 19.9 0.0 4232 2048 - R 12:15 10:03.76 sync
mrtg 33863 19.7 0.0 4232 2048 - R 10:20 42:24.80 sync
mrtg 48174 19.6 0.0 4232 2048 - R 11:40 18:29.01 sync
mrtg 24297 19.5 0.0 4232 2048 - R 09:30 64:03.59 sync
mrtg 44522 19.5 0.0 4232 2048 - R 11:15 25:02.50 sync
root 61355 19.5 0.0 4232 2048 2 R+ 12:51 2:06.40 sync
mrtg 38358 19.4 0.0 4232 2048 - R 10:50 32:39.67 sync
mrtg 36847 19.1 0.0 4232 2048 - R 10:40 35:44.49 sync
mrtg 45339 19.1 0.0 4232 2048 - R 11:20 23:45.21 sync
mrtg 60318 19.1 0.0 4232 2048 - R 12:45 3:27.70 sync
mrtg 13200 18.8 0.0 4232 2048 - R 08:30 102:11.46 sync
mrtg 18627 18.8 0.0 4232 2048 - R 09:05 78:06.64 sync
mrtg 41498 18.8 0.0 4232 2048 - R 11:10 26:30.43 sync
mrtg 47563 18.8 0.0 4232 2048 - R 11:35 19:42.28 sync
mrtg 11700 18.5 0.0 4232 2048 - R 08:20 110:05.39 sync
mrtg 19368 18.3 0.0 4232 2048 - R 09:10 75:03.78 sync
mrtg 25043 18.0 0.0 4232 2048 - R 09:35 61:45.86 sync
The same script is used on each of our servers without any problem.
This machine is running Apache and PHP without any problems. Web is
still responding even if the system load is about 60:
last pid: 59734; load averages: 61.76, 59.62, 57.24 up 2+12:33:30
12:42:31
387 processes: 64 running, 318 sleeping, 5 zombie
CPU: 14.1% user, 0.0% nice, 85.7% system, 0.1% interrupt, 0.1% idle
Mem: 1006M Active, 9907M Inact, 9056M Laundry, 2703M Wired, 1405M Buf,
1224M Free
Swap: 16G Total, 63M Used, 16G Free
The one thing that can be of some coincidence is rsync backup which is
running in the time between 6 - 17h.
Can anybody help where should I look to solve this problem?
# uname -srmi
FreeBSD 11.2-RELEASE-p4 amd64 GENERIC
/dev/mirror/gm0root on / (ufs, local, noatime)
/dev/mirror/gm0usr on /usr (ufs, local, noatime, soft-updates)
/dev/mirror/gm0var on /var (ufs, local, noatime, nosuid, soft-updates)
/dev/mirror/gm0tmp on /tmp (ufs, local, noatime, noexec, nosuid,
soft-updates)
/dev/mirror/gm0vol0 on /vol0 (ufs, local, noatime, noexec, nosuid,
soft-updates)
/dev/mirror/gm0vol1 on /vol1 (ufs, local, noatime, noexec, nosuid,
soft-updates)
No ZFS involved.
Kind regards
Miroslav Lachman
More information about the freebsd-stable
mailing list