Re: intermittent bsdtar/jemalloc failures

From: Michael Butler via freebsd-current <freebsd-current_at_freebsd.org>
Date: Thu, 07 Oct 2021 20:18:28 UTC 
On 10/7/21 15:39, Konstantin Belousov wrote:
> On Thu, Oct 07, 2021 at 03:28:44PM -0400, Michael Butler via freebsd-current wrote:
>> While building a local release bundle, I sometimes get bsdtar failing (and
>> dumping core) as follows below. Worse, as can be seen below, it doesn't stop
>> the build unless I happen to notice and it yields an incomplete package.
>>
>> a usr/src/sys/netgraph/ng_checksum.h
>> a usr/src/sys/netgraph/ng_message.h
>> a usr/src/sys/netgraph/ng_echo.c
>> a usr/src/sys/netgraph/ng_gif.h
>> <jemalloc>: jemalloc_arena.c:747: Failed assertion:
>> "nstime_compare(&decay->epoch, &time) <= 0"
>> Abort trap (core dumped)
>> sh /usr/src/release/scripts/make-manifest.sh *.txz > MANIFEST
>>
>> What causes this? Build machine is a 2x4-core Intel box with ZFS
>> file-systems all around. I tried stopping NTPD temporarily but the failures
>> persist .. sometimes :-(
>>
>> I've seen this at different points in the archiving process so it doesn't
>> seem specific to building kernel.txz.
> 
> What timecounter do you use? Perhaps show the whole output from
> sysctl kern.timecounter.

imb@vm01:/home/imb> sysctl kern.timecounter
kern.timecounter.tsc_shift: 1
kern.timecounter.smp_tsc_adjust: 0
kern.timecounter.smp_tsc: 1
kern.timecounter.invariant_tsc: 1
kern.timecounter.fast_gettime: 1
kern.timecounter.tick: 1
kern.timecounter.choice: ACPI-fast(900) HPET(950) i8254(0) TSC-low(1000) 
dummy(-1000000)
kern.timecounter.hardware: HPET
kern.timecounter.alloweddeviation: 5
kern.timecounter.timehands_count: 2
kern.timecounter.stepwarnings: 0
kern.timecounter.tc.ACPI-fast.quality: 900
kern.timecounter.tc.ACPI-fast.frequency: 3579545
kern.timecounter.tc.ACPI-fast.counter: 16124892
kern.timecounter.tc.ACPI-fast.mask: 16777215
kern.timecounter.tc.HPET.quality: 950
kern.timecounter.tc.HPET.frequency: 14318180
kern.timecounter.tc.HPET.counter: 1883995229
kern.timecounter.tc.HPET.mask: 4294967295
kern.timecounter.tc.i8254.quality: 0
kern.timecounter.tc.i8254.frequency: 1193182
kern.timecounter.tc.i8254.counter: 57
kern.timecounter.tc.i8254.mask: 65535
kern.timecounter.tc.TSC-low.quality: 1000
kern.timecounter.tc.TSC-low.frequency: 1413153007
kern.timecounter.tc.TSC-low.counter: 2352002295
kern.timecounter.tc.TSC-low.mask: 4294967295

I overrode the default selection of counter-type as NTPD drifted so 
badly as to require stepping almost hourly :-(

So .. I have this in /etc/sysctl.conf ..

kern.timecounter.hardware=HPET

While I hope it wouldn't make a difference, I also have powerd enabled 
in /etc/rc.conf to (marginally) reduce the power-consumption when the 
machine is near-idle. sysctl -a | grep ^dev.cpu | grep freq shows ..

dev.cpu.7.freq_levels: 2834/103000 2333/90000 2000/79000
dev.cpu.7.freq: 2834
dev.cpu.3.freq_levels: 2834/103000 2333/94000 2000/86000
dev.cpu.3.freq: 2834
dev.cpu.5.freq_levels: 2834/103000 2333/90000 2000/79000
dev.cpu.5.freq: 2834
dev.cpu.1.freq_levels: 2834/103000 2333/94000 2000/86000
dev.cpu.1.freq: 2834
dev.cpu.6.freq_levels: 2834/103000 2333/90000 2000/79000
dev.cpu.6.freq: 2834
dev.cpu.2.freq_levels: 2834/103000 2333/94000 2000/86000
dev.cpu.2.freq: 2834
dev.cpu.4.freq_levels: 2834/103000 2333/90000 2000/79000
dev.cpu.4.freq: 2834
dev.cpu.0.freq_levels: 2834/103000 2333/94000 2000/86000
dev.cpu.0.freq: 2834

	imb