RPi and powerd, was: Re: RPI4 clock speeds and serial port ( temperatures idle and -j4 buildworld buildkernel )

Wed Mar 24 21:13:29 UTC 2021

On 2021-Mar-23, at 16:15, Mark Millard <marklmi at yahoo.com> wrote:

> On 2021-Mar-23, at 12:57, Mark Millard <marklmi at yahoo.com> wrote:
>> 
>> 
>> On 2021-Mar-23, at 06:56, tech-lists <tech-lists at zyxst.net> wrote:
>> 
>>> Hi,
>>> 
>>> latest build run:
>> 
>> Had a -mcpu=cortext-a72 world and kernel been
>> installed and booted first? Was the system
>> running a world and kernel that had not been
>> tuned for the Cortex-A72?
> 
> I've started an experimental build in my
> -mcpu=cortex-a72 tuned context . . .
> 
>>>>>> World built in 22976 seconds, ncpu: 4, make -j6
>>> --------------------------------------------------------------
>>> 
>>> 6 Hours : 22 Minutes : 56 Seconds
>>> 
>>> created kernel.bin from kernel.full
>>> --------------------------------------------------------------
>>>>>> Kernel build for GENERIC-NODEBUG completed on Mon Mar 22 13:54:53
>>>>>> UTC 2021
>>> --------------------------------------------------------------
>>>>>> Kernel(s)  GENERIC-NODEBUG built in 2086 seconds, ncpu: 4, make -j6
>>> --------------------------------------------------------------
>>> 
>>> 0 Hours : 34 Minutes : 46 Seconds
>>> 
>>> commands used:
>>> 1. cd /usr/src
>>> 2. git pull --ff-only
> 
> I'm simply from-scratch rebuilding what I'm
> already running, based on main 7381bbee29df from
> 2021-03-12:
> 
> # ~/fbsd-based-on-what-freebsd-main.sh 
> merge-base: 7381bbee29df959e88ec59866cf2878263e7f3b2
> merge-base: CommitDate: 2021-03-12 20:29:42 +0000
> def0058cc690 (HEAD -> mm-src) mm-src snapshot for mm's patched build in git context.
> 7381bbee29df (freebsd/main, freebsd/HEAD, pure-src, main) cam: Run all XPT_ASYNC ccbs in a dedicated thread
> FreeBSD RPi4B 14.0-CURRENT FreeBSD 14.0-CURRENT mm-src-n245445-def0058cc690 GENERIC-NODBG  arm64 aarch64 1400005 1400005
> 
>>> 3. make -j10 cleanworld
>>> 4. make -j10 cleandir
>>> 5. make -j10 clean
> 
> My /usr/obj/cortexA72_clang/ was empty at the
> start of the buildworld buildkernel .
> devel/ccache is still not installed.
> 
>> This does not show ccache being cleared out
>> before the below. So the times may be examples
>> of "with ccache benefit" times. The contrast
>> with mine and Bob P.'s times suggests a
>> nice time-benefit can occur.
>> 
>>> 6. make -j6 buildworld
>>> 7. make -j6 buildkernel
> 
> I'm using "-j6 buildworld buildkernel".
> 
>>> here's the src.conf :
>>> https://cloud.zyxst.net/~john/FreeBSD/rpi4-main/src.conf
> 
> I'm using my normal src.conf equivalent, not
> yours. (So the experiment is comparable to my
> normal past experiments in this respect, matching
> what I've reported in the past.)
> 
>> I seem to get intermittent access to
>> https://cloud.zyxst.net/ but got to
>> see the file content eventually.
>> 
>>> relevant rc.conf settings:
>>> powerd_enable="YES"
>>> powerd_flags="-r 1"
> 
> I commented out the config.txt line that assigned
> arm_freq_min and the /etc/sysctl/conf line that
> assigned an arm frequency.
> 
> I put the 2 powerd_* lines above in my /etc/rc.conf .
> 
>>> sysctl.conf settings:
>>> vfs.read_max=128 # default 64 # Cluster read-ahead max block count
> 
> I added the above line to my /etc/sysctl.conf .
> 
>>> config.txt:
>>> kernel=u-boot.bin
>>> over_voltage=6
>>> arm_freq=2000
>>> sdram_freq_min=3200
> 
> Ignoring comment differences, mine matches
> for such lines.
> 
> I rebooted on the basis of all these changes
> before starting the "-j6 buildworld buildkernel"
> style build.
> 
>> Thanks much for the information.
>> 
> 
> So, 6..10(?) of hours from when the
> build started I should have time frames
> to report for a "no ccache benefit"
> build to compare to my past reported
> build times.
> 

Summary: Overall somewhat under 9 hrs historically
turned into somewhat under 15 hrs 35 min, adding
somewhat over 6.5 hours to the time. Not a
configuration that I'm likely to generally use.

The details:

First a reminder of the prior timing that I
reported for my normal configuration of my
normal -j4 buildworld buildkernel in my
usual overclocking style:

World build completed on Thu Mar 11 18:39:37 PST 2021
World built in 29780 seconds, ncpu: 4, make -j4
Kernel build for GENERIC-NODBG completed on Thu Mar 11 19:18:02 PST 2021
Kernel(s)  GENERIC-NODBG built in 2305 seconds, ncpu: 4, make -j4

So a few minutes under 9 hr total for my
normal configuration.

By contrast, for the configuration in this
experiment:

World build completed on Wed Mar 24 06:10:39 PDT 2021
World built in 52030 seconds, ncpu: 4, make -j6
Kernel build for GENERIC-NODBG completed on Wed Mar 24 07:16:50 PDT 2021
Kernel(s)  GENERIC-NODBG built in 3971 seconds, ncpu: 4, make -j6

Notes on some of what may be going on here:

Given the RPi4's memory subsystem and its RAM caching,
my first guess is that the -j6 (instead of -j4) leads
to the RAM caching being far less effective and so RAM
access looks far slower overall, with more waiting for
other threads memory activity (memory bus contention).

In some past experiments, I've seen configurations
where -j3 did buildworld buildkernel faster than
-j4 : before I started setting the RAM clock rate
minimum as well. So this "-jM < -jN" is faster for
the smaller M is not a new type of potential
conclusion and -j4 (or -j3) vs. -j6 may be another
example.

I've also done a type of benchmarking that saturates
what the RPi4 can do --with fewer than 4 cores
involved in order to reach saturation in the
benchmark.

(Benchmark on a scale-of-problem and RAM access
pattern that makes the RAM caching fairly ineffective.
A MACCHIATObin Double Shot also has 4 Cortex-A72
cores and does not have this property for the
benchmark: different RAM caching. Even runninf the
RPi4B and MACCHIATObin Double Shot at the same
arm CPU speed, the MACCHIATObin Double Shot takes
less time for the same work.)

So I might retry the build with, say, -j4 but the
rest being the same (after clearing out the existing
build). That would likely hint at if the hypothesis
has a chance of being correct vs. incorrect.

===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)