Re: HMP scheduling on FreeBSD

On 16 Jan 2026, at 04:37, Adrian Chadd <adrian@freebsd.org> wrote:
> 
> I've had to debug systems in the past with the big/little
> cores having different cache line sizes

For those following at home:

This is explicitly disallowed by Arm, but some people didn’t read the docs before shipping SoCs.  The issue is that there are instructions like DC ZVA, which zero a cache line.  If you probe the cache line size and start doing bulk zeroing, then are migrated to a core with a different line size, this is a problem.  You either zero every alternate chunk, or you zero more than you expected to.  Both are bad.

Arm’s memcpy-acceleration instructions have a similar property but are actually designed.  They are expressed as three instructions:

1. Do the annoying unaligned thing at the start.
2. Do the fast bulk operation.
3. Do the annoying unaligned thing at the end.

In small cores, they’re likely to be implemented as these three separate steps.  In big cores, instructions 1 and 3 are likely to be nops and all of the work will be done in instruction 2.  This means that migrating from a big core to a small one after instruction 1, or in the opposite direction after 2 can cause incorrect behaviour.

Arm documents the way to probe whether fixup is required and what the fixup needs to be.  But this means that the MD migration code needs hooks from the MI bit in the scheduler to say ‘I am migrating from this kind of core to this kind, please apply any fixups that are necessary’.

This probably won’t be the last CPU feature that looks like this.

David