Page Coloring Defines in vm_page.h

[Bruce M Simpson]

On Tue, Jun 24, 2003 at 06:17:48AM -0700, David Schultz wrote:
> The coloring is based on the size and associativity of the cache,
> not its speed relative to the processor's.

Yes. My comments were originally aimed at the fact the article seemed to
imply we were coloring for L1 rather than L2, sorry about the confusion.

Also, I was offline at the weekend; reading the CVS history confirms I'm
on the right track.

> The code makes no assumptions about whether the cache is primary
> or secondary, except that the variables are named that way, and
> the default is optimized for a 4-way 512K cache.  (You could
> determine the actual L2 cache size at boot time via the CPUID
> instruction on PII and later Intel chips if you wanted to.)  You

Yes, that was what I was really getting at; if you look at how PQ_L2_SIZE
is computed from PQ_CACHESIZE, it implies 4-way set associative is
the default optimization. This is fine for Intel chips but not for AMD ones.

The names of the definitions as they stand are perhaps slightly misleading
in this respect. PQ_L2_SIZE might be better renamed PQ_SETSIZE and defined
in terms of PQ_CACHESIZE/PQ_NSETS.

The page queue structures are sized according to the number of cache
colors in use. These structures are allocated once for the lifetime
of the kernel. So auto-tuning for an architecture's L2 cache
organization at boot-time is not out of the question.

Whilst it might be said that people who want to diddle with these defines
will generally know what they're doing anyway, it would be nice to make these
optimizations a wee bit more accessible; also, as we grow to support more
Tier 1 architectures, having page coloring work 'out-of-the-box' across them
might be seen as a desirable feature.

I'd like to know people's feelings on this.

Currently, the only arch in our tree which maintains cache organization
info is sparc64. amd64 reports this info but does not maintain it. 

> really want to color for the L2 cache, though; the L1 instruction
> and data caches are usually small and hard to color for, and
> occasional false sharing there doesn't cost much if you have an
> L2.  There used to be L1 coloring support, but it was removed some
> time ago.

I concur.

> >   - If someone could fill me in on how the primes are arrived at that
> >     would be very helpful.

> Good question.  I know why they're prime numbers slightly smaller
> than a power of two, but I can't figure out how that power of two
> was chosen.

I'd need to work this out before being confident enough to even begin
adding the above feature.

Never knowingly bikeshedded,
BMS