Merging 64 bit changes to -HEAD - part 2

[M. Warner Losh]

In message: <4C1CEDFB.9070802 at cs.rice.edu>
            Alan Cox <alc at cs.rice.edu> writes:
: On 6/19/2010 5:18 AM, Juli Mallett wrote:
: > On Fri, Jun 18, 2010 at 12:41, Neel Natu<neelnatu at gmail.com>  wrote:
: >    
: >> Hi JC,
: >>
: >> But what you really want here is to eliminate the intr_disable() and
: >> intr_restore() and keep sched_pin() and  sched_unpin().
: >>      
: > Are you sure?  I'm not.  By disabling interrupts we only have to
: > ensure that the fault path on any address we might access within those
: > routines doesn't need to use the large memory map.  This isn't
: > trivial, but I think we can acquire a reasonable confidence about it.
: > If we merely pin, we have to ensure that nothing else that can run
: > (including interrupts and threads that run via preemption) that would
: > access the large memory map — given that this includes routines like
: > pmap_zero_page, I think there's good reason for caution.  Disabling
: > interrupts is more conservative, but I think rightly-so.  I may be
: > mistaken.
: >    
: 
: You're not mistaken.  See, for example, the i386 pmap_zero_page().
: Pinning by itself is insufficient because a pinned thread can be
: preempted, and the thread that then runs (on the same processor) could
: call pmap_zero_page().  So, pinning must be combined with a
: per-processor mutex.
:
: I can imagine that blocking interrupts on mips is cheaper than the
: combination of pinning and a mutex.  However, do you want to have
: interrupts blocked for the time it takes to read 4KB from DRAM and
: write 4KB to DRAM for pmap_copy_page()?  Ultimately, that's the
: question that you need to answer.

Blocking interrupts on modern mips is literally just a EI/DI pair of
instructions.  I'm not sure that we implement it that way yet,
though, since we haven't gone through those sorts of things and
optimized them based on what the processor can do.  Even without those
special instructions, blocking all interrupts is cheap.

For < 256MB systems, we don't need any locking at all, since there's
no temporary mappings involved (I'm sorry, but I've lost the context
of the original patch).  For an n64-kernel we could also optimize for
all sane sizes of memory (59-bits or 512TB of space if I'm counting
bits correctly this morning) and do the copy.

I'm not sure if we'd need any interlock for caches, however.  So long
as the target range isn't mapped, and the source page isn't changing,
I think we dodge the cache aliasing problem (even with the direct-map
access, since that too is a VA access).  But to be honest, I've not
given this more than a moment or two of thought...

Warner