Multiple virtual mappings considered harmful on ARM
tinguely at casselton.net
Wed Dec 31 14:32:18 UTC 2008
> Marcel Moolenaar wrote:
> > On Dec 19, 2008, at 6:07 AM, Grzegorz Bernacki wrote:
> >> 2. Root cause.
> >> The root cause of the problem is additional virtual mapping of read/write
> >> buffers at cluster read/write (sys/kern/vfs_cluster.c, cluster_rbuild(),
> >> cluster_wbuild(). Buffers for sequential read/write operation are
> >> concatenated
> >> and sent to device as one big buffer. Concatenation of buffers uses
> >> pmap_qenter(), which puts *additional* mapping in the KVA for physical
> >> area
> >> already mapped. For each buffer we extract pages it contains and then
> >> all the
> >> pages from all the buffers are mapped into new virtual address of new
> >> buffer.
> >> So we end up with at least two virtual addresses for each page.
> > Could this also affect I-cache coherency by virtue of not
> > flushing the D-cache properly before synchronizing the
> > I-cache, as you mention reading?
> I am not sure. I can't think of scenario which might lead to I-cache incoherency.
> Have you experienced any issues with I-cache which might be related described problem?
If would be surprised for you to see an I-cache problems.
pmap_qenter() writes back any managed cache pages (already under the control
of a pv_entry) and then calls pmap_kenter_internal(), which tries to writeback
and invalid any previously mapped pages. At the end of the pmap_qenter() the
caches are clean.
The problem is the new mapping is added with cache turned on, and the old
mapping's page table caching is not modified either, so if either side
modifies the page, there can be this caching problem.
I think that the pmap_kenter_internal() should become a special kind of
managed page (controlled by a pv_entry and a special flag, (I called PVF_UNMAN)
that runs through the pmap_fix_cache(), so the mappings for the shared page
should have the cache turned off. There are special checks
The less obvious question is should PG_UNMANAGED pages be simularly managed?
Or put another way, could the PG_UNMANAGED page be remapped by the
I/O caching programs? My guts says yes. Doing so also forces BUS_DMA_COHERENT
pages to turn off the caches on the share page.
I have a crude workup of my idea, but I have to admit, I don't have the
equipment to even try to compile it. There concerns about pv_entry use
increase, there is lock issues, performance, and even could we cause a
problem trying get a pv_entry at the wrong time, such as interrupt. Maybe
the kernel pv_entrys need to come from a special pool.
I have some crude ideas to flush the caches less often in the ARM11's
virtual index / physical tag caches and ideas for the multicore
physical index / physical tag cache.
More information about the freebsd-arm