svn commit: r318021 - in head/sys/arm: arm include

[Michal Meloun]

On 09.05.2017 17:09, Andrew Turner wrote:
>> On 9 May 2017, at 13:40, Michal Meloun <melounmichal at gmail.com> wrote:
>>
>>
>>
>> On 09.05.2017 13:34, Andrew Turner wrote:
>>>> On 9 May 2017, at 12:05, Michal Meloun <mmel at FreeBSD.org> wrote:
>>>>
>>>> Author: mmel
>>>> Date: Tue May  9 11:05:32 2017
>>>> New Revision: 318021
>>>> URL: https://svnweb.freebsd.org/changeset/base/318021
>>>>
>>>> Log:
>>>> Introduce pmap_remap_vm_attr(),
>>>> it allows to remap one VM memattr class to another.
>>>>
>>>> This function is intent to be used as workaround for various SoC bugs,
>>>> mainly access ordering/sequencing related bugs in crossbar fabric.
>>> This seems quite heavy handed to change the attribute for all memory of a given type.
>> Yes, exactly.  See comment in D10218 -
>> /*
>> * Workaround for Marvell Armada38X family HW issue
>> * between Cortex-A9 CPUs and on-chip devices that may
>> * cause hang on heavy load.
>> * To avoid that, map all registers including PCIe IO
>> * as strongly ordered instead of device memory.
>> */
> I don’t think it’s been answered if this is just for PCIe, or all devices.
Well, I have not access to Armada errata, so I can't give you more exact
answer.
But this kind of bugs (ordering/sequencing problems  in crossbar fabric)
are not that uncommon,
I think that some early versions of NPX QorIQ have exactly same problem
(all devices must be mapped as SO),
i.MX6 can hangs if devices are not mapped with shared flag...

>
>>> Other architectures have pmap_change_attr to change the attribute on a specific range of memory.
>> Right. Problem is that I don't known any method how we can change 
>> memory attribute for live memory in SMP system,
>> without hitting undefined behavior.
> I would expect drivers to change the attributes early, before they access the memory.
So then why driver does not request right type of memory at allocation?
But yes, current situation around pmap_mapdev(), bus_space_map(), 
bus_activate_resource() and bus_map_resource() is
not an optimal. At least for systems where devices must be mapped with
different attributes - e.g. normal devices, PCI config space,
memory mapped PCI I/O bars,  normal/prefetchable memory bars.
I think that we needs more *common* VM_MEMATTR_  classes.

>  We could also use smp_rendezvous to ensure nothing else is running, this will have a performance code, however I would not expect pmap_change_attr to be used in the fast path.
>
pmap_change_attr() must work very early, so smp_rendezvous() can be
problematic.
But, for me, the problematic part is cache - mainly transition from
cached to uncached memory.
I'm not sure, if I known always working sequence of mapping/cache
operations witch ensure this transition in
architecturally clean way.
Next think is that pmap_change_attr() is used very rarely and probably
only as workaround for limited bus_activate_resource()
functionality.
 
Michal