PS3 livelock and pmap_remove()

[Nathan Whitehorn]

On 09/06/10 20:22, Nathan Whitehorn wrote:
> Now that my SLB allocation issue is solved, with help with Matthew and
> Alan, I have another VM puzzler.
>
> I have a simple program that tries to use all the memory on the system,
> which isn't very much on the PS3, so I use it to test swap as well.
> Shortly after it begins paging, the system locks up completely. I
> managed to duplicate this behavior on an emulator, and found out what it
> is actually doing. Somehow pmap_remove() is being called with arguments
> pmap_remove(userpmap, PAGE_SIZE, VM_MAXUSER_ADDRESS = USRSTACK). For
> powerpc64, VM_MAXUSER_ADDRESS is 0x7ffffffffffff000, so there are 10^15
> pages to unmap in that range and it was busy taking until the end of
> time unmapping them all.
>
> Here's the trace from KDB:
>
> moea64_remove()
> pmap_remove()
> vm_daemon()
> fork_exit()
> fork_trampoline()
> ----end-----
>
> Does anyone have any idea why this is happening?
>
>   

The culprit here is lines 703-706 of vm/vm_pageout.c:

        if (desired == 0 && nothingwired) {
                pmap_remove(vm_map_pmap(map), vm_map_min(map),
                    vm_map_max(map));
        }

It would be much better if it did this in sections, the way
vm_map_delete does. I'll take a crack at this, though any suggestions
for proper implementation would be appreciated.
-nathan