PHYSADDR
Tim Kientzle
tim at kientzle.com
Sat Mar 2 18:10:45 UTC 2013
On Mar 2, 2013, at 9:50 AM, Ian Lepore wrote:
> On Fri, 2013-03-01 at 09:19 -0800, Tim Kientzle wrote:
>> On Mar 1, 2013, at 8:33 AM, Ian Lepore wrote:
>>
>>> On Thu, 2013-02-28 at 09:44 -0800, Tim Kientzle wrote:
>>>> On Feb 28, 2013, at 8:58 AM, Tim Kientzle wrote:
>>>>
>>>>>
>>>>> On Feb 28, 2013, at 8:20 AM, Ian Lepore wrote:
>>>>>
>>>>>> On Wed, 2013-02-27 at 22:27 -0800, Tim Kientzle wrote:
>>>>>>> Starting to look at what is needed for a Generic ARM kernel.
>>>>>>> There's a lot here; I sincerely hope I'm not the only one… ;-)
>>>>>>>
>>>>>>> First up: Can we get rid of PHYSADDR?
>>>>>>>
>>>>>>
>>>>>> If you mean, can we get rid of it within the runtime kernel, I'd say
>>>>>> yes, because we can use a global variable instead which is easily
>>>>>> settable in the entry code.
>>>>>
>>>>> It doesn't seem to be used in the runtime kernel. As far as
>>>>> I can see, it's purely a bootstrap concept.
>>>>>
>>>
>>> Well, it's used to set up the early-init page tables in locore.s then
>>> again to set up the real page tables and related things in initarm() and
>>> then I think it isn't used after that, so I should have said "within the
>>> kernel init". The main point I was getting at is that I don't think we
>>> need a compile-time constant of any sort related to the physical address
>>> at which the kernel is loaded. We can get the physical address of the
>>> entry point (_start) using pc-relative math, and we can get the linker
>>> to give us a constant symbol which is the offset of the _start symbol
>>> within the load image. So we can get the load address at runtime
>>> without guessing what low-order bits to mask.
>>
>> Good. That's the approach I've been eyeing as well; I'm
>> glad you agree that makes sense.
>>
>>>
>>>>>> On the other hand, I've been working
>>>>>> towards getting that value set correctly in the kernel elf headers at
>>>>>> link time.
>>>>
>>>> A-ha! I think I just figured something out.
>>>>
>>>> How would the following work:
>>>>
>>>> * Rename PHYSADDR to KERNPHYSADDR_BASE
>>>>
>>>> * in the top of locore.s, we have a single conditional:
>>>>
>>>> #ifdef KERNPHYSADDR_BASE
>>>> _kpa_base = KERNPHYSADDR_BASE;
>>>> #else
>>>> _kpa_base = pc & 0xF0000000;
>>>> #endif
>>>>
>>>> I think this would DTRT on all of the configurations
>>>> we currently have in SVN.
>>>
>>> Hmm, so the basic assumption is that every SoC will have some physical
>>> memory aligned to a 256mb boundary.
>>
>> I'm assuming this only for ARM systems supported by the GENERIC
>> kernel.
>>
>> Ss far as I can tell, the 256mb boundary assumption works for
>> everything currently in SVN. But the code above does allow you
>> to custom-build a kernel for a system that doesn't satisfy that;
>> you just won't be able to run the GENERIC kernel there.
>>
>> Eventually, it seems we might pull this information out of the
>> FDT, but I'm not yet ready to tackle FDT parsing in locore.S. ;-)
>>
>> Of course, I'm not certain that it will matter when we're done.
>> If we only need PHYSADDR to set up the MMU paging,
>> then we just need to round the _start address down to
>> the next page boundary, don't we?
>>
>>> E.G., there'll never be a SoC with
>>> memory at 0xN1000000 that doesn't have memory at 0xN0000000. I'm not
>>> sure that's a safe assumption given things like the rpi where the gpu
>>> carves off some memory for itself and gives the rest to the arm. It
>>> works with the way rpi carves up the ram, but I could see similar
>>> designs that wouldn't work.
>>>
>>>>
>>>> * We redefine KERNPHYSADDR to be an *offset*
>>>> against _kpa_base. Then we could negotiate a single
>>>> offset (64k?) that works well on many platforms and use
>>>> that for the GENERIC kernel. Boot loaders would be
>>>> responsible for loading the kernel at an address that
>>>> preserves the KPA_OFFSET. The KPA_OFFSET would
>>>> be used in the ELF headers.
>>>>
>>>> Then there are routine code transformations to use _kpa_base
>>>> instead of the compile-time symbol and to use
>>>> _kpa_base + KERNPHYSADDR instead of KERNPHYSADDR.
>>>>
>>>> Does this sound reasonable as a starting point?
>>>>
>>>
>>> There are even more assumptions here about what would work in every
>>> case. Given the basic sequence of boot2->u-boot->ubldr->kernel, every
>>> one of those components has to load the next component at the physical
>>> address it's linked for, and that has to not overlap the addresses being
>>> used by the thing doing the loading. The MMU is off during all of this
>>> so we can't just map our way out of any conflicts.
>>
>> I've given up entirely on the first two of these being generic.
>>
>> I think we have a shot at making the kernel itself generic,
>> and maybe ubldr could be made truly PIC, but the earlier
>> boot stages are always going to be highly board-specific.
>>
>> So conflicts between the various pieces aren't really
>> my primary worry at the moment. Since we'll have to
>> customize the early boot pieces anyway, we can resolve
>> those on a case-by-case basis.
>>
>> The ELF load address vs. where physical memory is located
>> seems the sticky point. Any ideas for getting around that?
>> I feel like I have enough ideas to start chipping away at
>> locore.S if I could just figure out a strategy for the ELF
>> load address issue.
>>
>> (Of course, I still don't understand why the test image I've
>> been playing with seems able to load the same ubldr on
>> both RPi and BBone and that ubldr seems to have no trouble
>> loading a kernel into memory.)
I finally figured this out. RPi echoes memory (so 0x80000000-0xA0000000
is another window on 0x00000000-0x20000000), so my BeagleBone
boot bits linked for the 0x80000000-0x90000000 range actually did
work on RPi. (Which is annoying; I was hoping the different memory
ranges on these two boards would help me to catch memory layout
problems.)
> We may not have control over anything before ubldr. Not everything is
> an eval board that you can easily build a custom u-boot for.
Yep. Full agreement here.
> I'm not sure its safe to assume that (entry-pc & 0xfffff000) is the
> beginning of the kernel; it's true now but need not be so. But that's
> no big deal, we can tweak the linker script to give us the offset of the
> _start symbol so it'll work no matter what.
Patches? ;-)
> The more I ponder the fixed offset concept for a generic kernel the more
> it seems that it might be viable, providing that we require the use of
> ubldr. Then we can make sure that ubldr is always linked at an offset
> that doesn't overlap the kernel load offset. An offset number like 1mb
> or 4mb might work well for the kernel; it leaves lots of space for ubldr
> to be loaded down low in memory. I think putting the loader at a lower
> address than the kernel is required, because there's no upper bound on
> the kernel size (I did a 27MB kernel last month -- it contains a huge
> embedded md filesystem image).
Thanks for pointing this out. I've been consistently putting ubldr
in high memory but hadn't realized kernels varied that much in
size. I'll rethink that.
> This just pushes the real problem into ubldr, though, and that becomes
> the non-generic component that has to be linked at a different physical
> address for each SoC. A kernel could still be loaded without ubldr, but
> it may need to be built in a non-generic way for some platforms.
Among the many things I'd like to try: see if there's a way to build
ubldr as a PIC raw binary (instead of an ELF image). That might
help in a lot of case.
Tim
More information about the freebsd-arm
mailing list