PHYSADDR

[Ian Lepore]

On Wed, 2013-02-27 at 22:27 -0800, Tim Kientzle wrote:
> Starting to look at what is needed for a Generic ARM kernel.
> There's a lot here; I sincerely hope I'm not the only one… ;-)
> 
> First up:  Can we get rid of PHYSADDR?
> 

If you mean, can we get rid of it within the runtime kernel, I'd say
yes, because we can use a global variable instead which is easily
settable in the entry code.  On the other hand, I've been working
towards getting that value set correctly in the kernel elf headers at
link time.

Where to physically load the kernel has to be specified somewhere.
Often what does the loading right now is code that could be standard elf
loader code, but it's full of "the header values are wrong, so do these
bit manipulations to the virtual addresses instead" which I think is
horrible, and on top of that the code that does so is often wrong (but
works by accident with the chips we support).

I'd like to preserve the ability for the kernel elf headers to be
properly interpretted by a standard elf loader, at least as an option.
A loader would always be free to ignore the values in the header and use
its own empirical knowledge of the SoC to choose a load address.

So how would something like ubldr decide on a physical load address for
a generic kernel that had, say, zeroes in the paddr and e_entry fields?

The trampoline code replicates the "loader has empirical knowledge"
problem, because it is in effect another loader.  The address at which
the external loader loads the kernel.gz.tramp has nothing to do with the
address at which the trampoline places the decompressed kernel.  Right
now it's another case of the loader (trampoline) bit-twiddling the
virtual addresses from the kernel's elf headers.

The trampoline is relatively horrible for performance as well, it
decompresses the kernel to one place, then copies it to another.  I'm
curious whether anyone uses it at all, except us at Symmetricom?  I'm
not even sure there's any advantage to us using it.  I suspect it was at
one time better for performance to load a smaller image from sdcard
(because that's slow) and take the hit on the decompress time.  Recently
I enabled the MMU and caches in our low-level bootloader, and I have a
feeling that plus other changes to the boot sd code may mean it's faster
to load an uncompressed kernel now.  I need to do some testing.

I'm not sure we can completely wish away the trampoline, but I think we
can define some conventions that ensure its existance is not a barrier
to having a generic kernel, pretty much just by saying so.  For example,
we could require that a kernel have valid physical addresses in the elf
headers for the trampoline to operate on, and simply state that generic
kernels designed to be loaded anywhere can't use the trampoline.

I had to put some effort recently into figuring out some of the perverse
details of how an at91 SoC gets from first-electrons to a running
kernel, and I took notes at the time.  I started formatting the notes
into a wiki page, but only got the first half of them cleaned up.
Still, there may be some useful info even if the formatting is rough:

https://wiki.freebsd.org/FreeBSD/arm/BootProcess

-- Ian