About FreeBSD support one more mini-pc

[Ian Lepore]

On Fri, 2014-02-21 at 11:53 -0700, Warner Losh wrote:
> On Feb 20, 2014, at 12:45 PM, Ian Lepore wrote:
> 
> > On Thu, 2014-02-20 at 19:08 +0000, Weiß, Jürgen wrote:
> >>> -----Original Message-----
> >>> From: owner-freebsd-arm at freebsd.org [mailto:owner-freebsd-arm at freebsd.org] On Behalf Of
> >>> Ian Lepore
> >>> Sent: Thursday, February 20, 2014 5:04 AM
> >>> To: Tom Everett
> >>> Cc: freebsd-arm at freebsd.org
> >>> Subject: Re: About FreeBSD support one more mini-pc
> >>> 
> >>> On Wed, 2014-02-19 at 16:12 -0700, Ian Lepore wrote:
> >>>> On Wed, 2014-02-19 at 15:12 -0700, Tom Everett wrote:
> >>>>> Hey Ian, i am just about to ask for a pull from my repo with a working
> >>>>> ubldr for wandboard, if you wanted to take a look:
> >>>>> 
> >>>>> https://github.com/teverett/crochet-
> >>> freebsd/commit/f61660c4134ef495cb5f7de26c2048fba1e65c8f
> >>>>> 
> >>>> 
> >>>> My problem isn't with u-boot, it's in the ubldr->kernel handoff.  I'm
> >>>> using u-boot-2014.01 these days and it needs no patching other than to
> >>>> the wandboard config.h to turn on API and USB and a few other handy
> >>>> things.  But when I boot, sometimes it works but more often it goes like
> >>>> this:
> >>>> 
> [...] 
> >>>> And it hangs there forever.  The 'a' I just added, that shows me that it
> >>>> gets into the kernel, I print that 'a' from the first few instructions
> >>>> of locore.S.
> >>> 
> >>> Follow up on this... it is because I'm using a newer u-boot that has the
> >>> dcache enabled by default.  If I use the "dcache off" command to disable
> >>> it, it boots perfectly every time.  If I leave the cache enabled it
> >>> fails to boot most of the time with symptoms that pretty much exactly
> >>> match stale data in the caches.
> >>> 
> >>> We enable the MMU in locore.S without invalidating old cache contents
> >>> first, and that appears to be a bad thing.
> >>> 
> >>> -- Ian
> >>> 
> >> 
> >> Hm, I use an u-boot version from early December, which has already
> >> enabled the L1 Dcache. I did not experience any problems with that
> >> version. On Jan 29th code was committed to the u-boot repository to
> >> enable the L2 cache. I have not checked that version yet.
> >> 
> >> But without a Dcache invalidate, I had problems to start the second
> >> core.
> > 
> > I think it is the enabling of the L2 cache that's causing the problem,
> > because I added your code for invalidating L1 idcache to the first
> > processor startup path, and that didn't help.  
> > 
> > Flushing the L2 very early in startup is problematic, because we don't
> > know its register addresses until we can read the fdt data (we don't
> > even know what kind of l2 it is), and processing fdt while still running
> > in physical address mode with only pc-relative addressing allowed is
> > more or less impossible.
> 
> Do we have the tools to find out at least the kernel part of memory? We know the PA and VA of kernel area, and symbols like start and end would be enough to bound it, no? At least that would get us up and running to the point where we can parse the FDT to get its data... Or is there something more subtle afoot here....

The problem is that until the MMU is turned on we can't access normal
data, bss, or even call a named function.  You can only run position-
independant code, and that doesn't mean PLT and GOT stuff, it means you
can only get the address of something as an offset from the current PC.
The FDT code isn't amenable to running like that.

However, I think the suggestion offered by Juergen is a good one:  if
the MMU is on at entry, then our locore.S code needs to treat it as a
mapping (set ttb) change and do the cache maintenance sequences you
would do for that, as opposed to the sequences required to turn it on
for the first time.  Hopefully that's all that's needed and the L2 cache
isn't involved in the problem.  I'm going to give that a try this
weekend.

-- Ian