[PATCH] simplebus child device probe order control via FDT (motivated by BeagleBone Black)

[Ian Lepore]

On Tue, 2014-03-04 at 01:05 -0500, Patrick Kelsey wrote:
> On Sun, Mar 2, 2014 at 9:38 PM, Warner Losh <imp at bsdimp.com> wrote:
> 
> >
> > On Mar 2, 2014, at 4:56 PM, Patrick Kelsey <kelsey at ieee.org> wrote:
> >
> > > Hi,
> > >
> > > The attached patch (fdt_probe_order_control.patch) allows control over
> > the
> > > probe order of simplebus child devices by using a "probe-order" property
> > in
> > > simplebus child nodes in the .dts file
> >
> > Where is the probe-order property defined? I can't seem to find it in the
> > bindings.
> > Also, I don't think it is necessary. This is a software construct, so
> > doesn't really
> > belong in the dts file. I'd really like to avoid FreeBSD specific hacks in
> > the DTS
> > files.
> >
> 
> It is currently not in any bindings, and has the status of being something
> of my own invention and something possibly not ever used outside of private
> codebases.  I think this is something that would fit conceptually with the
> miscellaneous bindings defined in ePAPR, as it is a rather generic
> property.  As to this being a software construct, I appreciate and support
> the goal of keeping DTS files as pure hardware descriptions (and I'd also
> argue that the way the status property is sometimes used turns it into a
> software construct).
> 
> Let's completely avoid the issue of am I arguing for adding a software
> construct to DTS files by looking at this another way.  Instead of a
> 'probe-order' property that specifies a sort key, I could have the same
> quality of result (same warts and all, as discussed below) with a property
> that indicates whether the device is hardwired or has some sort of
> pluggable interface (a property, say, called 'pluggable').  That would be a
> pure hardware-as-it-is-designed description, and having that information
> would allow me to do a sort that orders non-pluggable ahead of pluggable in
> the probe/attach process.
> 
> 
> >
> > > This was motivated by booting FreeBSD from the eMMC on a BeagleBone
> > Black,
> > > which has a pluggable microSD card slot in addition to the eMMC.  The
> > order
> > > that the mmc units are defined in sys/boot/fdt/dts/am335x.dtsi causes the
> > > pluggable slot to be probed/attached first, so the device name assigned
> > to
> > > the eMMC soldered to the board changes depending on whether there is a
> > card
> > > in the pluggable slot, which makes establishing appropriate /etc/fstab
> > > contents less than convenient.
> >
> > Sounds like you'd like to have some sort of name to instance mapping.
> > The typical way this is solved is by naming the partitions so that ordering
> > doesn't matter. Even if you don't handle this at the filesystem level,
> > which
> > is how others cope, you'd want this to be more of a direct binding rather
> > than an
> > ordering so that you get the effect you want (constant name) directly,
> > rather
> > than as a side-effect of ordering.
> >
> 
> Yes, ideally I'd like to be able to assign a name to every MMC/SD card slot
> (hardwired or otherwise) in a system that depends only on the physical
> location of that slot, and be able to resolve that name to an mmcsd device
> instance.  This works-here-side-effect-based approach is a result of the
> ideal solution seemingly not being within reach given the available
> resources.  Using partition naming/glabel is a fair point, and does address
> the /etc/fstab issue, but it still leaves me short of being able to
> construct a product function that relies on knowing what's where when there
> is no partitioning/filesystem in place, for example, something that boils
> down to 'dd this image to the eMMC [or to the card in the card slot]'.
> Perhaps the answer to that is to build something using devinfo(3)/devd(8)
> to identify what mmcsdX is currently attached to each controller when I
> need to know.
> 
> 
> > If you insist on that, then having a something more like
> > "freesbd,unit-number"
> > property would also accomplish this. But that would only wire the
> > controller unit
> > number, and not the resulting mmcsdX device.
> >
> 
> I understand fully that the utility of this goes as far as having none of
> the hardwired devices sharing controllers with pluggable devices in such a
> way that the controllers can find the attached pluggable devices first.
> Subject to that limitation as it is, this does usefully apply to at least
> one real system now, and it is a behavior I can take advantage of when
> designing new hardware to make developing the associated FreeBSD-based
> product software easier (namely, by not having to cobble together a
> poor-man's udev [which is not saying users of udev are exactly rich],
> and/or worry about how a customer might label the fat partition on an SD
> card they insert).
> 
> More generally, whether this is or isn't a widely useful thing in the end,
> what is the current process for introducing a new property for use in DTS
> files?
> 
> 
> > > By using the "probe-order" property in
> > > sys/boot/fdt/dts/beaglebone-black.dts (see attached
> > > beaglebone_black_mmc_probe_order.patch), I am able to swap the order in
> > > which the mmc units are probed/attached for that board.  This avoids
> > > inappropriate hacking of the mmc definition order in am335x.dtsi, which
> > is
> > > shared among multiple boards.
> > >
> > > This is not a general solution to the problem of wanting stable device
> > > names for hard-wired MMC devices when pluggable slots are present in the
> > > system.  There could, for example, be a multi-slot MMC controller with a
> > > mixture of hard-wired and pluggable devices attached, and this would not
> > > address that case.  However, it does address the needs of BeagleBone
> > Black,
> > > and the mechanism may be of use for similar issues on other platforms.
> >
> > As it isn't a generic solution, I'd be biased against adopting it. What's
> > wrong
> > with using glabel to accomplish this (either with a label specific
> > property, or
> > by using a ufs label and mounting /dev/ufs/foo).
> >
> >
> It is not my intention to lobby for adding this to the kernel at this
> point.  Whether it is widely-enough or only very-narrowly useful I would
> say is currently unknown.  My intention in posting this patch was for
> others on the list(s) that might find it useful to see it and have access
> to it, and to attract thoughtful criticism such as yours.
> 
> I appreciate you taking the time to respond.
> 
> -Patrick


There's a standard property for mmc/sd devices, "non-removable" whose
presence denotes things like soldered-on eMMC parts.  Only one of our
many sdhci drivers supports it right now (imx).  In general the core
part of the driver (dev/sdhci) doesn't have good support for
insert/remove/presence detection that's handled off to the side (whether
it's non-removable or a gpio pin).

That alone doesn't solve the wider problem, though, which I think breaks
down into two pieces.  Let's say you've got two slots, call them left
and right.  You end up with these two problems...

      * Sometimes the right slot is mmcsd0, but it turns into mmcsd1 if
        there's a card in the left slot when I boot; I don't want it to
        change.
      * I want the slot on the left to be named '1' and the right to be
        '0'.

The first problem is easily solved without impacting dts in any way.  We
just wire down the relationship controllerN -> mmcN -> mmcsdN.  This is
exactly equivelent to the old ATA_STATIC_ID option in its effect -- you
don't get to choose the names, but you know they won't change based on
which devices are present.  It could be controlled with a tunable.

It's harder to envision the fix for the second part without adding an
ad-hoc property for the devices.  Even with a property I'm not sure how
easy it would be.

There's been talk of moving mmcsd towards using CAM.  I don't know that
much about cam, but I suspect it rules out doing anything about #1 as
well.  At least, I've never heard of wiring down /dev/daN to a specific
device/slot/whatever.

-- Ian