Two questions on Flattened Device Tree, newbus and device driver attaching
Ian Lepore
ian at FreeBSD.org
Wed Aug 6 14:36:14 UTC 2014
On Tue, 2014-08-05 at 22:31 -0300, Matas Perret Cantoni wrote:
> 2014-08-04 17:00 GMT-03:00 Ian Lepore <ian at freebsd.org>:
>
> > On Sun, 2014-08-03 at 16:41 -0300, Matas Perret Cantoni wrote:
> > > Hello everyone!
> > > I'm working with FreeBSD on the Zedboard (ported by Thomas Skibo
> > > <http://www.thomasskibo.com/zedbsd/>). Currently I'm trying to fully
> > > understand how the Flattened Device Tree (FDT) mechanism works and how it
> > > integrates with FreeBSD. What I've already understand (I think) is:
> > >
> > > (1) how to represent devices, memory mapping/ranges, interrupts,
> > etc...
> > > in a Device Tree Source (DTS) file,
> > > (2) how the newbus framework works, and
> > > (3) how the kernel manages resources, devices and drivers.
> > >
> > > Although I've read all the documents I could find (and some source code)
> > > there are still two things I don't understand:
> > >
> > > *1) The DTS source file and CPUs definition:*
> > >
> > > The DTS file for the zedboard,
> > /release/10.0.0/sys/boot/fdt/dts/zedboard.dts
> > > (here
> > > <
> > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/zedboard.dts?revision=260789&view=markup
> > >),
> > > has the CPU definition all commented out:
> > >
> > > ...
> > > // cpus {
> > > // #address-cells = <1>;
> > > // #size-cells = <0>;
> > > // cpu at 0 {
> > > // device-type = "cpu";
> > > // model = "ARM Cortex-A9";
> > > // };
> > > // };
> > > ...
> > >
> > > This sounds really strange to me! How can the system tell the CPU it's
> > > running on? I'v found some other DTS files for other boards that *do
> > > define* it's
> > > CPUs. For example:
> > >
> > > imx53x.dtsi: (here
> > > <
> > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/imx53x.dtsi?view=markup
> > >
> > > )
> > >
> > > ...
> > > cpus {
> > > #address-cells = <1>;
> > > #size-cells = <0>;
> > >
> > > cpu at 0 {
> > > device_type = "cpu";
> > > compatible = "ARM,MCIMX535";
> > > reg = <0x0>;
> > > d-cache-line-size = <32>;
> > > i-cache-line-size = <32>;
> > > d-cache-size = <0x8000>;
> > > i-cache-size = <0x8000>;
> > > l2-cache-line-size = <32>;
> > > l2-cache-line = <0x40000>;
> > > timebase-frequency = <0>;
> > > bus-frequency = <0>;
> > > clock-frequency = <0>;
> > > };
> > > ...
> > >
> > > or:
> > >
> > > p1020rdb.dts (here
> > > <
> > https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/p1020rdb.dts?view=markup
> > >
> > > )
> > >
> > > ...
> > > cpus {
> > > #address-cells = <1>;
> > > #size-cells = <0>;
> > >
> > > PowerPC,P1020 at 0 {
> > > device_type = "cpu";
> > > reg = <0x0>;
> > > next-level-cache = <&L2>;
> > > };
> > >
> > > PowerPC,P1020 at 1 {
> > > device_type = "cpu";
> > > reg = <0x1>;
> > > next-level-cache = <&L2>;
> > > };
> > > };
> > > ...
> > >
> > > *So my first question is: How can the system tell on wich CPU it running
> > > on? can I add the CPUs definition in my DTS file?*
> > >
> > > 2) The 'compatible' property of a node, finding the driver and attaching
> > it
> > > to the corresponding newbus node: During autoconfiguration the the .dtb
> > > (device tree blob) file is parsed and for each node of the device three
> > the
> > > autoconfiguration systen will create a new newbus node (with
> > > device_add_child()) and then it will find a suitable driver for it and
> > will
> > > attach it:
> > >
> > > */
> > > * This function is the core of the device autoconfiguration
> > > * system. Its purpose is to select a suitable driver for a device and
> > > * then call that driver to initialise the hardware appropriately. The
> > > * driver is selected by calling the DEVICE_PROBE() method of a set of
> > > * candidate drivers and then choosing the driver which returned the
> > > * best value. This driver is then attached to the device using
> > > * device_attach().
> > > *
> > > * The set of suitable drivers is taken from the list of drivers in
> > > * the parent device's devclass. If the device was originally created
> > > * with a specific class name (see device_add_child()), only drivers
> > > * with that name are probed, otherwise all drivers in the devclass
> > > * are probed. If no drivers return successful probe values in the
> > > * parent devclass, the search continues in the parent of that
> > > * devclass (see devclass_get_parent()) if any.
> > > *
> > > * @param dev the device to initialise
> > > */
> > >
> > > int device_probe(device_t dev)
> > >
> > > (I extracted this from here
> > > <
> > https://svnweb.freebsd.org/base/release/10.0.0/sys/kern/subr_bus.c?revision=260789&view=markup
> > >
> > > )
> > >
> > > I believe that the autoconfiguration system uses the
> > > fdt_node_check_compatible() function (from fdtlib
> > > <
> > https://svnweb.freebsd.org/base/release/10.0.0/sys/contrib/libfdt/libfdt.h?revision=260789&view=markup
> > >)
> > > to get the "compatible" property out of each node of the .dtb blob and
> > then
> > > it calls device_probe() to find the best driver and attach it to the
> > > corresponding newbus node. (is this correct?).
> > >
> > > From the ePAPR
> > > <
> > https://www.power.org/wp-content/uploads/2012/06/Power_ePAPR_APPROVED_v1.1.pdf
> > >
> > > standard
> > > we have this:
> > >
> > > 2.3.1 compatible
> > > Property: compatible
> > > Value type: <stringlist>
> > >
> > > Description: The compatible property value consists of one or more
> > strings
> > > that define the specific
> > > programming model for the device. This list of strings should be used by
> > a
> > > client program for
> > > device driver selection. The property value consists of a concatenated
> > list
> > > of null terminated
> > > strings, from most specific to most general. They allow a device to
> > express
> > > its compatibility
> > > with a family of similar devices, potentially allowing a single device
> > > driver to match against
> > > several devices.
> > > The recommended format is “manufacturer,model”, where manufacturer is a
> > > string describing the name of the manufacturer (such as a stock ticker
> > > symbol), and model
> > > specifies the model number.
> > >
> > > Example: *compatible=“fsl,mpc8641-uart”, “ns16550";*
> > >
> > > In this example, an operating system would first try to locate a device
> > > driver that supported
> > > fsl,mpc8641-uart. If a driver was not found, it would then try to locate
> > a
> > > driver that supported
> > > the more general ns16550 device type.
> > >
> > >
> > > *What I don't understand is how the system locates a device driver based
> > on
> > > the compatible property. For example the cpu node's compatible property
> > > ("ARM,MCIMX535") of the imx53x.dtsi example. More precisely: How can the
> > > system relate the string "ARM,MCIMX535" with the actual device driver in
> > > the file system.*
> > >
> > > I hope my questions are clear enough. Many thanks in advance.
> > >
> >
> > For #1, virtually none of our arm code uses the cpu information from the
> > fdt data, because we generally compile a custom kernel specific to each
> > cpu. We've been slowly (very slowly) moving towards a unified kernel
> > that can boot on multiple arm chips (or at least closely related chips
> > within a family), and that will make the cpu info more important some
> > day.
> >
> >
> Oh, I see. Now It is clearer if I take a look in /10.0.0/sys/arm/std.zynq7:
>
> ...
> #
> # std.zynq7 - Generic configuration for Xilinx Zynq-7000 PS.
> #
> # $FreeBSD$
>
> cpu CPU_CORTEXA
> machine arm armv6
> ...
>
>
> So #1 is solved. Thank you both!
>
> For #2, if you're looking for some big master table that maps compatible
> > strings to drivers, no such thing exists.
> >
> > Each driver source has one or more DRIVER_MODULE() macros that provides
> > some information about the driver. One of the things it provides is the
> > parent. The newbus system builds a metadata hierarchy that tracks which
> > drivers have described themselves as potential children on each bus.
> >
> > Usually a device's parent is some sort of bus such as PCI. In an
> > fdt-based system "simplebus" is an abstraction that can represent many
> > different types of buses (such as internal on-chip connections between
> > the cpu and internal devices). A hardware bus such as PCI has ways to
> > query the hardware to see what's connected. In the fdt world, simplebus
> > uses the fdt data to do this query... it looks at all the fdt device
> > entries that are described as its children in the fdt data.
> >
> > For each child in the fdt data, simplebus asks newbus to probe all the
> > drivers whose DRIVER_MODULE() said they could be children of simplebus.
> > The probe() routine of each driver has access to the fdt data for the
> > device simplebus is trying to probe. The driver compares the compatible
> > strings in that data to the compatible strings that it knows how to
> > handle, and returns a success/fail code from probe() to indicate whether
> > or not it is the driver for the device.
> >
> > Sometimes multiple drivers can handle the same hardware, so newbus
> > probes every child driver against every device on the bus. For example
> > a usb keyboard is a pretty generic thing, but a FooStar1000 keyboard
> > might have a special driver that understands extra keys. The generic
> > usb keyboard driver would return BUS_PROBE_GENERIC, and the FooStar1000
> > driver would return BUS_PROBE_SPECIFIC. After probing all potential
> > devices, newbus chooses the one with the highest return value from
> > probe() as being the one most-specific to that hardware. (In reality
> > this doesn't happen much; usually only one driver returns success and
> > all others return an error.)
> >
> > -- Ian
> >
>
> #2 is way more clearer now, but I still can't see a few things:
>
> I can see the function simplebus_attach() calling newbus to create a child,
> probe it and attach it for each of the ftd nodes claiming to be simplebus
> compatible. Here is a snippet from simplebus.c:
>
> /*
> * Walk simple-bus and add direct subordinates as our children.
> */
> dt_node = ofw_bus_get_node(dev);
> for (dt_child = OF_child(dt_node); dt_child != 0;
> dt_child = OF_peer(dt_child)) {
>
> ...
> /* Add newbus device for this FDT node */
> dev_child = device_add_child(dev, NULL, -1);
> }
> ...
> return (bus_generic_attach(dev));
> }
>
>
> So there are two functions from bus.h being called: device_add_child()
> which makes a new newbus child, and bus_generic_attach(dev) which causes
> newbus to probe and attach each of the new childs of the simplebus node.
>
> I can see fdtbus doing the same thing for each node of the fdt's root node.
> Here is a snippet from fdtbus.c:
>
> /*
> * Walk the FDT root node and add top-level devices as our children.
> */
> for (child = OF_child(root); child != 0; child = OF_peer(child)) {
>
> ...
> newbus_device_from_fdt_node(dev, child);
> ...
>
> return (bus_generic_attach(dev));
>
>
> _________
>
> So until here I think I understand how the newbus nodes (devices) are being
> created and how the drivers are being attached to them.
> But when I print the information about system device configuration with
> devinfo I can see that there is a "nexus0" node and a "ofwbus0" node in the
> newbus hierarchy:
>
> root at zedboard:~ # devinfo
> nexus0
> ofwbus0
> simplebus0
> zy7_slcr0
> gic0
> l2cache0
> ....
>
>
> I think that the nexus0 node is created by this function:
>
> /*
> * Determine i/o configuration for a machine.
> */
> static void
> configure_first(void *dummy)
> {
>
> device_add_child(root_bus, "nexus", 0);
> }
>
>
> (which is in the file /10.0.0/sys/arm/arm/autoconf.c and by the way I can't
> find any calling function to configure_first).
>
> And I guess the root node is declared in /10.0.0/sys/kern/subr_bus.c file:
>
> device_t root_bus;
> devclass_t root_devclass;
>
>
> But I can't find out how the "ofwbus0" node is created neither who is doing
> so!
>
> Thanks in advance.
> Best regards, Matias.-
configure_first() and the other functions in autoconf.c are started by
the system init code in mi_startup() in kern/init_main.c. Any part of
the kernel or modules that may be loaded by loader(8) can use a
SYSINIT() macro to declare that they have init code to run at kernel
startup. Linker magic combines all the SYSINIT info into a special
section in the kernel binary, and the mi_startup() code walks through
that info and calls all the functions.
You've already discovered that nexus gets added by configure_first() (I
think I knew that years ago, but had completely forgotten). That causes
nexus to be probed and attached. The probe always succeeds with nexus,
and nexus attach() calls bus_generic_probe(). That causes newbus to
call the identify() routine for every driver that said it might be a
child of nexus. Most drivers don't have an identify() routine, it's
rarely used. I think of it as a sort of pre-probe routine (man 9
DEVICE_IDENTIFY has more info on it.)
The ofwbus driver (dev/ofw/ofwbus.c) has a DRIVER_MODULE() macro that
declares it to be a potential child of nexus, and it has an identify()
routine that calls device_add_child() to add itself as a child of nexus.
I think of this as "forced adoption"... normally it is the parent bus
that does device_add_child(), but in this case the child attaches itself
to the parent. Once ofwbus is added to nexus in this way, its attach()
routine gets called when nexus does bus_generic_attach(), and the ofwbus
attach() then uses the FDT data to attach simplebus and other children
described in the data.
-- Ian
More information about the freebsd-arm
mailing list