Two questions on Flattened Device Tree, newbus and device driver attaching
Matías Perret Cantoni
perretcantonim at gmail.com
Wed Aug 13 18:15:00 UTC 2014
Thank you very much for your patient, Ian.
Things are much clearer to me now. I can move one with my project.
Regards,
Matias.-
2014-08-06 11:36 GMT-03:00 Ian Lepore <ian at freebsd.org>:
> On Tue, 2014-08-05 at 22:31 -0300, Matas Perret Cantoni wrote:
> > 2014-08-04 17:00 GMT-03:00 Ian Lepore <ian at freebsd.org>:
> >
> > > On Sun, 2014-08-03 at 16:41 -0300, Matas Perret Cantoni wrote:
> > > > Hello everyone!
> > > > I'm working with FreeBSD on the Zedboard (ported by Thomas Skibo
> > > > <http://www.thomasskibo.com/zedbsd/>). Currently I'm trying to fully
> > > > understand how the Flattened Device Tree (FDT) mechanism works and
> how it
> > > > integrates with FreeBSD. What I've already understand (I think) is:
> > > >
> > > > (1) how to represent devices, memory mapping/ranges, interrupts,
> > > etc...
> > > > in a Device Tree Source (DTS) file,
> > > > (2) how the newbus framework works, and
> > > > (3) how the kernel manages resources, devices and drivers.
> > > >
> > > > Although I've read all the documents I could find (and some source
> code)
> > > > there are still two things I don't understand:
> > > >
> > > > *1) The DTS source file and CPUs definition:*
> > > >
> > > > The DTS file for the zedboard,
> > > /release/10.0.0/sys/boot/fdt/dts/zedboard.dts
> > > > (here
> > > > <
> > >
> https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/zedboard.dts?revision=260789&view=markup
> > > >),
> > > > has the CPU definition all commented out:
> > > >
> > > > ...
> > > > // cpus {
> > > > // #address-cells = <1>;
> > > > // #size-cells = <0>;
> > > > // cpu at 0 {
> > > > // device-type = "cpu";
> > > > // model = "ARM Cortex-A9";
> > > > // };
> > > > // };
> > > > ...
> > > >
> > > > This sounds really strange to me! How can the system tell the CPU
> it's
> > > > running on? I'v found some other DTS files for other boards that *do
> > > > define* it's
> > > > CPUs. For example:
> > > >
> > > > imx53x.dtsi: (here
> > > > <
> > >
> https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/imx53x.dtsi?view=markup
> > > >
> > > > )
> > > >
> > > > ...
> > > > cpus {
> > > > #address-cells = <1>;
> > > > #size-cells = <0>;
> > > >
> > > > cpu at 0 {
> > > > device_type = "cpu";
> > > > compatible = "ARM,MCIMX535";
> > > > reg = <0x0>;
> > > > d-cache-line-size = <32>;
> > > > i-cache-line-size = <32>;
> > > > d-cache-size = <0x8000>;
> > > > i-cache-size = <0x8000>;
> > > > l2-cache-line-size = <32>;
> > > > l2-cache-line = <0x40000>;
> > > > timebase-frequency = <0>;
> > > > bus-frequency = <0>;
> > > > clock-frequency = <0>;
> > > > };
> > > > ...
> > > >
> > > > or:
> > > >
> > > > p1020rdb.dts (here
> > > > <
> > >
> https://svnweb.freebsd.org/base/release/10.0.0/sys/boot/fdt/dts/p1020rdb.dts?view=markup
> > > >
> > > > )
> > > >
> > > > ...
> > > > cpus {
> > > > #address-cells = <1>;
> > > > #size-cells = <0>;
> > > >
> > > > PowerPC,P1020 at 0 {
> > > > device_type = "cpu";
> > > > reg = <0x0>;
> > > > next-level-cache = <&L2>;
> > > > };
> > > >
> > > > PowerPC,P1020 at 1 {
> > > > device_type = "cpu";
> > > > reg = <0x1>;
> > > > next-level-cache = <&L2>;
> > > > };
> > > > };
> > > > ...
> > > >
> > > > *So my first question is: How can the system tell on wich CPU it
> running
> > > > on? can I add the CPUs definition in my DTS file?*
> > > >
> > > > 2) The 'compatible' property of a node, finding the driver and
> attaching
> > > it
> > > > to the corresponding newbus node: During autoconfiguration the the
> .dtb
> > > > (device tree blob) file is parsed and for each node of the device
> three
> > > the
> > > > autoconfiguration systen will create a new newbus node (with
> > > > device_add_child()) and then it will find a suitable driver for it
> and
> > > will
> > > > attach it:
> > > >
> > > > */
> > > > * This function is the core of the device autoconfiguration
> > > > * system. Its purpose is to select a suitable driver for a device and
> > > > * then call that driver to initialise the hardware appropriately. The
> > > > * driver is selected by calling the DEVICE_PROBE() method of a set of
> > > > * candidate drivers and then choosing the driver which returned the
> > > > * best value. This driver is then attached to the device using
> > > > * device_attach().
> > > > *
> > > > * The set of suitable drivers is taken from the list of drivers in
> > > > * the parent device's devclass. If the device was originally created
> > > > * with a specific class name (see device_add_child()), only drivers
> > > > * with that name are probed, otherwise all drivers in the devclass
> > > > * are probed. If no drivers return successful probe values in the
> > > > * parent devclass, the search continues in the parent of that
> > > > * devclass (see devclass_get_parent()) if any.
> > > > *
> > > > * @param dev the device to initialise
> > > > */
> > > >
> > > > int device_probe(device_t dev)
> > > >
> > > > (I extracted this from here
> > > > <
> > >
> https://svnweb.freebsd.org/base/release/10.0.0/sys/kern/subr_bus.c?revision=260789&view=markup
> > > >
> > > > )
> > > >
> > > > I believe that the autoconfiguration system uses the
> > > > fdt_node_check_compatible() function (from fdtlib
> > > > <
> > >
> https://svnweb.freebsd.org/base/release/10.0.0/sys/contrib/libfdt/libfdt.h?revision=260789&view=markup
> > > >)
> > > > to get the "compatible" property out of each node of the .dtb blob
> and
> > > then
> > > > it calls device_probe() to find the best driver and attach it to the
> > > > corresponding newbus node. (is this correct?).
> > > >
> > > > From the ePAPR
> > > > <
> > >
> https://www.power.org/wp-content/uploads/2012/06/Power_ePAPR_APPROVED_v1.1.pdf
> > > >
> > > > standard
> > > > we have this:
> > > >
> > > > 2.3.1 compatible
> > > > Property: compatible
> > > > Value type: <stringlist>
> > > >
> > > > Description: The compatible property value consists of one or more
> > > strings
> > > > that define the specific
> > > > programming model for the device. This list of strings should be
> used by
> > > a
> > > > client program for
> > > > device driver selection. The property value consists of a
> concatenated
> > > list
> > > > of null terminated
> > > > strings, from most specific to most general. They allow a device to
> > > express
> > > > its compatibility
> > > > with a family of similar devices, potentially allowing a single
> device
> > > > driver to match against
> > > > several devices.
> > > > The recommended format is “manufacturer,model”, where manufacturer
> is a
> > > > string describing the name of the manufacturer (such as a stock
> ticker
> > > > symbol), and model
> > > > specifies the model number.
> > > >
> > > > Example: *compatible=“fsl,mpc8641-uart”, “ns16550";*
> > > >
> > > > In this example, an operating system would first try to locate a
> device
> > > > driver that supported
> > > > fsl,mpc8641-uart. If a driver was not found, it would then try to
> locate
> > > a
> > > > driver that supported
> > > > the more general ns16550 device type.
> > > >
> > > >
> > > > *What I don't understand is how the system locates a device driver
> based
> > > on
> > > > the compatible property. For example the cpu node's compatible
> property
> > > > ("ARM,MCIMX535") of the imx53x.dtsi example. More precisely: How can
> the
> > > > system relate the string "ARM,MCIMX535" with the actual device
> driver in
> > > > the file system.*
> > > >
> > > > I hope my questions are clear enough. Many thanks in advance.
> > > >
> > >
> > > For #1, virtually none of our arm code uses the cpu information from
> the
> > > fdt data, because we generally compile a custom kernel specific to each
> > > cpu. We've been slowly (very slowly) moving towards a unified kernel
> > > that can boot on multiple arm chips (or at least closely related chips
> > > within a family), and that will make the cpu info more important some
> > > day.
> > >
> > >
> > Oh, I see. Now It is clearer if I take a look in
> /10.0.0/sys/arm/std.zynq7:
> >
> > ...
> > #
> > # std.zynq7 - Generic configuration for Xilinx Zynq-7000 PS.
> > #
> > # $FreeBSD$
> >
> > cpu CPU_CORTEXA
> > machine arm armv6
> > ...
> >
> >
> > So #1 is solved. Thank you both!
> >
> > For #2, if you're looking for some big master table that maps compatible
> > > strings to drivers, no such thing exists.
> > >
> > > Each driver source has one or more DRIVER_MODULE() macros that provides
> > > some information about the driver. One of the things it provides is
> the
> > > parent. The newbus system builds a metadata hierarchy that tracks
> which
> > > drivers have described themselves as potential children on each bus.
> > >
> > > Usually a device's parent is some sort of bus such as PCI. In an
> > > fdt-based system "simplebus" is an abstraction that can represent many
> > > different types of buses (such as internal on-chip connections between
> > > the cpu and internal devices). A hardware bus such as PCI has ways to
> > > query the hardware to see what's connected. In the fdt world,
> simplebus
> > > uses the fdt data to do this query... it looks at all the fdt device
> > > entries that are described as its children in the fdt data.
> > >
> > > For each child in the fdt data, simplebus asks newbus to probe all the
> > > drivers whose DRIVER_MODULE() said they could be children of simplebus.
> > > The probe() routine of each driver has access to the fdt data for the
> > > device simplebus is trying to probe. The driver compares the
> compatible
> > > strings in that data to the compatible strings that it knows how to
> > > handle, and returns a success/fail code from probe() to indicate
> whether
> > > or not it is the driver for the device.
> > >
> > > Sometimes multiple drivers can handle the same hardware, so newbus
> > > probes every child driver against every device on the bus. For example
> > > a usb keyboard is a pretty generic thing, but a FooStar1000 keyboard
> > > might have a special driver that understands extra keys. The generic
> > > usb keyboard driver would return BUS_PROBE_GENERIC, and the FooStar1000
> > > driver would return BUS_PROBE_SPECIFIC. After probing all potential
> > > devices, newbus chooses the one with the highest return value from
> > > probe() as being the one most-specific to that hardware. (In reality
> > > this doesn't happen much; usually only one driver returns success and
> > > all others return an error.)
> > >
> > > -- Ian
> > >
> >
> > #2 is way more clearer now, but I still can't see a few things:
> >
> > I can see the function simplebus_attach() calling newbus to create a
> child,
> > probe it and attach it for each of the ftd nodes claiming to be simplebus
> > compatible. Here is a snippet from simplebus.c:
> >
> > /*
> > * Walk simple-bus and add direct subordinates as our children.
> > */
> > dt_node = ofw_bus_get_node(dev);
> > for (dt_child = OF_child(dt_node); dt_child != 0;
> > dt_child = OF_peer(dt_child)) {
> >
> > ...
> > /* Add newbus device for this FDT node */
> > dev_child = device_add_child(dev, NULL, -1);
> > }
> > ...
> > return (bus_generic_attach(dev));
> > }
> >
> >
> > So there are two functions from bus.h being called: device_add_child()
> > which makes a new newbus child, and bus_generic_attach(dev) which causes
> > newbus to probe and attach each of the new childs of the simplebus node.
> >
> > I can see fdtbus doing the same thing for each node of the fdt's root
> node.
> > Here is a snippet from fdtbus.c:
> >
> > /*
> > * Walk the FDT root node and add top-level devices as our children.
> > */
> > for (child = OF_child(root); child != 0; child = OF_peer(child)) {
> >
> > ...
> > newbus_device_from_fdt_node(dev, child);
> > ...
> >
> > return (bus_generic_attach(dev));
> >
> >
> > _________
> >
> > So until here I think I understand how the newbus nodes (devices) are
> being
> > created and how the drivers are being attached to them.
> > But when I print the information about system device configuration with
> > devinfo I can see that there is a "nexus0" node and a "ofwbus0" node in
> the
> > newbus hierarchy:
> >
> > root at zedboard:~ # devinfo
> > nexus0
> > ofwbus0
> > simplebus0
> > zy7_slcr0
> > gic0
> > l2cache0
> > ....
> >
> >
> > I think that the nexus0 node is created by this function:
> >
> > /*
> > * Determine i/o configuration for a machine.
> > */
> > static void
> > configure_first(void *dummy)
> > {
> >
> > device_add_child(root_bus, "nexus", 0);
> > }
> >
> >
> > (which is in the file /10.0.0/sys/arm/arm/autoconf.c and by the way I
> can't
> > find any calling function to configure_first).
> >
> > And I guess the root node is declared in /10.0.0/sys/kern/subr_bus.c
> file:
> >
> > device_t root_bus;
> > devclass_t root_devclass;
> >
> >
> > But I can't find out how the "ofwbus0" node is created neither who is
> doing
> > so!
> >
> > Thanks in advance.
> > Best regards, Matias.-
>
> configure_first() and the other functions in autoconf.c are started by
> the system init code in mi_startup() in kern/init_main.c. Any part of
> the kernel or modules that may be loaded by loader(8) can use a
> SYSINIT() macro to declare that they have init code to run at kernel
> startup. Linker magic combines all the SYSINIT info into a special
> section in the kernel binary, and the mi_startup() code walks through
> that info and calls all the functions.
>
> You've already discovered that nexus gets added by configure_first() (I
> think I knew that years ago, but had completely forgotten). That causes
> nexus to be probed and attached. The probe always succeeds with nexus,
> and nexus attach() calls bus_generic_probe(). That causes newbus to
> call the identify() routine for every driver that said it might be a
> child of nexus. Most drivers don't have an identify() routine, it's
> rarely used. I think of it as a sort of pre-probe routine (man 9
> DEVICE_IDENTIFY has more info on it.)
>
> The ofwbus driver (dev/ofw/ofwbus.c) has a DRIVER_MODULE() macro that
> declares it to be a potential child of nexus, and it has an identify()
> routine that calls device_add_child() to add itself as a child of nexus.
> I think of this as "forced adoption"... normally it is the parent bus
> that does device_add_child(), but in this case the child attaches itself
> to the parent. Once ofwbus is added to nexus in this way, its attach()
> routine gets called when nexus does bus_generic_attach(), and the ofwbus
> attach() then uses the FDT data to attach simplebus and other children
> described in the data.
>
> -- Ian
>
>
>
More information about the freebsd-arm
mailing list