lspci XHCI "Memory at" for RPi4 (u-boot based context): FreeBSD vs. ubuntu vs. "Memory behind bridge" addresses

[Robert Crowston]

Yes, the 0xf8000000 is the PCI start address and the 0x600000000 is the CPU start address. It was (and perhaps is) a source of much confusion to me as well, especially as (AFAIK) FreeBSD did have any other examples of PCI drivers using distinct addressing schemes for upstream vs downstream DMA access. Therefore we only supported it "theoretically": in the sense that, as far as I know, that no one ever had to think about it before.

It may be that the tooling or diagnostics could be cleaned up to give the CPU view.

— RHC.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, 4 October 2020 02:27, Mark Millard <marklmi at yahoo.com> wrote:

>
>
> On 2020-Oct-3, at 13:01, Mark Millard <marklmi atyahoo.com> wrote:
>
> > [Be warned that I'm reporting differences that I do not
> > understand. I may end up being told "nothing of interest
> > in these differences".]
> > In the later lspci -v output diff, there is the following for
> > the XHCI (-: FreeBSD, +: ubuntu):
> >
> > -         Memory at f8000000 (64-bit, non-prefetchable)
> >
> >
> >
> > . . .
> >
> > -         Memory at 600000000 (64-bit, non-prefetchable) [size=4K]
> >
> >
>
> Looks like this is the two lspci's disagreeing about what
> to show for "Memory at":
>
> ubuntu "fdt print /" output:
>
> pcie at 7d500000 {
> . . .
> #address-cells = <0x00000003>;
> . . .
> #size-cells = <0x00000002>;
> . . .
> ranges = <0x02000000 0x00000000 0xf8000000 0x00000006 0x00000000 0x00000000 0x04000000>;
>
> Note that both the figures are in ranges:
>
> 0x00000000 0xf8000000
> and:
> 0x00000006 0x00000000
>
> . As for FreeBSD, the same is true:
>
> pcie at 7d500000 {
> #address-cells = <0x3>;
> . . .
> #size-cells = <0x2>;
> . . .
> ranges = <0x2000000 0x0 0xf8000000 0x6 0x0 0x0 0x4000000>;
> . . .
>
> So:
>
> 0x0 0xf8000000
> and:
> 0x6 0x0
>
> > Odder(?) is comparison of the FreeBSD address with what is
> > listed by both OS's for "Memory behind bridge":
> >
> >        Memory behind bridge: f8000000-f80fffff [size=1M]
> >
> >
> > . . .
> >
> > -         Memory at f8000000 (64-bit, non-prefetchable)
> >
> >
> >
> > ubuntu gets a distinct address (600000000) and FreeBSD gets
> > an exact match to the start of the "Memory behind bridge".
> > (There may be a possible 32-bit address vs. 64-bit address
> > distinction?)
>
> Looks like this part is not odd.
>
> > [There are also IRQ differences (81/82 for FreeBSD; 41/42 for
> > ubuntu) and ubuntu lists the kernel drivers used.]
> > For reference:
> >
> > diff -u ~/rpi4-lspci_v-*.txt | more
> >
> > ====================================
> >
> > --- /root/rpi4-lspci_v-fbsd.txt 2020-10-03 12:19:57.162261000 -0700
> > +++ /root/rpi4-lspci_v-ubuntu.txt 2020-10-03 12:21:01.448646000 -0700
> > @@ -1,5 +1,5 @@
> > -00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2711 (prog-if 00 [Normal decode])
> >
> > -         Flags: bus master, fast devsel, latency 0, IRQ 81
> >
> >
> >
> > +00:00.0 PCI bridge: Broadcom Inc. and subsidiaries Device 2711 (rev 10) (prog-if 00 [Normal decode])
> >
> > -         Flags: bus master, fast devsel, latency 0, IRQ 41
> >          Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
> >          I/O behind bridge: 00000000-00000fff [size=4K]
> >          Memory behind bridge: f8000000-f80fffff [size=1M]
> >
> >
> >
> > @@ -9,13 +9,15 @@
> > Capabilities: [100] Advanced Error Reporting
> > Capabilities: [180] Vendor Specific Information: ID=0000 Rev=0 Len=028 <?>
> > Capabilities: [240] L1 PM Substates
> >
> > -         Kernel driver in use: pcieport
> >
> >
> >
> > 01:00.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host Controller (rev 01) (prog-if 30 [XHCI])
> > Subsystem: VIA Technologies, Inc. VL805 USB 3.0 Host Controller
> >
> > -         Flags: bus master, fast devsel, latency 0, IRQ 82
> >
> >
> > -         Memory at f8000000 (64-bit, non-prefetchable)
> >
> >
> >
> > -         Flags: bus master, fast devsel, latency 0, IRQ 42
> >
> >
> > -         Memory at 600000000 (64-bit, non-prefetchable) [size=4K]
> >          Capabilities: [80] Power Management version 3
> >          Capabilities: [90] MSI: Enable+ Count=1/4 Maskable- 64bit+
> >          Capabilities: [c4] Express Endpoint, MSI 00
> >          Capabilities: [100] Advanced Error Reporting
> >
> >
> > -         Kernel driver in use: xhci_hcd
> >
> >
> >
> > Is this odd? Expected/reasonable?
>
> f8000000 vs. 600000000 Looks to be an odd lspci -v variation,
> with FreeBSD apparently showing the figure as if -b had been
> supplied on the command line ("Bus-centric view"): ubuntu
> shows f8000000 when the -b is supplied.
>
> ===============================================================================================================================================================================================================================
>
> Mark Millard
> marklmi at yahoo.com
> ( dsl-only.net went
> away in early 2018-Mar)