Writing a PCIe-Driver

[Farhan Khan]

Hi Rolf,

I have been reading up on this code myself. I do not claim expertise,
so hopefully others can chim in here if I make a mistake. On the WiFi
side of things, I've pretty consistently I've seen the following
routine:

A call to pci_find_cap(9), typically as follows from the WiFi code
I've looked into:
pci_find_cap(dev, PCIY_EXPRESS, &sc->sc_cap_off);

A write to clear the PCI Retry timeout, by writing 0x41 to the device
with pci_write_config(9):
pci_write_config(dev, 0x41, 0, 1);

Obtain the base address registers using PCIR_BAR(), an abstraction of
bus_alloc_resource(), and store that value.
rid = PCIR_BAR(0)

Allocate IO DMA memory to the device:
sc->sc_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE);
Most drivers check this return value and return ENXIO if the value is NULL.

I do not fully understand what rman_get_bustag(9) does, but I see this
called on all drivers I've looked at as follows:
sc->sc_st = rman_get_bustag(sc->sc_mem);
sc->sc_sh = rman_get_bushandle(sc->sc_mem);

Typically I see Interrupt handler code setup here. In the case of
rtwn(4) and iwm(4). This might not be relevant to your driver,
depending on what it does. I see a call to pci_alloc_msi as follows:
rid = 1;
if (pci_alloc_msi(dev, &rid) == 0)
rid = 1;
else
rid = 0;
pci->irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_ACTIVE |
(rid != 0 ? 0 : RF_SHAREABLE));

Then assign an interrupt handler function as follows so you know to
read from the DMA.
error = bus_setup_intr(dev, sc->sc_irq, INTR_TYPE_NET | INTR_MPSAFE,
NULL, your_function_here, sc, &sc->sc_ih);

Capture the DMA tag as follows:
sc->sc_dmat = bus_get_dma_tag(sc->sc_dev);

>From here, I see the following sequence:
1. bus_dma_tag_create(), where the first argument is the DMA tag
(sc->sc_dmat), and the last argument is the memory it is mapped to.
2. bus_dmamem_alloc()
3. bus_dmamap_load()
4. bus_dmamap_sync()

At this point, you may need to initialize the interrupts, depending on
your hardware. If/when the interrupt arrives, you need to determine
what type it is and read from the DMA memory, as allocated in the DMA
sequence. On every assign and allocation you should also do
error-checking on all return values.

That is my understanding. Again, someone else can jump in if I made a
mistake. I read rtwn(4), iwm(4) and iwi(4) while writing this and they
are excellent references.

Hope this helps!
--
Farhan Khan
PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE


On Sun, Sep 23, 2018 at 10:46 PM Dr. Rolf Jansen <rj at obsigna.com> wrote:
>
> Hello,
>
> A couple of years ago, with the valuable help of people on this list, I managed to write a PCI driver for the National Instruments DAQ card, NI PCI-6251. The driver was kept very simple, only map the BAR’s and some DMA memory into the user space. The DAC/ADC and DIO is then all controlled from user space by writing command codes to certain offsets from the mapped BAR’s, and reading data from the mapped DMA memory and/or from other given offsets.
>
> See: How to map device addresses into user space
> https://lists.freebsd.org/pipermail/freebsd-drivers/2013-January/thread.html
>
>
> Now, I need to do exactly the same for a PCIe card, namely the NI PCIe-6351. However, I even cannot see where to start. For PCI cards, the respective chapter 11 in the FreeBSD Architecture Handbook was quite helpful, for getting started, in no time, I got the PCI card probed and attached.
>
> https://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/pci.html
>
> The dumb approach, probing the device ID and attaching the PCI(e) driver to the PCIe device in a similar fashion didn’t work - I didn’t expect this either, however, where would I start? How do I attach my driver to a PCIe card on FreeBSD 11 or 12?
>
> Please can somebody push me into the right direction. Once I got the driver attached to the card, I guess, I would be able to adapt the mem-mapping code of my old driver for the new card.
>
> Best regards
>
> Rolf
> _______________________________________________
> freebsd-drivers at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
> To unsubscribe, send any mail to "freebsd-drivers-unsubscribe at freebsd.org"