Writing a PCIe-Driver

Wed Sep 26 17:04:52 UTC 2018

I got it working. Actually, it was more easy than expected.

I was looking at the output of pciconf -lcbv and there I confused the card identifier (card=0x742f1093) with the chip identifier (chip=0xc4c41093) of the NI PCIe-6351 board, and in the first place I took the first one for the probe function of my driver, vendor = 0x1093 (National Instruments) device = 0x742f (wrong). Looking closer at the respective output of the old PCI board NI PCI-6251, I saw, that this one doesn’t got a card ID assigned (0x00000000). I only need to use the corrected device ID 0xc4c4 in the probe routine of my driver, and now I got it attached:

   # kldload ./cydaq.ko; tail /var/log/messages
   ...
   Sep 26 12:39:25 CyStat-220 kernel: cydaq0: CyDAQ found DAQ-Board NI PCIe-6351; VendorID 0x1093, DeviceID 0xC4C4
   Sep 26 12:39:25 CyStat-220 kernel: cydaq0: <DAQ-Board NI PCIe-6351> mem 0xdf000000-0xdf03ffff irq 18 at device 0.0 on pci2
   Sep 26 12:39:25 CyStat-220 kernel: cydaq0: CyDAQ PCI device driver has been loaded

   # kldstat
   Id Refs Address            Size     Name
   ...
   14    1 0xffffffff827da000 1700     cydaq.ko

   # sysctl -a | grep cydaq
   cydaq.AO_dma_buffer_size: 50331648
   cydaq.AI_dma_buffer_size: 268435456
   dev.cydaq.0.%parent: pci2
   dev.cydaq.0.%pnpinfo: vendor=0x1093 device=0xc4c4 subvendor=0x1093 subdevice=0x742f class=0x118000
   dev.cydaq.0.%location: slot=0 function=0 dbsf=pci0:2:0:0 handle=\_SB_.PCI0.RP11.PXSX
   dev.cydaq.0.%driver: cydaq
   dev.cydaq.0.%desc: DAQ-Board NI PCIe-6351
   dev.cydaq.%parent: 

Everything looks good now. As I said already, the actual DAC/ADC/DIO is all controlled from user space. The driver only provides the base addresses of the DAQ board and of the reserved DMA ranges via respective ioctl calls. Some measurement tasks are very math intensive, and the kernel is anyway not the right place for doing this.

Best regards

Rolf

> Am 24.09.2018 um 15:30 schrieb Dr. Rolf Jansen <rj at obsigna.com>:
> 
> Farhan,
> 
> Thank you very much for sharing your findings. I will also look into the code, and I started with checking out the FreeBSD source tree to my development machine, so examine the code of the various drivers is more easy. Anyway, the outline which you revealed is already a good starting point, and it will help me a lot to find my way in the jungle.
> 
> Best regards
> 
> Rolf
> 
>> Am 24.09.2018 um 01:53 schrieb Farhan Khan <khanzf at gmail.com>:
>> 
>> Hi Rolf,
>> 
>> I have been reading up on this code myself. I do not claim expertise,
>> so hopefully others can chim in here if I make a mistake. On the WiFi
>> side of things, I've pretty consistently I've seen the following
>> routine:
>> 
>> A call to pci_find_cap(9), typically as follows from the WiFi code
>> I've looked into:
>> pci_find_cap(dev, PCIY_EXPRESS, &sc->sc_cap_off);
>> 
>> A write to clear the PCI Retry timeout, by writing 0x41 to the device
>> with pci_write_config(9):
>> pci_write_config(dev, 0x41, 0, 1);
>> 
>> Obtain the base address registers using PCIR_BAR(), an abstraction of
>> bus_alloc_resource(), and store that value.
>> rid = PCIR_BAR(0)
>> 
>> Allocate IO DMA memory to the device:
>> sc->sc_mem = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &rid, RF_ACTIVE);
>> Most drivers check this return value and return ENXIO if the value is NULL.
>> 
>> I do not fully understand what rman_get_bustag(9) does, but I see this
>> called on all drivers I've looked at as follows:
>> sc->sc_st = rman_get_bustag(sc->sc_mem);
>> sc->sc_sh = rman_get_bushandle(sc->sc_mem);
>> 
>> Typically I see Interrupt handler code setup here. In the case of
>> rtwn(4) and iwm(4). This might not be relevant to your driver,
>> depending on what it does. I see a call to pci_alloc_msi as follows:
>> rid = 1;
>> if (pci_alloc_msi(dev, &rid) == 0)
>> rid = 1;
>> else
>> rid = 0;
>> pci->irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid, RF_ACTIVE |
>> (rid != 0 ? 0 : RF_SHAREABLE));
>> 
>> Then assign an interrupt handler function as follows so you know to
>> read from the DMA.
>> error = bus_setup_intr(dev, sc->sc_irq, INTR_TYPE_NET | INTR_MPSAFE,
>> NULL, your_function_here, sc, &sc->sc_ih);
>> 
>> Capture the DMA tag as follows:
>> sc->sc_dmat = bus_get_dma_tag(sc->sc_dev);
>> 
>> From here, I see the following sequence:
>> 1. bus_dma_tag_create(), where the first argument is the DMA tag
>> (sc->sc_dmat), and the last argument is the memory it is mapped to.
>> 2. bus_dmamem_alloc()
>> 3. bus_dmamap_load()
>> 4. bus_dmamap_sync()
>> 
>> At this point, you may need to initialize the interrupts, depending on
>> your hardware. If/when the interrupt arrives, you need to determine
>> what type it is and read from the DMA memory, as allocated in the DMA
>> sequence. On every assign and allocation you should also do
>> error-checking on all return values.
>> 
>> That is my understanding. Again, someone else can jump in if I made a
>> mistake. I read rtwn(4), iwm(4) and iwi(4) while writing this and they
>> are excellent references.
>> 
>> Hope this helps!
>> --
>> Farhan Khan
>> PGP Fingerprint: B28D 2726 E2BC A97E 3854 5ABE 9A9F 00BC D525 16EE
>> 
>> 
>> On Sun, Sep 23, 2018 at 10:46 PM Dr. Rolf Jansen <rj at obsigna.com> wrote:
>>> 
>>> Hello,
>>> 
>>> A couple of years ago, with the valuable help of people on this list, I managed to write a PCI driver for the National Instruments DAQ card, NI PCI-6251. The driver was kept very simple, only map the BAR’s and some DMA memory into the user space. The DAC/ADC and DIO is then all controlled from user space by writing command codes to certain offsets from the mapped BAR’s, and reading data from the mapped DMA memory and/or from other given offsets.
>>> 
>>> See: How to map device addresses into user space
>>> https://lists.freebsd.org/pipermail/freebsd-drivers/2013-January/thread.html
>>> 
>>> 
>>> Now, I need to do exactly the same for a PCIe card, namely the NI PCIe-6351. However, I even cannot see where to start. For PCI cards, the respective chapter 11 in the FreeBSD Architecture Handbook was quite helpful, for getting started, in no time, I got the PCI card probed and attached.
>>> 
>>> https://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/pci.html
>>> 
>>> The dumb approach, probing the device ID and attaching the PCI(e) driver to the PCIe device in a similar fashion didn’t work - I didn’t expect this either, however, where would I start? How do I attach my driver to a PCIe card on FreeBSD 11 or 12?
>>> 
>>> Please can somebody push me into the right direction. Once I got the driver attached to the card, I guess, I would be able to adapt the mem-mapping code of my old driver for the new card.
>>> 
>>> Best regards
>>> 
>>> Rolf
>>> _______________________________________________
>>> freebsd-drivers at freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
>>> To unsubscribe, send any mail to "freebsd-drivers-unsubscribe at freebsd.org"
> 
> _______________________________________________
> freebsd-drivers at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
> To unsubscribe, send any mail to "freebsd-drivers-unsubscribe at freebsd.org"