How to map device addresses into user space

[Dr. Rolf Jansen]

Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen:

> I am building a loadable kernel module for FreeBSD 9.1-RELEASE x86_64 for a PCI Data Acquisition board from National Instruments.
> 
> I need to map the Base Address Registers into user space memory, in order to pass the BAR's to the National Instruments Drivers Development Kit (NI-DDK). The DDK is a complex set of C++ classes running in user space, that read/write directly from/into the BAR's.
> 
> The FreeBSD bus_space_* functions are useless in this respect, because the DDK isn't designed that way, I need the BAR addresses mapped into user space.
> 
> Having the measurement done by the kernel module is not an option either, because it is math intensive, and kernel modules are build without SSE and with soft float.
> 
> I got tiny kernel modules/extensions only providing the mapped addresses of the PCI BAR's running together with the Measurement Routines using the NI-DDK on Darwin (Mac OS X) and Linux.
> 
> So, how can I map device addresses into user space on FreeBSD?



Many Thanks to everybody, who responded.

I got it working. I wrote a PCI device driver, which in addition to open/close/read/write responds also to ioctl and mmap requests from user space. In its pci_attach routine, it assembles a custom address_space struct containing the addresses of the two BAR's of my NI PCI-DAQ board and two huge allocated memory regions for DMA.

   static struct address_space space;
   ...
   ...
   int              bar0_id,  bar1_id; 
   struct resource *bar0res, *bar1res;

   bar0_id = PCIR_BAR(0);
   bar1_id = PCIR_BAR(1);

   // these resources must be teared down in pci_detach
   bar0res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &bar0id, RF_ACTIVE);
   bar1res = bus_alloc_resource_any(dev, SYS_RES_MEMORY, &bar1id, RF_ACTIVE);

   // assemble the address_space struct, i.e.
   // virtual addr., size, and offset of its members
   space.bar[0].virtual = rman_get_bushandle(bar0res);
   space.bar[1].virtual = rman_get_bushandle(bar1res);
   space.dma[0].virtual = malloc(dmaAIBufSize, M_MYDAQ, M_WAITOK);
   space.dma[1].virtual = malloc(dmaAOBufSize, M_MYDAQ, M_WAITOK);

   space.bar[0].size = rman_get_end(bar0res) - rman_get_start(bar0res) + 1;
   space.bar[1].size = rman_get_end(bar1res) - rman_get_start(bar1res) + 1;
   space.dma[0].size = dmaAIBufSize;
   space.dma[1].size = dmaAOBufSize;

   // this is the crucial part -- The Offsets!
   // care must be taken, that the offsets are not negative,
   // therefore, find out the minimum and take this as the base address
   space.base = MIN(MIN(space.bar[0].virt, space.bar[1].virt),
                    MIN(space.dma[0].virt, space.dma[1].virt));
   space.bar[0].offs = space.bar[0].virt - space.base;
   space.bar[1].offs = space.bar[1].virt - space.base;
   space.dma[0].offs = space.dma[0].virt - space.base;
   space.dma[1].offs = space.dma[1].virt - space.base;


By a call to the ioctl handler of my PCI device driver, this readily assembled address_space struct is transferred from the driver to my user space measurement controller, and here it must actually be completed by subsequent calls to mmap of my device driver:

   space.bar[0].physical = mmap(0, space.bar[0].size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, space.bar[0].offset);
   space.bar[1].physical = mmap(0, space.bar[1].size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, space.bar[1].offset);
   space.dma[0].physical = mmap(0, space.dma[0].size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, space.dma[0].offset);
   space.dma[1].physical = mmap(0, space.dma[1].size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, space.dma[1].offset);


Because of the coherent offsets, the mmap handler of my PCI device driver can be kept quite simple:

static int mydaq_mmap(struct cdev *dev, vm_ooffset_t offset, vm_paddr_t *paddr, int nprot, vm_memattr_t *memattr)
{
   *paddr = vtophys(space.base + offset);
   return 0;
}


This works for me.

Best regards

Rolf