How to map device addresses into user space

[Dr. Rolf Jansen]

Am 07.01.2013 um 13:11 schrieb John Baldwin:

> On Monday, January 07, 2013 08:52:01 AM Dr. Rolf Jansen wrote:
> 
>> Am 03.01.2013 um 14:45 schrieb Dr. Rolf Jansen:
>> 
>>> ...
>>> 
>>> So, how can I map device addresses into user space on FreeBSD?
>> 
>> Many Thanks to everybody, who responded.
>> 
>> I got it working. I wrote a PCI device driver, which in addition to
>> open/close/read/write responds also to ioctl and mmap requests from user
>> space. In its pci_attach routine, it assembles a custom address_space
>> struct containing the addresses of the two BAR's of my NI PCI-DAQ board
>> and two huge allocated memory regions for DMA...
>> 
> 
> Note that if your BARs are not contiguous you can allow mapping of things in 
> the middle.  This is a potentital security hole if your driver can be opened 
> by non-root.

I can understand this. Actually, on the present model of the DAQ board, the
BARs are contiguous, and in the present incarnation, my driver can only be
addressed by root. Anyway, I want to do it correctly, that means, I would
like to avoid right now potential holes bubbling up in the future, when using
my driver with other DAQ boards, and in different usage scenarios.

> A more robust approach is to define a virtual address space for 
> your device. This is also a lot easier to program against in userland.
> For example, you could define offset 0 as mapping to BAR 0 and the next chunk
> of virtual address space (the offset) map to BAR 1, etc.:

I did this, but id didn't work out. The 2 BARs are contiguous, and the 2 DMA
regions are also, but there is huge gap between BAR and DMA. And depending on
which mapping is done first, either the DMAs or the BARs receive invalid paddr.
So, I guess, defining a virtual address space is a little bit more involved
than simply getting the offsets straight as you lined out. 

I am still learning things, so please bear with me, if it is a dumb question.
How do I define a virtual address space for the device?

> static int
> mydaq_mmap(...)
> {
> 	struct mydaq_softc *sc;  // you should be storing things like the pointers
>                             // to your BARs in a softc
> 
> 	sc = dev->si_drv1;
> 	if (offset <= rman_get_size(sc->bar0res))
> 		return (rman_get_start(sc->bar0res) + offset);
> 	offset -= rman_get_size(sc->bar0res);
> 	if (offset <= rman_get_size(sc->bar1res))
> 		return (rman_get_start(sc->bar1res) + offset);
> 	offset -= rman_get_size(sc->bar1res);
> 	if (offset <= dmaAIBufSize)
> 		return (vtophys(sc->dma[0]) + offset);
> 	offset -= dmaAIBufSize;
> 	if (offset <= dmaAOBufSize)
> 		return (vtophys(sc->dma[1]) + offset);
> 	
> 	/* Invalid offset */
> 	return (-1);
> }

I guess, you hacked this quickly together, and I picked the concept, so
this is fine. In order to make this working, the addresses should be assigned
to the *paddr parameter, and a status code should be returned. In order to
check things, of course I applied these minor corrections, but it doesn't
work for either the BAR block or the DMA block, depending on which is mapped first.

> Also, rman_get_bushandle() is not appropriate to get a virtual address, that is an 
> implementation detail you should not count on.  You could use rman_get_virtual(),
> but that is not the best practice.  Using rman_get_start() 
> to get a PA directly is a better approach.

Yes, this is much more reasonable than my approach. I was already concerned about
rman_get_bushandle() making a vaddr from a paddr in kernel space, only to translate
it back to a paddr in user space. rman_get_start() is of course much better, and I
will change to this, once I got the definition of a virtual address space straight.

Many thanks for your very helpful response.

Best regards

Rolf