correct use of bus_dmamap_sync

[John Baldwin]

On Wednesday 26 October 2005 02:13 am, Dinesh Nair wrote:
> On 10/26/05 04:10 John Baldwin said the following:
> > Yes, and on some archs the sync() operations do have memory barriers in
> > place, but there isn't any bounce buffering with bus_dmamem_alloc()
> > memory.
>
> and in _bus_dmamap_load() in /usr/src/sys/i386/i386/busdma_machdep.c,
> apparently if the second argument to bus_dmamap_load (the pointer to
> bus_dmamap_t)) is NULL, the syscall code sets it to &nobounce_dmamap, a
> static struct which doesnt seem to be used/allocated, except within the
> syscall.
>
> what would the implications of using NULL for the dmamap address be ?
>
> > Well, you need it to get the physical address to pass to your device for
> > it to do DMA against.
>
> on freebsd 4.x, vtophys(buffer) returns the same value as the this address.
>   (i.e, when the callback function from bus_dmamap_load() is called, the
> address of the segment returned is the same as vtophys(buffer)). this is
> the current observed behaviour on 4.x.

On i386, yes.  It won't on sparc64 when using an IOMMU for example.  The whole 
point of using bus_dma is to not use vtophys() since by doing that you are 
assuming that the PA's used by the CPU map 1:1 to the addresses used by your 
device to do DMA, and on architectures with an IOMMU such as sparc64, G5 ppc 
boxes, and probably amd64 boxes in the future, that is not a valid assumption 
at all.

> >>have things changed between freebsd 4.x (which i'm using) and freebsd 5.x
> >> ?
> >
> > I don't think so as far as the interface.
>
> the values of the BUS_DMASYNC_XXXX constants have changed though. they're
> an enum with values 0-3 in 4.x but in 5.x they're defined as 0x01, 0x02,
> 0x04 and 0x08. due to this, combining BUS_DMASYNC_XXX thru an OR could
> possibly give different behaviour on 4.x and 5.x.
>
> an example would be using (BUS_DMASYNC_POSTREAD|BUS_DMASYNC_PREWRITE) which
> would be 0x03 in freebsd 4.x and 0x06 in freebsd 5.x. the gotcha is that
> 0x03 in freebsd 4.x is BUS_DMASYNC_POSTWRITE. so therefore,
> BUS_DMASYNC_POSTREAD|BUS_DMASYNC_PREWRITE will be BUS_DMASYNC_POSTWRITE in
> 4.x which in the syscall is actually a no op.

Yes, that is fugly.  Just don't use the | versions for now I would guess.

> also, in both 4.x and 5.x, only POSTREAD and PREWRITE have any real
> meaning, as PREREAD and POSTWRITE are no ops.

On i386, yes.  Eventually those operations might be used to manipulate IOMMU 
mappings for example.

> it's due to these that the importance of correctly using the correct
> PRE/POST READ/WRITE and in the correct places seem important and the source
> of my confusion. :)
>
> >>>thus when you send data to your device, that is a WRITE operation (even
> >>>though your device is doing a DMA to read data), and when you get data
> >>>back from your device, that is a READ operation (even though your device
> >>>is doing a DMA to write the data into the buffer).
>
> taking ruslan's suggestion, i looked up the HEAD manpage at
> http://www.freebsd.org/cgi/man.cgi?query=bus_dmamap_sync&apropos=0&sektion=
>0&manpath=FreeBSD+6.0-current&format=html
>
> i've quoted the relevant descriptions below:
>
> BUS_DMASYNC_PREWRITE
> Perform any synchronization required after an update of memory by the CPU
> but prior to DMA write operations.
>
> BUS_DMASYNC_POSTREAD
> Perform any synchronization required after DMA read operations, but prior
> to CPU access of the memory.
>
> which would indicate that we'd need to use POSTREAD /before/ reading the
> buffer and PREWRITE /after/ the CPU writes to the buffer, for the following
> pseudo code:
>
> 	/*cpu reads from device */
> 	bus_dmamap_sync(..., BUS_DMASYNC_POSTREAD)
> 	memcpy(myreceivebuf, mappedreceivebuf)
>
> 	/* do some computation on data read from device */
>
> 	/* cpu writes to device */
> 	memcpy(mappedtransmitbuf, mytransmitbuf)
> 	bus_dmamap_sync(..., BUS_DMASYNC_PREWRITE)
>
> where mappedreceivebuf and mappedtransmitbuf is the bufferspace allocated
> in bus_dmamem_alloc() and myreceivebuf/mytransmitbuf is a temporary holding
> area before writing to the device.
>
> is this reasoning correct ?

Yes.

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org