How to use the DMA Engine in FreeBSD?

[Rajesh Kumar]

Any thoughts (or) suggestions  here?

On Mon, Dec 17, 2018 at 4:54 PM Rajesh Kumar <rajfbsd at gmail.com> wrote:

> Hi,
>
> Thanks everyone for your inputs. I assume, "PCI busmaster"ing refers to a
> peripheral device being made as the owner of the bus by the DMA engine and
> do data transfer.  Please correct me if I am wrong. But, will this bus_dma
> API uses the DMA engine in hardware (does it support PCI busmastering)?
>
> As I understand, DMA engine will expose DMA channels, which can be used by
> the client drivers to queue their DMA requests (without CPU intervention)
> and completions are given through the callback functions. If this is
> considered the real DMA operation, I don't see options to request DMA
> channels using bus_dma API's, and the callbacks in bus_dma API is meant to
> receive just the DMA map details (doesn't indicate a DMA completion).
> ioat(4) driver seems to be doing what I described above, but that seems to
> be Intel specific. Can it be used with any DMA controllers?
>
> So, I am wondering what needs to be done in FreeBSD to do the actual DMA
> involving DMA engine in a more generic way?
>
> Thanks,
> Rajesh.
>
>
> On Sat, Dec 15, 2018 at 6:30 AM Conrad Meyer <cem at freebsd.org> wrote:
>
>> On Fri, Dec 14, 2018 at 8:17 AM Alan Somers <asomers at freebsd.org> wrote:
>> > > For some Intel based server hardware, there is the "ioat" driver,
>> which
>> > > allows for user code to schedule DMA operations.  See ioat(4) for
>> > > details, including a pointer to the test program.
>>
>> This isn't true (or at least, only for testing).  It's only usable by
>> the kernel at the moment.
>>
>> > * In what context are callbacks called?  Are they called from a signal
>> > handler, or in a separate thread, or something else?
>>
>> Directly from the ithread, mostly.  Callers can't take sleep locks or
>> do very much besides set a flag and wakeup, or kick off a callout or
>> taskqueue.
>>
>> > * Why isn't ioat.h installed?
>>
>> Why would it be?  It's a kernel interface.
>>
>> > * Are "interrupts" synonymous with callbacks?
>>
>> I don't understand the question.  Interrupts are synonymous with
>> interrupts.  Like, MSI-X.
>>
>> > * Do you have a rough idea for about the minimum buffer size that
>> > makes sense to use with ioat?
>>
>> What makes sense will depend on your use case.  It's not necessarily
>> faster than CPU memcpy, but it may be worth some slowdown to offload
>> latency-insensitive bulk copies.  (It may make a lot of sense for
>> asynchronous page zero, if someone wants to bring that back.)  It's
>> worth benchmarking your specific model.
>>
>> We use it for >= 8kB copies on Broadwell, although that's largely an
>> artifact of our use case (write sizes are exactly 8 bytes, 512 bytes,
>> or multiples of 8kB).  IIRC, it shows up as a gen3 PCIe device with
>> some small number of lanes on Broadwell, although I may be mistaken;
>> and it may not communicate with RAM over the PCIe bus anyway, given
>> the tight integration with the CPU.
>>
>> Best,
>> Conrad
>> _______________________________________________
>> freebsd-drivers at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-drivers
>> To unsubscribe, send any mail to "freebsd-drivers-unsubscribe at freebsd.org
>> "
>>
>