bus_dmamem_alloc doesn't follow alignment specified

[Konstantin Belousov]

On Tue, Nov 27, 2018 at 04:37:57PM +0530, Rajesh Kumar wrote:
> Hi,
> 
> I am writing a PCI-E driver, where I am trying to allocate DMA buffers. But
> seems, like the alignment requirements are not satisfied by
> bus_dmamem_alloc function. I am using stable/11 branch.
> 
> *Code :*
> dma_setup(size, align, alimit)  {
> 
>     printf ("size 0x%x align 0x%x alimit 0x%x\n", size, align, alimit);
> 
>      if (bus_dma_tag_create(parent, align, 0,
>                alimit, BUS_SPACE_MAXADDR,
>                 NULL, NULL, size, 1, size,
>                 0, NULL, NULL, &dma_tag)) {
>                 return (ENOMEM);
>     }
> 
>     if (bus_dmamem_alloc(dma_tag, &virt_addr,
>                 BUS_DMA_WAITOK | BUS_DMA_ZERO, &dma_map)) {
>                     bus_dma_tag_destroy(dma_tag);
>                     return (ENOMEM);
>     }
> 
>     if (bus_dmamap_load(dma_tag, dma_map, virt_addr,
>                 size, dma_cbfn, &arg, BUS_DMA_NOWAIT)) {
>                    bus_dmamem_free(dma_tag, virt_addr, dma_map);
>                    bus_dma_tag_destroy(mw->dma_tag);
>                    return (ENOMEM);
>    }
> 
>    printf("dma_addr 0x%lx virt_addr 0x%lx\n", (uint64_t)arg->addr,
> (uint64_t)virt_addr);
> 
> }
> 
> dma_cbfn(void *tmp, bus_dma_segment_t *segs, int nsegs, int error)
> {
>         struct dma_args *arg = (struct dma_args *)tmp;
>         arg->addr = segs[0].ds_addr;
> }
> 
> *Logs: *
> size 0x100000 align 0x100000 alimit 0xffffffff
> dma_addr *0x78e00000*  virt_addr *0xfffffe1c3665f000*
> 
> As seen above, dma_addr is as per alignment requirement, but virtual
> address from bus_dmamem_alloc doesn't seem to satisfy the alignment
> requirement).
> 
> My understanding is both the DMA addr and the virtual address should be as
> per the alignment specified.  Can anyone please clarify?

Only bus address is aligned.

For single-segment i.e. contiguous allocations, amd64 could benefit from
using dmap VA instead of kmem_alloc.  This would be only an optimization,
not done right now.