From nobody Tue Nov 03 09:02:29 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4jJjJQ292bz6yQ1g for ; Tue, 03 Nov 2026 09:02:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4jJjJQ05ftz4Kk4 for ; Tue, 03 Nov 2026 09:02:30 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793696550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=p5lfpa3mhJQbiOz60BP127/e8CCzl/JS4Tf7MTe0OZM=; b=D6hyj59mUzUscda0mb1xr/P3zhXNJA7BHePb0zUXaUYv1W3y1SRCWd3XZyrzw9WhWiQ7wi LLKzZknm3yDUSbrl87QdxJAZUW3fEqtY25Duvx2msBpsatEOKQrmme7oEoH2udwV46GrNV pb0l/NHvGxv3lGxxCnxfrQi7nEXuZM34rJ1HrPYLxAkXpxskP0q6pQSGytzZpunvZUg5QR gxnTaB/cOJMLW8HRI2UNkEUHlP0GeGjnTc6NkAK8wy20i1XpP4j1g0rKO3IZjqoli2dSV1 qALIa/qvIRWRnl81JIb2uMJ4UiV6SklAZ7N6HOjGvVoPcwSFF56GOWxdH/2oSA== ARC-Seal: i=1; a=rsa-sha256; d=freebsd.org; s=dkim; cv=none; t=1793696550; b=RBkayKpQPb9oa6zYHdpKtdtWom16nRHgcO663dPYr/oSVhTw/nkJmn7+3MOQXn2t9CGrCR 7AsZBirIdmCYfH9I74A9diATFUmbNQkQ+F85DXhQV1Q5pydyG+2XEOSRMDZBpVytCbrieI J6ZRc4TQFnznYBzrasJ6bCjA8iZyqb5uxfU1Ls8c/jI5O7ujdKbVlGplTzzOSIbx835Zhx 8itlhpCZWjkQbPsSiH1EODqNPxZB5QGH5HlWU2f7x1hFkU5CyDl+WTTAac4tlyRMeioIBV Z7gduHMN/AnL27khpODUNupOp+DPV4REprifRy90K3TSxFPOhMH6udPWhIc/gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1793696550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=p5lfpa3mhJQbiOz60BP127/e8CCzl/JS4Tf7MTe0OZM=; b=kumP6xarl8Lt6FwApx8hv6stDD5nsxZJ2+KVBgDXeFQ36iprUpxs6SztF4pR+PSBF5ovP4 m9ftmnb+xt3YgH9RWzH4Bgnd6gEc3XeLRLbaldK65CBbRZGlh4tENxGY7PNULvBkMLK/lI lqD0CM0RYZam4LJN6ipU12TbbGCFv4umOqwpU+5lIc8qYpgPr3NnD5cW3anJ2TZ7Obtezf bTX8FuFnWVNkSOXAU9tBsVftlFu1wEMdlPwa/Tu36mZ+TFWIAGjpONgLYhDHt+qAyQFc8C 2gnULIuvCaJIs0VS0PZOHvmSrAZIKLC8MR38Mefcy0BP++3rip6eiJpKaEJL4g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4jJjJP5hHvz1PTd for ; Tue, 03 Nov 2026 09:02:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 459dd by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 03 Nov 2026 09:02:29 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Cc: Sarah Walker From: Konstantin Belousov Subject: git: 6695c6bd0bf2 - stable/15 - virtio_net: Use bus_dma for rxq/txq buffers List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org List-Id: Commits to the stable branches of the FreeBSD src repository List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 6695c6bd0bf248927f34479a9ea2e13a6bf7db3d Auto-Submitted: auto-generated Date: Tue, 03 Nov 2026 09:02:29 +0000 Message-Id: <6ae9a325.459dd.1cdb19aa@gitrepo.freebsd.org> The branch stable/15 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=6695c6bd0bf248927f34479a9ea2e13a6bf7db3d commit 6695c6bd0bf248927f34479a9ea2e13a6bf7db3d Author: Sarah Walker AuthorDate: 2026-02-16 14:19:13 +0000 Commit: Konstantin Belousov CommitDate: 2026-11-03 09:01:20 +0000 virtio_net: Use bus_dma for rxq/txq buffers (cherry picked from commit 8193fa3b8e1a9425bd8ac8b707cab565816b8584) --- sys/dev/virtio/network/if_vtnet.c | 298 +++++++++++++++++++++++++++++++++-- sys/dev/virtio/network/if_vtnetvar.h | 10 ++ 2 files changed, 291 insertions(+), 17 deletions(-) diff --git a/sys/dev/virtio/network/if_vtnet.c b/sys/dev/virtio/network/if_vtnet.c index cc49c743ee97..faae9273c182 100644 --- a/sys/dev/virtio/network/if_vtnet.c +++ b/sys/dev/virtio/network/if_vtnet.c @@ -96,6 +96,17 @@ #define VTNET_ETHER_ALIGN ETHER_ALIGN #endif +/* + * Worst case offset to ensure header doesn't share any cache lines with + * payload. + */ +#define VTNET_RX_BUFFER_HEADER_OFFSET 128 + +struct vtnet_rx_buffer_header { + bus_addr_t addr; + bus_dmamap_t dmap; +}; + static int vtnet_modevent(module_t, int, void *); static int vtnet_probe(device_t); @@ -392,6 +403,17 @@ MODULE_DEPEND(vtnet, netmap, 1, 1, 1); VIRTIO_SIMPLE_PNPINFO(vtnet, VIRTIO_ID_NETWORK, "VirtIO Networking Adapter"); +static struct vtnet_rx_buffer_header * +vtnet_mbuf_to_rx_buffer_header(struct vtnet_softc *sc, struct mbuf *m) +{ + if (VTNET_ETHER_ALIGN != 0 && sc->vtnet_hdr_size % 4 == 0) + return (struct vtnet_rx_buffer_header *)((uintptr_t)m->m_data - + VTNET_RX_BUFFER_HEADER_OFFSET - VTNET_ETHER_ALIGN); + else + return (struct vtnet_rx_buffer_header *)((uintptr_t)m->m_data - + VTNET_RX_BUFFER_HEADER_OFFSET); +} + static int vtnet_modevent(module_t mod __unused, int type, void *unused __unused) { @@ -466,6 +488,60 @@ vtnet_attach(device_t dev) goto fail; } + mtx_init(&sc->vtnet_rx_mtx, device_get_nameunit(dev), + "VirtIO Net RX lock", MTX_DEF); + + error = bus_dma_tag_create( + bus_get_dma_tag(dev), /* parent */ + 1, /* alignment */ + 0, /* boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + MJUM9BYTES, /* max request size */ + 1, /* max # segments */ + MJUM9BYTES, /* maxsegsize - worst case */ + BUS_DMA_COHERENT, /* flags */ + busdma_lock_mutex, /* lockfunc */ + &sc->vtnet_rx_mtx, /* lockarg */ + &sc->vtnet_rx_dmat); + if (error) { + device_printf(dev, "cannot create bus_dma_tag\n"); + goto fail; + } + + mtx_init(&sc->vtnet_tx_mtx, device_get_nameunit(dev), + "VirtIO Net TX lock", MTX_DEF); + + error = bus_dma_tag_create( + bus_get_dma_tag(dev), /* parent */ + 1, /* alignment */ + 0, /* boundary */ + BUS_SPACE_MAXADDR, /* lowaddr */ + BUS_SPACE_MAXADDR, /* highaddr */ + NULL, NULL, /* filter, filterarg */ + sc->vtnet_tx_nsegs * MJUM9BYTES, /* max request size */ + sc->vtnet_tx_nsegs, /* max # segments */ + MJUM9BYTES, /* maxsegsize */ + BUS_DMA_COHERENT, /* flags */ + busdma_lock_mutex, /* lockfunc */ + &sc->vtnet_tx_mtx, /* lockarg */ + &sc->vtnet_tx_dmat); + if (error) { + device_printf(dev, "cannot create bus_dma_tag\n"); + goto fail; + } + +#ifdef __powerpc__ + /* + * Virtio uses physical addresses rather than bus addresses, so we + * need to ask busdma to skip the iommu physical->bus mapping. At + * present, this is only a thing on the powerpc architectures. + */ + bus_dma_tag_set_iommu(sc->vtnet_rx_dmat, NULL, NULL); + bus_dma_tag_set_iommu(sc->vtnet_tx_dmat, NULL, NULL); +#endif + error = vtnet_alloc_rx_filters(sc); if (error) { device_printf(dev, "cannot allocate Rx filters\n"); @@ -1574,6 +1650,11 @@ static struct mbuf * vtnet_rx_alloc_buf(struct vtnet_softc *sc, int nbufs, struct mbuf **m_tailp) { struct mbuf *m_head, *m_tail, *m; + struct vtnet_rx_buffer_header *vthdr; + bus_dma_segment_t segs[1]; + bus_dmamap_t dmap; + int nsegs; + int err; int i, size; m_head = NULL; @@ -1591,13 +1672,43 @@ vtnet_rx_alloc_buf(struct vtnet_softc *sc, int nbufs, struct mbuf **m_tailp) } m->m_len = size; + vthdr = (struct vtnet_rx_buffer_header *)m->m_data; + + /* Reserve space for header */ + m_adj(m, VTNET_RX_BUFFER_HEADER_OFFSET); + /* * Need to offset the mbuf if the header we're going to add * will misalign. */ - if (VTNET_ETHER_ALIGN != 0 && sc->vtnet_hdr_size % 4 == 0) { + if (VTNET_ETHER_ALIGN != 0 && sc->vtnet_hdr_size % 4 == 0) m_adj(m, VTNET_ETHER_ALIGN); + + err = bus_dmamap_create(sc->vtnet_rx_dmat, 0, &dmap); + if (err) { + printf("Failed to create dmamap, err :%d\n", + err); + m_freem(m); + return (NULL); } + + nsegs = 0; + err = bus_dmamap_load_mbuf_sg(sc->vtnet_rx_dmat, dmap, m, segs, + &nsegs, BUS_DMA_NOWAIT); + if (err != 0) { + printf("Failed to map mbuf into DMA visible memory, err: %d\n", + err); + m_freem(m); + bus_dmamap_destroy(sc->vtnet_rx_dmat, dmap); + return (NULL); + } + KASSERT(nsegs == 1, + ("%s: unexpected number of DMA segments for rx buffer: %d", + __func__, nsegs)); + + vthdr->addr = segs[0].ds_addr; + vthdr->dmap = dmap; + if (m_head != NULL) { m_tail->m_next = m; m_tail = m; @@ -1623,7 +1734,7 @@ vtnet_rxq_replace_lro_nomrg_buf(struct vtnet_rxq *rxq, struct mbuf *m0, int len, clustersz, nreplace, error; sc = rxq->vtnrx_sc; - clustersz = sc->vtnet_rx_clustersz; + clustersz = sc->vtnet_rx_clustersz - VTNET_RX_BUFFER_HEADER_OFFSET; /* * Need to offset the mbuf if the header we're going to add will * misalign, account for that here. @@ -1738,9 +1849,12 @@ vtnet_rxq_replace_buf(struct vtnet_rxq *rxq, struct mbuf *m, int len) static int vtnet_rxq_enqueue_buf(struct vtnet_rxq *rxq, struct mbuf *m) { + struct vtnet_rx_buffer_header *hdr; struct vtnet_softc *sc; struct sglist *sg; int header_inlined, error; + bus_addr_t paddr; + struct mbuf *mp; sc = rxq->vtnrx_sc; sg = rxq->vtnrx_sg; @@ -1753,28 +1867,38 @@ vtnet_rxq_enqueue_buf(struct vtnet_rxq *rxq, struct mbuf *m) header_inlined = vtnet_modern(sc) || (sc->vtnet_flags & VTNET_FLAG_MRG_RXBUFS) != 0; /* TODO: ANY_LAYOUT */ + hdr = vtnet_mbuf_to_rx_buffer_header(sc, m); + paddr = hdr->addr; + /* * Note: The mbuf has been already adjusted when we allocate it if we * have to do strict alignment. */ - if (header_inlined) - error = sglist_append_mbuf(sg, m); - else { - struct vtnet_rx_header *rxhdr = - mtod(m, struct vtnet_rx_header *); + if (header_inlined) { + error = sglist_append_phys(sg, paddr, m->m_len); + } else { MPASS(sc->vtnet_hdr_size == sizeof(struct virtio_net_hdr)); /* Append the header and remaining mbuf data. */ - error = sglist_append(sg, &rxhdr->vrh_hdr, sc->vtnet_hdr_size); + error = sglist_append_phys(sg, paddr, sc->vtnet_hdr_size); if (error) return (error); - error = sglist_append(sg, &rxhdr[1], + error = sglist_append_phys(sg, + paddr + sizeof(struct vtnet_rx_header), m->m_len - sizeof(struct vtnet_rx_header)); if (error) return (error); - if (m->m_next != NULL) - error = sglist_append_mbuf(sg, m->m_next); + mp = m->m_next; + while (mp) { + hdr = vtnet_mbuf_to_rx_buffer_header(sc, mp); + paddr = hdr->addr; + error = sglist_append_phys(sg, paddr, mp->m_len); + if (error) + return (error); + + mp = mp->m_next; + } } if (error) @@ -2002,6 +2126,7 @@ vtnet_rxq_merged_eof(struct vtnet_rxq *rxq, struct mbuf *m_head, int nbufs) m_tail = m_head; while (--nbufs > 0) { + struct vtnet_rx_buffer_header *vthdr; struct mbuf *m; uint32_t len; @@ -2011,6 +2136,10 @@ vtnet_rxq_merged_eof(struct vtnet_rxq *rxq, struct mbuf *m_head, int nbufs) goto fail; } + vthdr = vtnet_mbuf_to_rx_buffer_header(sc, m); + bus_dmamap_sync(sc->vtnet_rx_dmat, vthdr->dmap, + BUS_DMASYNC_POSTREAD); + if (vtnet_rxq_new_buf(rxq) != 0) { rxq->vtnrx_stats.vrxs_iqdrops++; vtnet_rxq_discard_buf(rxq, m); @@ -2019,6 +2148,9 @@ vtnet_rxq_merged_eof(struct vtnet_rxq *rxq, struct mbuf *m_head, int nbufs) goto fail; } + bus_dmamap_unload(sc->vtnet_rx_dmat, vthdr->dmap); + bus_dmamap_destroy(sc->vtnet_rx_dmat, vthdr->dmap); + if (m->m_len < len) len = m->m_len; @@ -2131,6 +2263,7 @@ static int vtnet_rxq_eof(struct vtnet_rxq *rxq) { struct virtio_net_hdr lhdr, *hdr; + struct vtnet_rx_buffer_header *vthdr; struct vtnet_softc *sc; if_t ifp; struct virtqueue *vq; @@ -2146,14 +2279,31 @@ vtnet_rxq_eof(struct vtnet_rxq *rxq) CURVNET_SET(if_getvnet(ifp)); while (count-- > 0) { - struct mbuf *m; + struct mbuf *m, *mp; uint32_t len, nbufs, adjsz; + uint32_t synced; m = virtqueue_dequeue(vq, &len); if (m == NULL) break; deq++; + mp = m; + + /* + * Sync all mbufs in this packet. There will only be a single + * mbuf unless LRO is in use. + */ + synced = 0; + while (mp && synced < len) { + vthdr = vtnet_mbuf_to_rx_buffer_header(sc, mp); + bus_dmamap_sync(sc->vtnet_rx_dmat, vthdr->dmap, + BUS_DMASYNC_POSTREAD); + + synced += mp->m_len; + mp = mp->m_next; + } + if (len < sc->vtnet_hdr_size + ETHER_HDR_LEN) { rxq->vtnrx_stats.vrxs_ierrors++; vtnet_rxq_discard_buf(rxq, m); @@ -2187,6 +2337,18 @@ vtnet_rxq_eof(struct vtnet_rxq *rxq) continue; } + mp = m; + synced = 0; + while (mp && synced < len) { + vthdr = vtnet_mbuf_to_rx_buffer_header(sc, mp); + + bus_dmamap_unload(sc->vtnet_rx_dmat, vthdr->dmap); + bus_dmamap_destroy(sc->vtnet_rx_dmat, vthdr->dmap); + + synced += mp->m_len; + mp = mp->m_next; + } + m->m_pkthdr.len = len; m->m_pkthdr.rcvif = ifp; m->m_pkthdr.csum_flags = 0; @@ -2413,6 +2575,14 @@ vtnet_txq_free_mbufs(struct vtnet_txq *txq) while ((txhdr = virtqueue_drain(vq, &last)) != NULL) { if (kring == NULL) { + bus_dmamap_unload(txq->vtntx_sc->vtnet_tx_dmat, + txhdr->dmap); + bus_dmamap_destroy(txq->vtntx_sc->vtnet_tx_dmat, + txhdr->dmap); + bus_dmamap_unload(txq->vtntx_sc->vtnet_tx_dmat, + txhdr->hdr_dmap); + bus_dmamap_destroy(txq->vtntx_sc->vtnet_tx_dmat, + txhdr->hdr_dmap); m_freem(txhdr->vth_mbuf); uma_zfree(vtnet_tx_header_zone, txhdr); } @@ -2582,15 +2752,36 @@ drop: return (NULL); } +static void +vtnet_txq_enqueue_callback(void *arg, bus_dma_segment_t *segs, + int nsegs, int error) +{ + vm_paddr_t *hdr_paddr; + + if (error != 0) + return; + + KASSERT(nsegs == 1, ("%s: %d segments returned!", __func__, nsegs)); + + hdr_paddr = (vm_paddr_t *)arg; + *hdr_paddr = segs[0].ds_addr; +} + static int vtnet_txq_enqueue_buf(struct vtnet_txq *txq, struct mbuf **m_head, struct vtnet_tx_header *txhdr) { + bus_dma_segment_t segs[VTNET_TX_SEGS_MAX]; + int nsegs; struct vtnet_softc *sc; struct virtqueue *vq; struct sglist *sg; struct mbuf *m; int error; + vm_paddr_t hdr_paddr; + bus_dmamap_t hdr_dmap; + bus_dmamap_t dmap; + int i; sc = txq->vtntx_sc; vq = txq->vtntx_vq; @@ -2598,15 +2789,55 @@ vtnet_txq_enqueue_buf(struct vtnet_txq *txq, struct mbuf **m_head, m = *m_head; sglist_reset(sg); - error = sglist_append(sg, &txhdr->vth_uhdr, sc->vtnet_hdr_size); + + error = bus_dmamap_create(sc->vtnet_tx_dmat, 0, &hdr_dmap); + if (error) + goto fail; + + error = bus_dmamap_load(sc->vtnet_tx_dmat, hdr_dmap, &txhdr->vth_uhdr, + sc->vtnet_hdr_size, vtnet_txq_enqueue_callback, &hdr_paddr, + BUS_DMA_NOWAIT); + if (error) + goto fail_hdr_dmamap_destroy; + + error = sglist_append_phys(sg, hdr_paddr, sc->vtnet_hdr_size); if (error != 0 || sg->sg_nseg != 1) { KASSERT(0, ("%s: cannot add header to sglist error %d nseg %d", __func__, error, sg->sg_nseg)); - goto fail; + goto fail_hdr_dmamap_unload; } - error = sglist_append_mbuf(sg, m); + bus_dmamap_sync(sc->vtnet_tx_dmat, hdr_dmap, BUS_DMASYNC_PREWRITE); + + error = bus_dmamap_create(sc->vtnet_tx_dmat, 0, &dmap); + if (error) + goto fail_hdr_dmamap_unload; + + nsegs = 0; + error = bus_dmamap_load_mbuf_sg(sc->vtnet_tx_dmat, dmap, m, segs, + &nsegs, BUS_DMA_NOWAIT); + if (error != 0) + goto fail_dmamap_destroy; + KASSERT(nsegs <= sc->vtnet_tx_nsegs, + ("%s: unexpected number of DMA segments for tx buffer: %d (max %d)", + __func__, nsegs, sc->vtnet_tx_nsegs)); + + bus_dmamap_sync(sc->vtnet_tx_dmat, dmap, BUS_DMASYNC_PREWRITE); + + for (i = 0; i < nsegs && !error; i++) + error = sglist_append_phys(sg, segs[i].ds_addr, segs[i].ds_len); + if (error) { + sglist_reset(sg); + bus_dmamap_unload(sc->vtnet_tx_dmat, dmap); + + error = sglist_append_phys(sg, hdr_paddr, sc->vtnet_hdr_size); + if (error != 0 || sg->sg_nseg != 1) { + KASSERT(0, ("%s: cannot add header to sglist error %d nseg %d", + __func__, error, sg->sg_nseg)); + goto fail_dmamap_destroy; + } + m = m_defrag(m, M_NOWAIT); if (m == NULL) { sc->vtnet_stats.tx_defrag_failed++; @@ -2616,16 +2847,41 @@ vtnet_txq_enqueue_buf(struct vtnet_txq *txq, struct mbuf **m_head, *m_head = m; sc->vtnet_stats.tx_defragged++; - error = sglist_append_mbuf(sg, m); + nsegs = 0; + error = bus_dmamap_load_mbuf_sg(sc->vtnet_tx_dmat, dmap, m, + segs, &nsegs, BUS_DMA_NOWAIT); + if (error != 0) + goto fail_dmamap_destroy; + KASSERT(nsegs <= sc->vtnet_tx_nsegs, + ("%s: unexpected number of DMA segments for tx buffer: %d (max %d)", + __func__, nsegs, sc->vtnet_tx_nsegs)); + + bus_dmamap_sync(sc->vtnet_tx_dmat, dmap, BUS_DMASYNC_PREWRITE); + + for (i = 0; i < nsegs && !error; i++) + error = sglist_append_phys(sg, segs[i].ds_addr, + segs[i].ds_len); + if (error) - goto fail; + goto fail_dmamap_unload; } txhdr->vth_mbuf = m; + txhdr->dmap = dmap; + txhdr->hdr_dmap = hdr_dmap; + error = virtqueue_enqueue(vq, txhdr, sg, sg->sg_nseg, 0); return (error); +fail_dmamap_unload: + bus_dmamap_unload(sc->vtnet_tx_dmat, dmap); +fail_dmamap_destroy: + bus_dmamap_destroy(sc->vtnet_tx_dmat, dmap); +fail_hdr_dmamap_unload: + bus_dmamap_unload(sc->vtnet_tx_dmat, hdr_dmap); +fail_hdr_dmamap_destroy: + bus_dmamap_destroy(sc->vtnet_tx_dmat, hdr_dmap); fail: m_freem(*m_head); *m_head = NULL; @@ -2905,6 +3161,7 @@ vtnet_txq_tq_intr(void *xtxq, int pending __unused) static int vtnet_txq_eof(struct vtnet_txq *txq) { + struct vtnet_softc *sc; struct virtqueue *vq; struct vtnet_tx_header *txhdr; struct mbuf *m; @@ -2914,6 +3171,8 @@ vtnet_txq_eof(struct vtnet_txq *txq) deq = 0; VTNET_TXQ_LOCK_ASSERT(txq); + sc = txq->vtntx_sc; + while ((txhdr = virtqueue_dequeue(vq, NULL)) != NULL) { m = txhdr->vth_mbuf; deq++; @@ -2923,6 +3182,11 @@ vtnet_txq_eof(struct vtnet_txq *txq) if (m->m_flags & M_MCAST) txq->vtntx_stats.vtxs_omcasts++; + bus_dmamap_unload(sc->vtnet_tx_dmat, txhdr->dmap); + bus_dmamap_destroy(sc->vtnet_tx_dmat, txhdr->dmap); + bus_dmamap_unload(sc->vtnet_tx_dmat, txhdr->hdr_dmap); + bus_dmamap_destroy(sc->vtnet_tx_dmat, txhdr->hdr_dmap); + m_freem(m); uma_zfree(vtnet_tx_header_zone, txhdr); } diff --git a/sys/dev/virtio/network/if_vtnetvar.h b/sys/dev/virtio/network/if_vtnetvar.h index 6ee6049e064b..73782d2f1dff 100644 --- a/sys/dev/virtio/network/if_vtnetvar.h +++ b/sys/dev/virtio/network/if_vtnetvar.h @@ -191,6 +191,12 @@ struct vtnet_softc { struct mtx vtnet_mtx; char vtnet_mtx_name[16]; uint8_t vtnet_hwaddr[ETHER_ADDR_LEN]; + + bus_dma_tag_t vtnet_rx_dmat; + struct mtx vtnet_rx_mtx; + + bus_dma_tag_t vtnet_tx_dmat; + struct mtx vtnet_tx_mtx; }; /* vtnet flag descriptions for use with printf(9) %b identifier. */ #define VTNET_FLAGS_BITS \ @@ -274,6 +280,10 @@ struct vtnet_tx_header { } vth_uhdr; struct mbuf *vth_mbuf; + + bus_dmamap_t dmap; + + bus_dmamap_t hdr_dmap; }; /*