From nobody Thu Oct 21 00:09:08 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B9B681805BAD; Thu, 21 Oct 2021 00:09:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HZSWm4vd6z3FNZ; Thu, 21 Oct 2021 00:09:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 849A8110EA; Thu, 21 Oct 2021 00:09:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 19L098wV015465; Thu, 21 Oct 2021 00:09:08 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 19L098c3015464; Thu, 21 Oct 2021 00:09:08 GMT (envelope-from git) Date: Thu, 21 Oct 2021 00:09:08 GMT Message-Id: <202110210009.19L098c3015464@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: ba2a30ed753c - stable/12 - cxgbe(4): Catch up with in-flight netmap rx before destroying queues. List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: ba2a30ed753cafa9b1a12a1cc5ec290a780422b6 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=ba2a30ed753cafa9b1a12a1cc5ec290a780422b6 commit ba2a30ed753cafa9b1a12a1cc5ec290a780422b6 Author: Navdeep Parhar AuthorDate: 2020-11-21 03:27:32 +0000 Commit: Navdeep Parhar CommitDate: 2021-10-20 18:24:03 +0000 cxgbe(4): Catch up with in-flight netmap rx before destroying queues. The netmap application using the driver is responsible for replenishing the receive freelists and they may be totally depleted when the application exits. Packets in flight, if any, might block the pipeline in case there aren't enough buffers left in the freelist. Avoid this by filling up the freelists with a driver allocated buffer. Sponsored by: Chelsio Communications (cherry picked from commit b3718e2d7ebfe7fc0bb57bd0906aacc28dd755b2) --- sys/dev/cxgbe/adapter.h | 3 +++ sys/dev/cxgbe/t4_netmap.c | 48 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) diff --git a/sys/dev/cxgbe/adapter.h b/sys/dev/cxgbe/adapter.h index 18dd334a391c..a1006df3e987 100644 --- a/sys/dev/cxgbe/adapter.h +++ b/sys/dev/cxgbe/adapter.h @@ -737,6 +737,9 @@ struct sge_nm_rxq { bus_dma_tag_t fl_desc_tag; bus_dmamap_t fl_desc_map; bus_addr_t fl_ba; + + void *bb; /* bit bucket for packets with nowhere to go. */ + uma_zone_t bb_zone; }; #define INVALID_NM_TXQ_CNTXT_ID ((u_int)(-1)) diff --git a/sys/dev/cxgbe/t4_netmap.c b/sys/dev/cxgbe/t4_netmap.c index 2bee6040b10e..3dd3f42c5636 100644 --- a/sys/dev/cxgbe/t4_netmap.c +++ b/sys/dev/cxgbe/t4_netmap.c @@ -42,6 +42,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include #include @@ -586,6 +588,8 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, alloc_nm_rxq_hwq(vi, nm_rxq, tnl_cong(vi->pi, nm_cong_drop)); nm_rxq->fl_hwidx = hwidx; + nm_rxq->bb_zone = rxb->zone; + nm_rxq->bb = uma_zalloc(nm_rxq->bb_zone, M_WAITOK); slot = netmap_reset(na, NR_RX, i, 0); MPASS(slot != NULL); /* XXXNM: error check, not assert */ @@ -628,6 +632,31 @@ cxgbe_netmap_on(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, return (cxgbe_netmap_rss(sc, vi, ifp, na)); } +static void +flush_nm_rxq(struct adapter *sc, struct vi_info *vi, struct sge_nm_rxq *nm_rxq) +{ + int i, n; + u_int fl_pidx, fl_pidx_target, hw_cidx_desc; + const uint64_t ba = pmap_kextract((vm_offset_t)nm_rxq->bb); + + hw_cidx_desc = nm_rxq->fl_cidx / 8; + if (hw_cidx_desc == 0) + fl_pidx_target = nm_rxq->fl_sidx2 - 8; + else + fl_pidx_target = (hw_cidx_desc - 1) * 8; + MPASS((fl_pidx_target & 7) == 0); + + fl_pidx = nm_rxq->fl_pidx; + MPASS((fl_pidx & 7) == 0); + for (n = 0; fl_pidx != fl_pidx_target; n++) { + for (i = 0; i < 8; i++, fl_pidx++) + nm_rxq->fl_desc[fl_pidx] = htobe64(ba | nm_rxq->fl_hwidx); + if (__predict_false(fl_pidx == nm_rxq->fl_sidx2)) + fl_pidx = 0; + } + t4_write_reg(sc, sc->sge_kdoorbell_reg, nm_rxq->fl_db_val | V_PIDX(n)); +} + static int cxgbe_netmap_off(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, struct netmap_adapter *na) @@ -652,6 +681,23 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, if (rc != 0) return (rc); /* error message logged already. */ + /* + * First pass over the rx queues to make sure they're all caught up. + * + * The freelists could be out of buffers and we may need to arrange + * things so that any packets still in flight (after TP's cong_drop + * logic but not yet DMA'd) have somewhere to go and do not block the + * pipeline. Do this before trying to free any queue. + */ + for_each_nm_rxq(vi, i, nm_rxq) { + nm_state = atomic_load_int(&nm_rxq->nm_state); + kring = na->rx_rings[nm_rxq->nid]; + if (nm_state == NM_OFF || !nm_kring_pending_off(kring)) + continue; + MPASS(nm_rxq->iq_cntxt_id != INVALID_NM_RXQ_CNTXT_ID); + flush_nm_rxq(sc, vi, nm_rxq); + } + for_each_nm_txq(vi, i, nm_txq) { struct sge_qstat *spg = (void *)&nm_txq->desc[nm_txq->sidx]; @@ -688,6 +734,8 @@ cxgbe_netmap_off(struct adapter *sc, struct vi_info *vi, struct ifnet *ifp, pause("nmst", 1); free_nm_rxq_hwq(vi, nm_rxq); + uma_zfree(nm_rxq->bb_zone, nm_rxq->bb); + nm_rxq->bb = NULL; /* XXX: netmap, not the driver, should do this. */ kring->rhead = kring->rcur = kring->nr_hwcur = 0;