From nobody Thu Nov 04 09:18:39 2021 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 959F818291E2 for ; Thu, 4 Nov 2021 09:18:48 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from bizet.nethelp.no (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4HlJ3W75H2z3tCX for ; Thu, 4 Nov 2021 09:18:47 +0000 (UTC) (envelope-from sthaug@nethelp.no) Received: from localhost (bizet.nethelp.no [IPv6:2001:8c0:9e04:500::1]) by bizet.nethelp.no (Postfix) with ESMTP id D9B9EE604B; Thu, 4 Nov 2021 10:18:39 +0100 (CET) Date: Thu, 04 Nov 2021 10:18:39 +0100 (CET) Message-Id: <20211104.101839.22506556.sthaug@nethelp.no> To: Michael Pounov Cc: freebsd-net@freebsd.org Subject: Re: Broadcom bnxt driver issue with promisc mode From: sthaug@nethelp.no In-Reply-To: <20211103215952.fefc8cf8eb96ebaf22bb4935@elwix.org> References: <20211103215952.fefc8cf8eb96ebaf22bb4935@elwix.org> X-Mailer: Mew version 6.7 on Emacs 26 / Mule 6.0 (HANACHIRUSATO) List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4HlJ3W75H2z3tCX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N > After survey about solution for Broadcom bnxt driver issue with promiscous mode. I am not found any normal solution with that situation. > Only workaround to using promiscous mode to have normal operations with these cards. This isn't normal by me! > > I look and do some debug on bnxt driver. Issue came from hardware filter in ASIC. On init driver and its ring buffers default states of filter are all block. Also I saw that we have not ability to append configured VLANs on the host into broadcom vlan tag list to be allowed for further processing from network stack. > Then my solution is simple and effective in such situation. I append to filter mask bit for passing further for processing all tagged vlans and native/primary vlan too. > > This is enough to have normal packet processing for bnxt driver. > We have such BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller. I tested patch on them. > > Patch is make on FreeBSD 14.0-CURRENT #3 main-n244973-c02a28754bc-dirty: Tue Nov 2 20:53:08 EET 2021 > > If you are thinking it is ok. You can feel free to merge it to FreeBSD existing driver. This patch solved a big problem for me also. My environment is FreeBSD 12.2-STABLE (r370686), with a 2 port BCM57412. The bnxt interfaces work fine as long as I configure IP address on the main (physical) interface - no VLANs. If I configure a VLAN interface (e.g. bnxt1.91), this only works if I put the main interface in promiscuous mode. After updating if_bnxt.c with the patch from Michael Pounov, I am now able to use VLAN interfaces without putting the main interface in promiscuous mode. This is the output from pciconf -lv: bnxt0@pci0:24:0:0: class=0x020000 card=0x412014e4 chip=0x16d614e4 rev=0x01 hdr=0x00 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller' class = network subclass = ethernet bnxt1@pci0:24:0:1: class=0x020000 card=0x412014e4 chip=0x16d614e4 rev=0x01 hdr=0x00 vendor = 'Broadcom Inc. and subsidiaries' device = 'BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller' class = network subclass = ethernet Below I have enclosed the patch I used for 12.2-STABLE - the only difference, as far as I can see, is slightly different line numbers. Thank you Michael Pounov! Steinar Haug, Nethelp consulting, sthaug@nethelp.no ---------------------------------------------------------------------- --- sys/dev/bnxt/if_bnxt.c.orig 2021-09-28 21:25:17.000000000 +0200 +++ sys/dev/bnxt/if_bnxt.c 2021-11-04 07:59:24.720236000 +0100 @@ -587,7 +587,8 @@ softc->vnic_info.def_ring_grp = (uint16_t)HWRM_NA_SIGNATURE; softc->vnic_info.cos_rule = (uint16_t)HWRM_NA_SIGNATURE; softc->vnic_info.lb_rule = (uint16_t)HWRM_NA_SIGNATURE; - softc->vnic_info.rx_mask = HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_BCAST; + softc->vnic_info.rx_mask = HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_BCAST | + HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_ANYVLAN_NONVLAN; softc->vnic_info.mc_list_count = 0; softc->vnic_info.flags = BNXT_VNIC_FLAG_DEFAULT; rc = iflib_dma_alloc(ctx, BNXT_MAX_MC_ADDRS * ETHER_ADDR_LEN, @@ -1384,8 +1385,7 @@ HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_ANYVLAN_NONVLAN; else softc->vnic_info.rx_mask &= - ~(HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_PROMISCUOUS | - HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_ANYVLAN_NONVLAN); + ~(HWRM_CFA_L2_SET_RX_MASK_INPUT_MASK_PROMISCUOUS); rc = bnxt_hwrm_cfa_l2_set_rx_mask(softc, &softc->vnic_info);