From nobody Mon May 08 13:50:03 2023 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QFN2l1zjXz49x0d for ; Mon, 8 May 2023 13:50:07 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4QFN2j6sPCz4NQT; Mon, 8 May 2023 13:50:05 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm3 header.b=BV36qjwS; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=M2w5G45c; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 66.111.4.29 as permitted sender) smtp.mailfrom=yuri@aetern.org Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 08A145C003F; Mon, 8 May 2023 09:50:05 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Mon, 08 May 2023 09:50:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1683553805; x=1683640205; bh=EgodNuHwyxCu13MlOLp0DntcAL7x60l8FBn 5KViZt7Q=; b=BV36qjwSqEoSpULUZ2qW9lxvg/yh0eTbco2y3PPnKCmtJnS54fs EvlHv9Hrka8trH76IKO8UK/bM9BDRHC9YKDCvOXXV4r3nDbcCLKkuwp+lh/gFhhf TzGXn4mNXrh0EbAOCTXO20EUR+3BdFvPmPYANOnVcR4bpjOUAcopuHb/lv5vILTQ JSHG7GziQ407jzhOK9+lEfdGum/2KWaAbxbwqGR4D8huTHOorI0xDltEuLhdikWe 5cRlLbSGVebNodkM9nvAmcnyLrg4uNlowczoqvkIqNeuZt99bxQQWjGXQkFGg9cu es2i5SCdPHvePWG09lhtAZFggce/G1cRvWQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1683553805; x=1683640205; bh=EgodNuHwyxCu13MlOLp0DntcAL7x60l8FBn 5KViZt7Q=; b=M2w5G45cHoXXEin7MxsoUsWHJLhfWzfAgHnYi0fqBUFETz7btwZ lFiPzkuOgBWEEJGscvcQhOgtyNVs2wOs6u1E7NFGiuLYzXfQSUGLF4IVVAfX31/d AR4QC/tUYQUSx3FFsB6qGAFHLn39aPylN4XETEe1NTVaYhbcE2VIxhMZqqn9UMiI lmq6vegrolvEf/mHo7nsjK/73KeRZi5Lz+A0a3xiX32vl0bK+lmw3R0ammMVSG6K yReQvf85Kps5m33ZLFe5+jt5a9L9LLKbZ51ChrUb3Iq/2Fgq/TbU+cOh4zzkz8LY FQc0GOnberhloWM0Jj4R1pdvM9msnU33tdA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfeefkedgjedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpegjuhhr ihcuoeihuhhrihesrggvthgvrhhnrdhorhhgqeenucggtffrrghtthgvrhhnpefhueevhf ehleeifeejgfetgfffiefghfdtfefhvddvjeekgfektdfgieffleeiteenucffohhmrghi nhepfhhrvggvsghsugdrohhrghdpthigqhdtrdhnohdpvghmrdhsmhgrrhhtnecuvehluh hsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihurhhisegrvght vghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 8 May 2023 09:50:04 -0400 (EDT) Message-ID: <738dfff6-8a63-f462-e0ba-1be3b966a99b@aetern.org> Date: Mon, 8 May 2023 15:50:03 +0200 List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.1 Subject: Re: missing hw.em.msix support in FreeBSD 14.0-CURRENT? Content-Language: en-US To: Cheng Cui Cc: freebsd-net@freebsd.org References: From: Yuri In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4QFN2j6sPCz4NQT X-Spamd-Bar: / X-Spamd-Result: default: False [0.60 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm3,messagingengine.com:s=fm3]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; local_wl_from(0.00)[yuri@aetern.org]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Cheng Cui wrote: > A followup question: Shouldn't the man page for em(4) be updated > accordingly, as the sysctls are different now? > > https://man.freebsd.org/cgi/man.cgi?em(4) > https://reviews.freebsd.org/D39999 > On Sun, May 7, 2023 at 9:46 PM Cheng Cui > wrote: > > Found the RSS support, only if the hardware is ">= e1000_82571" from > the if_em.c. > > |if (hw->mac.type >= em_mac_min) // #define em_mac_min > e1000_82571 ... scctx->isc_txrx = &em_txrx; ...| > > > Best Regards, > Cheng Cui > > > On Sun, May 7, 2023 at 7:28 PM Cheng Cui > wrote: > > Hi Yuri, > > Thanks for the update. I am using an Emulab testbed node under > the same type of hardware. Below are the new outputs. > It is supposed to have multiple tx/rx queues under msix. Why are > there only rxq0/txq0? Is it because the CPU has only one core? > How can I check if RSS is supported? > > root@s1:~ # sysctl -d dev.em.0.iflib.disable_msix > dev.em.0.iflib.disable_msix: disable MSI-X (default 0) > root@s1:~ # sysctl dev.em.0.iflib.disable_msix > dev.em.0.iflib.disable_msix: 0 > root@s1:~ # sysctl dev.em.0.iflib > dev.em.0.iflib.rxq0.rxq_fl0.buf_size: 2048 > dev.em.0.iflib.rxq0.rxq_fl0.credits: 1023 > dev.em.0.iflib.rxq0.rxq_fl0.cidx: 959 > dev.em.0.iflib.rxq0.rxq_fl0.pidx: 958 > dev.em.0.iflib.rxq0.cpu: 1 > dev.em.0.iflib.txq0.r_abdications: 0 > dev.em.0.iflib.txq0.r_restarts: 0 > dev.em.0.iflib.txq0.r_stalls: 0 > dev.em.0.iflib.txq0.r_starts: 110443 > dev.em.0.iflib.txq0.r_drops: 0 > dev.em.0.iflib.txq0.r_enqueues: 110444 > dev.em.0.iflib.txq0.ring_state: pidx_head: 1903 pidx_tail: 1903 > cidx: 1903 state: IDLE > dev.em.0.iflib.txq0.txq_cleaned: 115208 > dev.em.0.iflib.txq0.txq_processed: 115251 > dev.em.0.iflib.txq0.txq_in_use: 42 > dev.em.0.iflib.txq0.txq_cidx_processed: 565 > dev.em.0.iflib.txq0.txq_cidx: 525 > dev.em.0.iflib.txq0.txq_pidx: 568 > dev.em.0.iflib.txq0.no_tx_dma_setup: 0 > dev.em.0.iflib.txq0.txd_encap_efbig: 0 > dev.em.0.iflib.txq0.tx_map_failed: 0 > dev.em.0.iflib.txq0.no_desc_avail: 0 > dev.em.0.iflib.txq0.mbuf_defrag_failed: 0 > dev.em.0.iflib.txq0.m_pullups: 0 > dev.em.0.iflib.txq0.mbuf_defrag: 0 > dev.em.0.iflib.txq0.cpu: 0 > dev.em.0.iflib.override_nrxds: 0 > dev.em.0.iflib.override_ntxds: 0 > dev.em.0.iflib.use_logical_cores: 0 > dev.em.0.iflib.separate_txrx: 0 > dev.em.0.iflib.core_offset: 0 > dev.em.0.iflib.tx_abdicate: 0 > dev.em.0.iflib.rx_budget: 0 > dev.em.0.iflib.disable_msix: 0 > dev.em.0.iflib.override_qs_enable: 0 > dev.em.0.iflib.override_nrxqs: 0 > dev.em.0.iflib.override_ntxqs: 0 > dev.em.0.iflib.driver_version: 7.7.8-fbsd > root@s1:~ # > > > Best Regards, > Cheng Cui > > > On Sun, May 7, 2023 at 4:18 PM Yuri > wrote: > > Cheng Cui wrote: > > Hello, > > > > I am using this em(4) driver for some tests. According to > the man page > > below, there should be a "hw.em.msix" sysctl to > enable/disable MSI-X > > style interrupts. But why is it missing in my system below? > > https://man.freebsd.org/cgi/man.cgi?em(4) > > > > > > > > root@s1:~ # dmesg | egrep "8254" > > Timecounter "i8254" frequency 1193182 Hz quality 0 > > Event timer "i8254" frequency 1193182 Hz quality 100 > > em0: port > 0xdcc0-0xdcff mem > > 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6 > > em1: port > 0xccc0-0xccff mem > > 0xdf8e0000-0xdf8fffff irq 65 at device 8.0 on pci7 > > em2: port > 0xbcc0-0xbcff > > mem 0xdf5e0000-0xdf5fffff irq 106 at device 4.0 on pci9 > > em3: port > 0xbc80-0xbcbf > > mem 0xdf5c0000-0xdf5dffff irq 107 at device 4.1 on pci9 > > em4: port > 0xacc0-0xacff > > mem 0xdf3e0000-0xdf3fffff irq 101 at device 3.0 on pci10 > > em5: port > 0xac80-0xacbf > > mem 0xdf3c0000-0xdf3dffff irq 102 at device 3.1 on pci10 > > > > root@s1:~ # ifconfig em5 > > em5: flags=8843 > metric 0 mtu 1500 > > > options=481009b > > ether 00:04:23:b7:1e:1d > > inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255 > > media: Ethernet 1000baseT > > status: active > > nd6 options=29 > > > > root@s1:~ # sysctl hw.em > > hw.em.max_interrupt_rate: 8000 > > hw.em.eee_setting: 1 > > hw.em.rx_process_limit: 100 > > hw.em.sbp: 0 > > hw.em.smart_pwr_down: 0 > > hw.em.rx_abs_int_delay: 66 > > hw.em.tx_abs_int_delay: 66 > > hw.em.rx_int_delay: 0 > > hw.em.tx_int_delay: 66 > > hw.em.disable_crc_stripping: 0 > > root@s1:~ # > > As if_em was converted to iflib, I guess the related sysctl > would now be > dev.em.0.iflib.disable_msix (I could be wrong, don't have > h/w to check). >