Re: missing hw.em.msix support in FreeBSD 14.0-CURRENT?

From: Yuri <yuri_at_aetern.org>
Date: Mon, 08 May 2023 13:50:03 UTC
Cheng Cui wrote:
> A followup question: Shouldn't the man page for em(4) be updated
> accordingly, as the sysctls are different now?
> 
> https://man.freebsd.org/cgi/man.cgi?em(4)
> <https://man.freebsd.org/cgi/man.cgi?em(4)>

https://reviews.freebsd.org/D39999

> On Sun, May 7, 2023 at 9:46 PM Cheng Cui <cc@freebsd.org
> <mailto:cc@freebsd.org>> wrote:
> 
>     Found the RSS support, only if the hardware is ">= e1000_82571" from
>     the if_em.c.
> 
>         |if (hw->mac.type >= em_mac_min) // #define em_mac_min
>         e1000_82571 ... scctx->isc_txrx = &em_txrx; ...|
> 
> 
>     Best Regards,
>     Cheng Cui
> 
> 
>     On Sun, May 7, 2023 at 7:28 PM Cheng Cui <cc@freebsd.org
>     <mailto:cc@freebsd.org>> wrote:
> 
>         Hi Yuri,
> 
>         Thanks for the update. I am using an Emulab testbed node under
>         the same type of hardware. Below are the new outputs.
>         It is supposed to have multiple tx/rx queues under msix. Why are
>         there only rxq0/txq0? Is it because the CPU has only one core?
>         How can I check if RSS is supported?
> 
>         root@s1:~ # sysctl -d dev.em.0.iflib.disable_msix
>         dev.em.0.iflib.disable_msix: disable MSI-X (default 0)
>         root@s1:~ # sysctl dev.em.0.iflib.disable_msix
>         dev.em.0.iflib.disable_msix: 0
>         root@s1:~ # sysctl dev.em.0.iflib
>         dev.em.0.iflib.rxq0.rxq_fl0.buf_size: 2048
>         dev.em.0.iflib.rxq0.rxq_fl0.credits: 1023
>         dev.em.0.iflib.rxq0.rxq_fl0.cidx: 959
>         dev.em.0.iflib.rxq0.rxq_fl0.pidx: 958
>         dev.em.0.iflib.rxq0.cpu: 1
>         dev.em.0.iflib.txq0.r_abdications: 0
>         dev.em.0.iflib.txq0.r_restarts: 0
>         dev.em.0.iflib.txq0.r_stalls: 0
>         dev.em.0.iflib.txq0.r_starts: 110443
>         dev.em.0.iflib.txq0.r_drops: 0
>         dev.em.0.iflib.txq0.r_enqueues: 110444
>         dev.em.0.iflib.txq0.ring_state: pidx_head: 1903 pidx_tail: 1903
>         cidx: 1903 state: IDLE
>         dev.em.0.iflib.txq0.txq_cleaned: 115208
>         dev.em.0.iflib.txq0.txq_processed: 115251
>         dev.em.0.iflib.txq0.txq_in_use: 42
>         dev.em.0.iflib.txq0.txq_cidx_processed: 565
>         dev.em.0.iflib.txq0.txq_cidx: 525
>         dev.em.0.iflib.txq0.txq_pidx: 568
>         dev.em.0.iflib.txq0.no_tx_dma_setup: 0
>         dev.em.0.iflib.txq0.txd_encap_efbig: 0
>         dev.em.0.iflib.txq0.tx_map_failed: 0
>         dev.em.0.iflib.txq0.no_desc_avail: 0
>         dev.em.0.iflib.txq0.mbuf_defrag_failed: 0
>         dev.em.0.iflib.txq0.m_pullups: 0
>         dev.em.0.iflib.txq0.mbuf_defrag: 0
>         dev.em.0.iflib.txq0.cpu: 0
>         dev.em.0.iflib.override_nrxds: 0
>         dev.em.0.iflib.override_ntxds: 0
>         dev.em.0.iflib.use_logical_cores: 0
>         dev.em.0.iflib.separate_txrx: 0
>         dev.em.0.iflib.core_offset: 0
>         dev.em.0.iflib.tx_abdicate: 0
>         dev.em.0.iflib.rx_budget: 0
>         dev.em.0.iflib.disable_msix: 0
>         dev.em.0.iflib.override_qs_enable: 0
>         dev.em.0.iflib.override_nrxqs: 0
>         dev.em.0.iflib.override_ntxqs: 0
>         dev.em.0.iflib.driver_version: 7.7.8-fbsd
>         root@s1:~ #
> 
> 
>         Best Regards,
>         Cheng Cui
> 
> 
>         On Sun, May 7, 2023 at 4:18 PM Yuri <yuri@aetern.org
>         <mailto:yuri@aetern.org>> wrote:
> 
>             Cheng Cui wrote:
>             > Hello,
>             >
>             > I am using this em(4) driver for some tests. According to
>             the man page
>             > below, there should be a "hw.em.msix" sysctl to
>             enable/disable MSI-X
>             > style interrupts. But why is it missing in my system below?
>             > https://man.freebsd.org/cgi/man.cgi?em(4)
>             <https://man.freebsd.org/cgi/man.cgi?em(4)>
>             > <https://man.freebsd.org/cgi/man.cgi?em(4)
>             <https://man.freebsd.org/cgi/man.cgi?em(4)>>
>             >
>             > root@s1:~ # dmesg | egrep "8254"
>             > Timecounter "i8254" frequency 1193182 Hz quality 0
>             > Event timer "i8254" frequency 1193182 Hz quality 100
>             > em0: <Intel(R) Legacy PRO/1000 MT 82541GI> port
>             0xdcc0-0xdcff mem
>             > 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6
>             > em1: <Intel(R) Legacy PRO/1000 MT 82541GI> port
>             0xccc0-0xccff mem
>             > 0xdf8e0000-0xdf8fffff irq 65 at device 8.0 on pci7
>             > em2: <Intel(R) Legacy PRO/1000 MT 82546EB (Copper)> port
>             0xbcc0-0xbcff
>             > mem 0xdf5e0000-0xdf5fffff irq 106 at device 4.0 on pci9
>             > em3: <Intel(R) Legacy PRO/1000 MT 82546EB (Copper)> port
>             0xbc80-0xbcbf
>             > mem 0xdf5c0000-0xdf5dffff irq 107 at device 4.1 on pci9
>             > em4: <Intel(R) Legacy PRO/1000 MT 82546EB (Copper)> port
>             0xacc0-0xacff
>             > mem 0xdf3e0000-0xdf3fffff irq 101 at device 3.0 on pci10
>             > em5: <Intel(R) Legacy PRO/1000 MT 82546EB (Copper)> port
>             0xac80-0xacbf
>             > mem 0xdf3c0000-0xdf3dffff irq 102 at device 3.1 on pci10
>             >
>             > root@s1:~ # ifconfig em5
>             > em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
>             metric 0 mtu 1500
>             >
>             options=481009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWFILTER,NOMAP>
>             > ether 00:04:23:b7:1e:1d
>             > inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255
>             > media: Ethernet 1000baseT <full-duplex>
>             > status: active
>             > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>             >
>             > root@s1:~ # sysctl hw.em
>             > hw.em.max_interrupt_rate: 8000
>             > hw.em.eee_setting: 1
>             > hw.em.rx_process_limit: 100
>             > hw.em.sbp: 0
>             > hw.em.smart_pwr_down: 0
>             > hw.em.rx_abs_int_delay: 66
>             > hw.em.tx_abs_int_delay: 66
>             > hw.em.rx_int_delay: 0
>             > hw.em.tx_int_delay: 66
>             > hw.em.disable_crc_stripping: 0
>             > root@s1:~ #
> 
>             As if_em was converted to iflib, I guess the related sysctl
>             would now be
>             dev.em.0.iflib.disable_msix (I could be wrong, don't have
>             h/w to check).
>