Changing MTU on cxgbe

[Navdeep Parhar]

On Thu, May 26, 2016 at 11:52:45PM -0500, Dustin Marquess wrote:
> After my many issues with ixgbe & ixv, I ended up removing the Intel
> X520s with Chelsio T420-CR and the Intel X710s with Chelsio T520-CR.
> So far so good, except I can't seem to change the MTU away from 1500.
> In fact, ifconfig can't seem to change the mtu at all.  On -CURRENT as
> of yesterday:
> 
> # ifconfig -m cxgbe0
> cxgbe0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
> metric 0 mtu 1500
> options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> capabilities=ecc7bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,TOE4,TOE6,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
> ether 00:07:43:11:2b:80
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet 10Gbase-Twinax <full-duplex>
> status: active
> supported media:
> media 10Gbase-Twinax mediaopt full-duplex
> 
> # ifconfig cxgbe0 mtu 9000
> ifconfig: ioctl SIOCSIFMTU (set mtu): Invalid argument
> 
> # ifconfig cxgbe0 mtu 1500
> ifconfig: ioctl SIOCSIFMTU (set mtu): Invalid argument
> 
> Any ideas?

It should have worked.  I tried just now on a system running -CURRENT as
of today (r300785) and didn't encounter any problems.

# ifconfig cxl0 up
# ifconfig cxl0 mtu 500
# ifconfig cxl0 mtu 3333
# ifconfig cxl0 mtu 7992
...

Is the cxgbe/cxl ifnet part of a lagg or bridge or something like that?
Run this while you try the ifconfig and see what ioctls are failing with
EINVAL.  You may have to do a "kldload dtraceall" before this.

# dtrace -n 'fbt::*_ioctl:return /arg1 == 22/ {}'

> Also, the Chelsios don't seem to use iovctl to do SR-IOV like the
> Intels did (I'm using bhyve).  I assume that's what the
> hw.cxgbe.num_vis loader setting is for?

num_vis does create multiple autonomous ifnets that share the same
physical port but it doesn't involve SR-IOV.

Regards,
Navdeep