unable to set mtu 9000 on Intel x520-t2 adapter

[Jack Vogel]

Opps, i meant kern.ipc.nmb* of course...

Jack


On Fri, Jan 13, 2012 at 9:58 AM, Jack Vogel <jfvogel at gmail.com> wrote:

> Its a system resource thing, when you increase to a 9K mtu the driver will
> use 9K
> mbuf clusters, and with the queues and rings you have its unable to get
> enough.
>
> Very important to look at kern.ipc.nbm*
>
> In your case nmbjumbo9 is the pool, if you had 4K it would those clusters.
> Change the defaults in /etc/sysctl.conf.
>
> Cheers,
>
> Jack
>
>
>
> On Fri, Jan 13, 2012 at 9:22 AM, Iordan Iordanov <iordan at cdf.toronto.edu>wrote:
>
>> Hello,
>>
>> We are testing FreeBSD 9.0-RELEASE on a pair of machines with 10GBASE-T
>> Intel x520-t2 adapters interconnected with a straight-through cat6 rj45
>> cable. With mtu 1500, we are achieving aproximately 4 gigabits/s both
>> directions as demonstrated with iperf. We wanted to test with mtu 9000 to
>> see whether there would be any difference, but we encountered a problem
>> setting it. We ran the following commands to alter the MTU to 9000 on the
>> two machines (fw3 and fw4):
>>
>> fw3# ifconfig ix1 192.168.1.100 mtu 9000
>> fw4# ifconfig ix1 192.168.1.200 mtu 9000
>>
>> However, instead of getting mtu set to 9000, what we see is the following
>> message in the kernel output of both machines:
>>
>> ix1: Could not setup receive structures
>> ix1: Could not setup receive structures
>>
>> and no traffic getting through. The moment we run:
>>
>> fw3# ifconfig ix1 192.168.1.100 mtu 1500
>> fw4# ifconfig ix1 192.168.1.200 mtu 1500
>>
>> we see:
>>
>> ix1: link state changed to DOWN
>> ix1: link state changed to UP
>>
>> and traffic starts getting through as before. More information about our
>> hardware is:
>>
>> Motherboard: Supermicro H8DGU-F
>> CPU:         AMD 6128 8-core
>> Memory:      16GB DDR3
>> NICs:        Intel x520-t2 10GbaseT adapters
>>
>> An ifconfig ix1 output for one of the machines:
>>
>> fw4# ifconfig ix1
>> ix1: flags=8843<UP,BROADCAST,**RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
>> 1500
>>
>> options=401bb<RXCSUM,TXCSUM,**VLAN_MTU,VLAN_HWTAGGING,JUMBO_**
>> MTU,VLAN_HWCSUM,TSO4,VLAN_**HWTSO>
>>        ether 00:1b:21:d6:19:51
>>        inet 192.168.1.200 netmask 0xffffff00 broadcast 192.168.1.255
>>        nd6 options=29<PERFORMNUD,**IFDISABLED,AUTO_LINKLOCAL>
>>        media: Ethernet autoselect (autoselect <full-duplex>)
>>        status: active
>>
>>
>> Many thanks for any help/action on this!
>>
>> Sincerely,
>>
>> Iordan Iordanov
>> CDF System Administrator
>> University of Toronto
>> ______________________________**_________________
>> freebsd-bugs at freebsd.org mailing list
>> http://lists.freebsd.org/**mailman/listinfo/freebsd-bugs<http://lists.freebsd.org/mailman/listinfo/freebsd-bugs>
>> To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@**freebsd.org<freebsd-bugs-unsubscribe at freebsd.org>
>> "
>>
>
>