ixgbe(4) and "Could not setup receive structures"

[K. Macy]

On Thu, Apr 14, 2011 at 9:44 PM, Leon Meßner
<l.messner at physik.tu-berlin.de> wrote:
> On Thu, Apr 14, 2011 at 08:55:17PM +0200, K. Macy wrote:
>> That isn't guaranteed to work if he is KVA limited.
>>
>> On Thu, Apr 14, 2011 at 6:44 PM, Jack Vogel <jfvogel at gmail.com> wrote:
>> > If you get this message its only for one reason, you don't have enough mbufs
>> > to
>> > fill your rings. You must do one of two things, either reduce the number of
>> > queues,
>> > or increase the relevant mbuf pool.
>> >
>> > Increase the 9K mbuf cluster pool.
>
> I did set it to twice the default, and now it works and netstat -m
> shows:
>
> 8192/391/8583/12800 9k jumbo clusters in use (current/cache/total/max)
>
> Whats a reasonable amount to set kern.ipc.nmbjumbo9 to and is there
> any
> form of auto-tuning (i have absolutely no load on this machine and
> mbufs
> are higher than default pool size).


The auto-tuning for jumbo clusters works poorly at best. It isn't load
consuming them, it is the preallocation to large receive queues.

>
> Thanks to all,
> Leon
>
>> > On Thu, Apr 14, 2011 at 6:05 AM, Leon Meßner
>> > <l.messner at physik.tu-berlin.de>wrote:
>> >
>> >> Hi,
>> >>
>> >> i tried setting the mtu on one of my ixgbe(4) intel NICs to support
>> >> jumbo frames. This is on a box with RELENG_8 from today.
>> >>
>> >> # ifconfig ix0 mtu 9198
>> >>
>> >> I then get the following error:
>> >>
>> >> # tail -n 1 /var/log/messages
>> >> Apr 14 12:48:43 siloneu kernel: ix0: Could not setup receive structures
>> >>
>> >> I already tried the following patch because of Jack Vogel's advice given
>> >> in the following thread on -stable in Oct. last year, which still
>> >> produces the same error message and leaves the box unpingable:
>> >>
>> >> http://lists.freebsd.org/pipermail/freebsd-stable/2010-October/059541.html
>> >>
>> >> # cat ~/patches/ixgbe.num_queues_to_4.patch
>> >> --- /root/.vimbackup/ixgbe.c~   2011-04-12 22:14:27.000000000 +0000
>> >> +++ sys/dev/ixgbe/ixgbe.c       2011-04-12 22:14:27.000000000 +0000
>> >> @@ -273,7 +273,7 @@ TUNABLE_INT("hw.ixgbe.hdr_split", &ixgbe
>> >>  * number of cpus. Each queue is a pair
>> >>    * of RX and TX rings with a msix vector
>> >>      */
>> >>      -static int ixgbe_num_queues = 0;
>> >>      +static int ixgbe_num_queues = 4;
>> >>       TUNABLE_INT("hw.ixgbe.num_queues", &ixgbe_num_queues);
>> >>
>> >>         /*
>> >>
>> >> _______________________________________________
>> >> freebsd-stable at freebsd.org mailing list
>> >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>> >>
>> > _______________________________________________
>> > freebsd-stable at freebsd.org mailing list
>> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>> >
>> _______________________________________________
>> freebsd-stable at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>