ixgbe(4) and "Could not setup receive structures"

[Leon Meßner]

On Thu, Apr 14, 2011 at 04:20:19PM +0200, K. Macy wrote:
> That should be plenty, but how large are your receive queues?
> 
How do i show that ? Random sysctl output following ;)

# sysctl net.inet.tcp.recvspace
net.inet.tcp.recvspace: 65536

# sysctl -a | grep ix.0.queue\*
dev.ix.0.queue0.interrupt_rate: 1000000
dev.ix.0.queue0.txd_head: 59
dev.ix.0.queue0.txd_tail: 59
dev.ix.0.queue0.no_desc_avail: 0
dev.ix.0.queue0.tx_packets: 32
dev.ix.0.queue0.rxd_head: 942
dev.ix.0.queue0.rxd_tail: 941
dev.ix.0.queue0.rx_packets: 942
dev.ix.0.queue0.rx_bytes: 60114
dev.ix.0.queue0.lro_queued: 0
dev.ix.0.queue0.lro_flushed: 0
dev.ix.0.queue1.interrupt_rate: 1000000
dev.ix.0.queue1.txd_head: 0
dev.ix.0.queue1.txd_tail: 0
dev.ix.0.queue1.no_desc_avail: 0
dev.ix.0.queue1.tx_packets: 0
dev.ix.0.queue1.rxd_head: 106
dev.ix.0.queue1.rxd_tail: 105
dev.ix.0.queue1.rx_packets: 106
dev.ix.0.queue1.rx_bytes: 12215
dev.ix.0.queue1.lro_queued: 0
dev.ix.0.queue1.lro_flushed: 0
dev.ix.0.queue2.interrupt_rate: 1000000
dev.ix.0.queue2.txd_head: 0
dev.ix.0.queue2.txd_tail: 0
dev.ix.0.queue2.no_desc_avail: 0
dev.ix.0.queue2.tx_packets: 0
dev.ix.0.queue2.rxd_head: 68
dev.ix.0.queue2.rxd_tail: 67
dev.ix.0.queue2.rx_packets: 68
dev.ix.0.queue2.rx_bytes: 17965
dev.ix.0.queue2.lro_queued: 0
dev.ix.0.queue2.lro_flushed: 0
dev.ix.0.queue3.interrupt_rate: 111111
dev.ix.0.queue3.txd_head: 817
dev.ix.0.queue3.txd_tail: 821
dev.ix.0.queue3.no_desc_avail: 0
dev.ix.0.queue3.tx_packets: 1323
dev.ix.0.queue3.rxd_head: 1396
dev.ix.0.queue3.rxd_tail: 1395
dev.ix.0.queue3.rx_packets: 1396
dev.ix.0.queue3.rx_bytes: 145665
dev.ix.0.queue3.lro_queued: 0
dev.ix.0.queue3.lro_flushed: 0



> On Thu, Apr 14, 2011 at 4:18 PM, Leon Meßner
> <l.messner at physik.tu-berlin.de> wrote:
> > On Thu, Apr 14, 2011 at 03:44:23PM +0200, K. Macy wrote:
> >> How many 9k jumbo clusters are available?
> >
> > Does this output suffice as information ?
> >
> > # netstat -m
> > 8194/1031/9225 mbufs in use (current/cache/total)
> > 8192/518/8710/25600 mbuf clusters in use (current/cache/total/max)
> > 8192/512 mbuf+clusters out of packet secondary zone in use
> > (current/cache)
> > 0/5/5/12800 4k (page size) jumbo clusters in use
> > (current/cache/total/max)
> > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max)
> > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max)
> > 18432K/1313K/19746K bytes allocated to network (current/cache/total)
> > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> > 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> > 0/0/0 sfbufs in use (current/peak/max)
> > 0 requests for sfbufs denied
> > 0 requests for sfbufs delayed
> > 0 requests for I/O initiated by sendfile
> > 0 calls to protocol drain routines
> >
> >> On Thu, Apr 14, 2011 at 3:05 PM, Leon Meßner
> >> <l.messner at physik.tu-berlin.de> wrote:
> >> > Hi,
> >> >
> >> > i tried setting the mtu on one of my ixgbe(4) intel NICs to support
> >> > jumbo frames. This is on a box with RELENG_8 from today.
> >> >
> >> > # ifconfig ix0 mtu 9198
> >> >
> >> > I then get the following error:
> >> >
> >> > # tail -n 1 /var/log/messages
> >> > Apr 14 12:48:43 siloneu kernel: ix0: Could not setup receive structures
> >> >
> >> > I already tried the following patch because of Jack Vogel's advice given
> >> > in the following thread on -stable in Oct. last year, which still
> >> > produces the same error message and leaves the box unpingable:
> >> >
> >> > http://lists.freebsd.org/pipermail/freebsd-stable/2010-October/059541.html
> >> >
> >> > # cat ~/patches/ixgbe.num_queues_to_4.patch
> >> > --- /root/.vimbackup/ixgbe.c~   2011-04-12 22:14:27.000000000 +0000
> >> > +++ sys/dev/ixgbe/ixgbe.c       2011-04-12 22:14:27.000000000 +0000
> >> > @@ -273,7 +273,7 @@ TUNABLE_INT("hw.ixgbe.hdr_split", &ixgbe
> >> >  * number of cpus. Each queue is a pair
> >> >    * of RX and TX rings with a msix vector
> >> >      */
> >> >      -static int ixgbe_num_queues = 0;
> >> >      +static int ixgbe_num_queues = 4;
> >> >       TUNABLE_INT("hw.ixgbe.num_queues", &ixgbe_num_queues);
> >> >
> >> >         /*
> >> >
> >> > _______________________________________________
> >> > freebsd-stable at freebsd.org mailing list
> >> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> >> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
> >> >
> > _______________________________________________
> > freebsd-stable at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
> >