Strange igb befavior
Jack Vogel
jfvogel at gmail.com
Thu May 27 16:18:39 UTC 2010
Well, it might make sense to increase the mbuf pool, do: sysctl
kern.ipc.nmbclusters
this is a parameter that can be set in /etc/sysctl.conf. Our testers set
this on all their
machines to 262144.
As for the aim stuff, and that includes those latency tuneables, they have
all been
replaced in my latest code with a simplified, more automatic approach.
Jack
On Thu, May 27, 2010 at 3:30 AM, Kirill Yelizarov <ykirill at yahoo.com> wrote:
> Thank You Jack
>
> i'll keep it at 2048 now. I have plans to add two more igb interfaces.
> Should i decrease values to 1024 in case i will have four interfaces in
> server?
>
> I found there are additional tweaks for igb card available:
> hw.igb.enable_aim=1 this one enabled by default and i understand its
> meaning
> Where can i read about the rest?
> hw.igb.low_latency=1000
> hw.igb.ave_latency=2000
> hw.igb.bulk_latency=4000
> hw.igb.rx_process_limit=400
> hw.igb.fc_setting=0
> hw.igb.lro=0
>
> Some years ago i downloaded an article about em card from intel site.
> Perhaps there is one for igb as well?
>
> Kirill
>
> --- On Thu, 5/27/10, Jack Vogel <jfvogel at gmail.com> wrote:
>
> > From: Jack Vogel <jfvogel at gmail.com>
> > Subject: Re: Strange igb befavior
> > To: "Kirill Yelizarov" <ykirill at yahoo.com>
> > Cc: freebsd-stable at freebsd.org
> > Date: Thursday, May 27, 2010, 1:06 PM
> > Panic is due to a failure to get
> > enough mbufs, when you make your ring that
> > big you
> > hit the problem, I have been experimenting with a change to
> > fix it but am
> > not yet
> > completely confident, for the moment don't make your ring
> > so big :)
> >
> > Jack
> >
> >
> > On Thu, May 27, 2010 at 1:08 AM, Kirill Yelizarov <ykirill at yahoo.com>
> > wrote:
> >
> > > Hi
> > >
> > > I'm having reproducible panics with 8-Stable of May13
> > 2010. Panic occurs in
> > > igb code. Panic start to happen when i set
> > hw.igb.rxd="4096" and
> > > hw.igb.txd="4096" in /boot/loader.conf. Panic happens
> > immediately after boot
> > > in igb1 code in my case. igb1 is connected to 100Mbit
> > 3COM switch and switch
> > > is not connected to anything else.
> > >
> > > Here is dmesg for igb
> > > # dmesg | grep igb
> > > igb0: <Intel(R) PRO/1000 Network Connection version
> > - 1.9.5> port
> > > 0x2020-0x203f mem
> > 0xb1a20000-0xb1a3ffff,0xb1a44000-0xb1a47fff irq 40 at
> > > device 0.0 on pci1
> > > igb0: Using MSIX interrupts with 5 vectors
> > > igb0: [ITHREAD]
> > > igb0: [ITHREAD]
> > > igb0: [ITHREAD]
> > > igb0: [ITHREAD]
> > > igb0: [ITHREAD]
> > > igb0: Ethernet address: 00:15:17:ba:2e:00
> > > igb1: <Intel(R) PRO/1000 Network Connection version
> > - 1.9.5> port
> > > 0x2000-0x201f mem
> > 0xb1a00000-0xb1a1ffff,0xb1a40000-0xb1a43fff irq 28 at
> > > device 0.1 on pci1
> > > igb1: Using MSIX interrupts with 5 vectors
> > > igb1: [ITHREAD]
> > > igb1: [ITHREAD]
> > > igb1: [ITHREAD]
> > > igb1: [ITHREAD]
> > > igb1: [ITHREAD]
> > > igb1: Ethernet address: 00:15:17:ba:2e:01
> > > igb1: link state changed to UP
> > > igb0: link state changed to UP
> > >
> > > border2# ifconfig
> > > igb0:
> > flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> > metric 0 mtu 1500
> > >
> > options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
> > > ether 00:15:17:ba:2e:00
> > > inet 192.168.10.2 netmask
> > 0xffffff00 broadcast 192.168.10.255
> > > inet 192.168.10.201 netmask
> > 0xffffffff broadcast 192.168.10.201
> > > inet 192.168.10.202 netmask
> > 0xffffffff broadcast 192.168.10.202
> > > inet 192.168.10.203 netmask
> > 0xffffffff broadcast 192.168.10.203
> > > inet 192.168.10.204 netmask
> > 0xffffffff broadcast 192.168.10.204
> > > media: Ethernet autoselect
> > (1000baseT <full-duplex>)
> > > status: active
> > > igb1:
> > flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST>
> > metric 0 mtu 1500
> > >
> > options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
> > > ether 00:15:17:ba:2e:01
> > > inet XXX.74.229.230 netmask
> > 0xfffffff0 broadcast XXX.74.229.239
> > > inet XXX.74.229.226 netmask
> > 0xffffffff broadcast XXX.74.229.226
> > > inet XXX.74.229.227 netmask
> > 0xffffffff broadcast XXX.74.229.227
> > > media: Ethernet autoselect
> > (100baseTX <full-duplex>)
> > > status: active
> > > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST>
> > metric 0 mtu 16384
> > >
> > options=3<RXCSUM,TXCSUM>
> > > inet 127.0.0.1 netmask
> > 0xff000000
> > > pfsync0: flags=0<> metric 0 mtu 1460
> > > syncpeer: 224.0.0.240
> > maxupd: 128
> > > pflog0: flags=141<UP,RUNNING,PROMISC> metric 0
> > mtu 33152
> > >
> > > #pciconf -lv
> > > igb0 at pci0:1:0:0:
> > class=0x020000 card=0x34de8086 chip=0x10a78086
> > > rev=0x02 hdr=0x00
> > > vendor = 'Intel
> > Corporation'
> > > device = '82575EB
> > Gigabit Network Connection'
> > > class = network
> > > subclass = ethernet
> > > igb1 at pci0:1:0:1:
> > class=0x020000 card=0x34de8086 chip=0x10a78086
> > > rev=0x02 hdr=0x00
> > > vendor = 'Intel
> > Corporation'
> > > device = '82575EB
> > Gigabit Network Connection'
> > > class = network
> > > subclass = ethernet
> > >
> > > # netstat -i
> > > Name Mtu Network
> > Address
> > Ipkts Ierrs Idrop Opkts
> > > Oerrs Coll
> > > igb0 1500 <Link#1>
> > 00:15:17:ba:2e:00 2315
> > 0 0
> > 1415
> > > 0 0
> > > igb0 1500 192.168.10.0
> > border2
> > 1664 -
> > - 1412
> > > - -
> > > igb0 1500 192.168.10.20
> > mysql-border2
> > 0 - -
> > 0
> > > - -
> > > igb0 1500 192.168.10.20
> > apache-border2
> > 0 -
> > - 0
> > > - -
> > > igb0 1500 192.168.10.20
> > squid-border2
> > 0 - -
> > 0
> > > - -
> > > igb0 1500 192.168.10.20
> > postgresql-border 1
> > - -
> > 0
> > > - -
> > > igb1 1500 <Link#2>
> > 00:15:17:ba:2e:01 129
> > 0 0
> > 0
> > > 0 0
> > > igb1 1500 XXX.74.229.22 border2
> >
> > 0 - -
> > 0
> > > - -
> > > igb1 1500 XXX.74.229.22
> > apache-border2
> > 0 -
> > - 0
> > > - -
> > > igb1 1500 XXX.74.229.22
> > squid-border2
> > 0 - -
> > 0
> > > - -
> > > lo0 16384 <Link#3>
> >
> > 1
> > 0 0
> > 1
> > > 0 0
> > > lo0 16384 your-net
> > localhost
> > 0 -
> > - 1
> > > - -
> > > pfsyn 1460 <Link#4>
> >
> > 0
> > 0 0
> > 0
> > > 0 0
> > > pflog 33152 <Link#5>
> >
> > 0
> > 0 0
> > 0
> > > 0 0
> > >
> > > There are several jails on this server.
> > >
> > > When i set hw.igb.rxd="2048" and hw.igb.txd="2048" i
> > don't have panic
> > > anymore.
> > >
> > > The reason i tried to add more buffers than default is
> > because i have NFS
> > > export on this server. But it is on igb0. igb1 is
> > currently doing nothing
> > > but it will soon once the server will be ready for
> > production.
> > >
> > > I didn't get core dump because my system is rather
> > old. But i can do it if
> > > needed.
> > >
> > > Regards,
> > > Kirill
> > >
> > >
> > >
> > > _______________________________________________
> > > freebsd-stable at freebsd.org
> > mailing list
> > > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > > To unsubscribe, send any mail to "
> freebsd-stable-unsubscribe at freebsd.org"
> > >
> > _______________________________________________
> > freebsd-stable at freebsd.org
> > mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org
> "
> >
>
>
>
>
More information about the freebsd-stable
mailing list