em3 no carrier
John-Mark Gurney
jmg at funkthat.com
Wed Dec 18 01:03:55 UTC 2013
Claude Marinier wrote this message on Tue, Dec 17, 2013 at 19:43 -0500:
> -----Original Message-----
> From: Michael Tuexen [mailto:Michael.Tuexen at lurchi.franken.de]
> Sent: Tuesday, December 17, 2013 5:24 PM
> To: Claude Marinier
> Cc: freebsd-net at freebsd.org
> Subject: Re: em3 no carrier
>
> On Dec 17, 2013, at 10:51 PM, Claude Marinier <claude.marinier at cae.com> wrote:
>
> >
> >>> -----Original Message-----
> >>> From: Rodrigo Osorio [mailto:rodrigo at bebik.net]
> >>> Sent: Tuesday, December 17, 2013 11:40 AM
> >>> To: Claude Marinier
> >>> Cc: freebsd-net at freebsd.org
> >>> Subject: Re: em3 no carrier
> >>>
> >>> Hi,
> >>>
> >>> Can you give us more informations about the network card / chips ?
> >>> Did you found any error in /var/log/ ?
> >>>
> >>> - rodrigo
> >>
> >> My apologies, I missed that bit of information.
> >>
> >> Yes, there is an error in /var/log/messages.
> >>
> >> Dec 17 11:33:07 WANemu bird: OSPF: Socket error on em3: No buffer
> >> space available Dec 17 11:33:47 WANemu last message repeated 4 times
> >> Dec 17 11:35:57 WANemu last message repeated 13 times Dec 17 11:45:57
> >> WANemu last message repeated 60 times Dec 17 11:55:47 WANemu last
> >> message repeated 59 times
> >>
> >>
> >> Also note that the HP NC364T uses an Intel 82571EB chipset.
> >>
> >>> On 17/12/13 11:27 -0500, Claude Marinier wrote:
> >>>> Hi,
> >>>>
> >>>> This is a WAN emulator (ipfw, DummyNet, and BIRD). FreeBSD 9.2
> >>>> Release (amd64) is running on DL360 G5 with a new NC364T quad-port
> >>>> Ethernet NIC. I installed the new NIC yesterday. The four ports are
> >>>> connected to routers in a lab.
> >>>>
> >>>> FreeBSD WAMemu 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep
> >>>> 26 22:50:31 UTC 2013 root at
> >>>> bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> >>>>
> >>>> Dec 16 17:13:19 WANemu kernel: em3: <Intel(R) PRO/1000 Network
> >>>> Connection 7.3.8> port 0x6020-0x603f mem
> >>>> 0xfdee0000-0xfdefffff,0xfde00000-0xfde7ffff irq 17 at device 0.1 on
> >>>> pci22 Dec 16 17:13:19 WANemu kernel: em3: Using an MSI interrupt
> >>>> Dec 16 17:13:19 WANemu kernel: em3: Ethernet address:
> >>>> e8:39:35:13:21:6e
> >>>>
> >>>> The four em interfaces are configured the same way (lines in
> >>>> rc.conf are identical except for IP address.
> >>>>
> >>>> ifconfig_em3="inet x.y.113.197/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em0="inet x.y.113.14/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em2="inet x.y.113.109/29 media 100baseTX mediaopt full-duplex"
> >>>> ifconfig_em1="inet x.y.113.189/29 media 100baseTX mediaopt full-duplex"
> >>>>
> >>>> The GE 0/1 port on the Cisco 2821 shows no link and ifconfig shows
> >>>> "no carrier" for em3. Oddly, it also shows autoselect.
> >>>>
> >>>> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6d
> >>>> inet x.y.113.14 netmask 0xfffffff8 broadcast x.y.113.15
> >>>> inet6 fe80::ea39:35ff:fe13:216d%em0 prefixlen 64 scopeid 0x1
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6c
> >>>> inet x.y.113.189 netmask 0xfffffff8 broadcast x.y.113.191
> >>>> inet6 fe80::ea39:35ff:fe13:216c%em1 prefixlen 64 scopeid 0x2
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6f
> >>>> inet 131.140.113.109 netmask 0xfffffff8 broadcast 131.140.113.111
> >>>> inet6 fe80::ea39:35ff:fe13:216f%em2 prefixlen 64 scopeid 0x3
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex>
> >>>> status: active
> >>>> em3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> >>>> options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> >>>> ether e8:39:35:13:21:6e
> >>>> inet 131.140.113.197 netmask 0xfffffff8 broadcast 131.140.113.199
> >>>> inet6 fe80::ea39:35ff:fe13:216e%em3 prefixlen 64 scopeid 0x4
> >>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> >>>> media: Ethernet 100baseTX <full-duplex> (autoselect)
> >>>> status: no carrier
> >>>>
> >>>> I have tried three Ethernet cables with no difference. This
> >>>> morning, I briefly booted Puppy Linux from USB and manually
> >>>> configured the four Ethernet interfaces. They all work (ping good
> >>>> to all four routers); the Cisco on em3 shows happy lights. I then
> >>>> booted back into FreeBSD, the problem with em3 remains. I have not
> >>>> had any success searching with Google; I may not have used suitable search terms.
> >>>>
> >>>> One more odd symptom: bmon has trouble with these interfaces.
> >>>>
> >>>> em0è95^S!m on WANemu
> >>>> Name
> >>>> qqqqqqqqqqqqqqqqqqqqqqqq
> >>>> WANemu (local)
> >>>> 0 em0è95^S!m
> >>>> 1 em1è95^S!l
> >>>> 2 em2è95^S!o
> >>>> 3 em3è95^S!n
> >>>> 4 usbus0
> >>>> 5 usbus1
> >>>> 6 usbus2
> >>>> 7 usbus3
> >>>> 8 usbus4
> >>>> 9 usbus5
> >>>> 10 ipfw0
> >>>> 11 lo0
> >>>>
> >>>> It looks like FreeBSD is somehow mishandling these interfaces.
> >>>> Before purchasing it, I searched with Google and found reports of
> >>>> people using this device without problems. I do not know how to proceed.
> >>>>
> >>>> Thank you for your help.
> >
> > Oddly, replacing em3 with bce1 left the system with only two
> > functionning interfaces. The order of the interface definitions in
> > rc.local makes a difference. The worst order is bce1, em0, em2, em1
> > which left only
> > em0 and em2 working. Changing the order to bce1, em0, em1, em2 allows
> > the three 'em' interfaces to work. In both, bce1 is not working (no
> > carrier). Changing the order to em0, em1, em2, bce1 did not improve
> > the situation.
> >
> > This is a WAN emulator and the rc.conf definitions are generated by
> > the configuration program from site names, hence the odd order. I am
> > changing the order manually.
> >
> > I have reconfigured the Cisco router to use G0/0 instead of G0/1. This
> > has made no noticeable difference.
>
> Did you configure it to 100MBit, full-duplex?
>
> -------
>
> Yes, I configured everything (all eight interfaces: four on FreeBSD and one on each of the routers) to be 100baseTX and full-duplex.
>
> As a test, I left out the 'media' and 'mediaopt' portions of the interface definitions in rc.conf and all the interfaces came up but three were in half-duplex mode, I think em0 was full-duplex (I am not in the lab to check).
>
> I have heard horror stories about Ethernet auto sensing, so I make sure the Cisco routers and FreeBSD specify 100 Mbps and full-duplex. I begin to wonder if even that is safe. :-(
>
> [ why does Outlook not quote properly ]
It could be that there is a race condition when the media opts are
set at the same time the interface is brought up... I have you tried
to down the interface, set the media opts, wait a second or two, and
then bring up the interface?
I would go back to autoneg... Even though the 2821 is old, the autoneg
issue was mostly when both sides were playing bad, and your nics are
modern...
At a previous company, we had an issue w/ our box at a client site...
The box would work fine for months, and then for some reason it would
suffer packet loss... After a long bit of puzzling, (I don't even
remeber how we figured it out, but I think we got a dump of the switch
configuration), it turned out that someone had configured our box to
be 100mbit/full duplex, but not the switch... It turned out that this
was "ok" for fxp as it would autoneg when brought up, and then it would
switch to 100mbit/full... Both sides would be happy till the switch
rebooted and then tried to autoneg again, but now the fxp had disabled
it, and the switch would fall back to 100mbit/half...
I fixed fxp to down/up the phy when setting fixed modes so that the
switch would loose it's autoneg and then the customer would immediately
see the issue instead of weeks/months/years later when the switch was
power cycled...
after the above, I considered anything not autoneg to be bad...
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
More information about the freebsd-net
mailing list