[SOLVED] LAN unreachable after 5.3 install

[Wes Santee]

On Mon, 15 Nov 2004 11:02:11 -0800, Wes Santee <wsantee at gmail.com> wrote:
> On Mon, 15 Nov 2004 12:39:49 +0000, Josh Paetzel <josh at tcbug.org> wrote:
> > On Monday 15 November 2004 18:22, Wes Santee wrote:
> > > my problem is that after install, I can't
> > > see my LAN.  From the FreeBSD box I try this:
> > >
> > > ping -S 10.0.0.1 10.0.0.254
> > >
> > > but none of the pings are responded to.  From 10.0.0.254, I try
> > > pinging 10.0.0.1, but the result is the same.
> > >
> > > Okay, here is the WEIRD part:  When I run tcpdump to see what's
> > > going on, all of a sudden everything starts working!  It's as if
> > > going into promiscuous mode shuts off some kind of block that I
> > > can't figure out. When I kill tcpdump, it goes back to not working
> > > again.
> >
> > The output of ifconfig may be useful in tracking down your problem
> 
> Sure, here it is.  xl1 is the LAN interface, tun0->xl0 is the PPPoE interface:
> 
> [wes at lister:/etc] 7 $ ifconfig -a
> xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         options=8<VLAN_MTU>
>         ether 00:60:97:a7:c9:01
>         media: Ethernet autoselect (100baseTX <full-duplex>)
>         status: active
> xl1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>         options=9<RXCSUM,VLAN_MTU>
>         inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
>         ether 00:10:5a:9a:11:8c
>         media: Ethernet autoselect (100baseTX <full-duplex>)
>         status: active
> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
>         inet6 ::1 prefixlen 128
>         inet 127.0.0.1 netmask 0xff000000
> tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1492
>         inet 216.113.200.107 --> 216.113.192.225 netmask 0xffffffff
>         Opened by PID 1895
> 

Well, I've solved this, kind of.  I noticed that hosts on my LAN had
no problems pinging each other.  Only pinging 10.0.0.1 or originating
a ping from 10.0.0.1 would fail.

Then I swapped the interfaces (made xl0 internal, and xl1 the PPPoE
interface).  The problem swapped with it.  Now I could see my LAN
(ping from 10.0.0.1 to all LAN hosts, and vice versa), but I couldn't
connect to my ISP.  I din't really suspect the card as it did work if
I did the tcpdump trick.

So then I thought it might be some weird IRQ/PCI conflict and took out
an unused PCI card (SB Live), and moved the NICs so they wouldn't be
sharing IRQs.  This didn't solve the problem either.

You notice that PROMISC in the flags for xl1 above?  That's because I
took the ifconfig while tcpdump was running on that interface.  As a
desperation move, I just tacked "promisc" on to the end of the
interface configuration in rc.conf and rebooted.  Now everything works
fine.  I don't know why, but the interface must be in promiscious mode
to work.  It worked fine as is on my FreeBSD 4.9 machine, so I really
have no idea what's up.

A look at dmesg shows this:

xl0: <3Com 3c905-TX Fast Etherlink XL> port 0xa400-0xa43f irq 15 at device 10.0
on pci0
xl1: <3Com 3c905B-TX Fast Etherlink XL> port 0xa000-0xa07f mem 0xdd800000-0xdd80
007f irq 10 at device 11.0 on pci0

Any clues here as to what might be up?  Perhaps this is something to
do with the RXCSUM flag?  Maybe I should turn it off?  I don't really
want to continue blindly trying things without some idea of what I'm
trying to affect.  If anyone has any other intellegent ideas, I'm all
ears.  If not, I'll leave it as is.

Cheers,
-Wes