kernel panics in in_lltable_lookup (with INVARIANTS)

[Brian Somers]

On Fri, 21 Aug 2009 17:13:45 -0700 Kip Macy <kmacy at freebsd.org> wrote:
> Try this:
> 
> Index: sys/net/flowtable.c
> ===================================================================
> --- sys/net/flowtable.c (revision 196382)
> +++ sys/net/flowtable.c (working copy)
> @@ -688,6 +688,12 @@
>                 struct rtentry *rt = ro->ro_rt;
>                 struct ifnet *ifp = rt->rt_ifp;
> 
> +               if (ifp->if_flags & IFF_POINTOPOINT) {
> +                       RTFREE(rt);
> +                       ro->ro_rt = NULL;
> +                       return (ENOENT);
> +               }
> +
>                 if (rt->rt_flags & RTF_GATEWAY)
>                         l3addr = rt->rt_gateway;
>                 else
> 
> You'll need to apply this by hand as gmail munges the formatting.
> 
> -Kip

Hi,

That certainly stops the panic, however data routed to the tun
interface doesn't come out the back end and data written
to the back end doesn't come out the tun interface.

After connecting a 7-stable and a -current box, I get this sending
a ping packet from the stable box:

-stable:
TCP/IP: OUT ICMP: 172.16.1.22:8 ---> 172.16.1.20 (36/84)
Physical: write
Physical:  7e 3d c0 1c fd 00 12 02 1b 3a c1 1c 64 e8 04 75  ~=.......:..d..u
Physical:  84 a1 93 62 d5 e2 c1 86 1e 63 60 9a f4 82 54 43  ...b.....c`...TC
Physical:  01 00 70 7c 7e                                   ..p|~

meaning that ppp sees the ICMP when it reads the back of the
tun interface, and writes the encapsulated data to its physical
link (the "other end").

-current:
Physical: read
Physical:  7e 3d c0 1c fd 00 12 02 1b 3a c1 1c 64 e8 04 75  ~=.......:..d..u
Physical:  84 a1 93 62 d5 e2 c1 86 1e 63 60 9a f4 82 54 43  ...b.....c`...TC
Physical:  01 00 70 7c 7e                                   ..p|~
TCP/IP: IN ICMP: 172.16.1.22:8 ---> 172.16.1.20 (36/84)

meaning that ppp reads data from the physical link, decapsulates
it and writes an ICMP to the back of the tun interface.

Then nothing (no ICMP reply).  Sending an icmp from the -current
box doesn't wake ppp at all.

On the -stable box:
brian at gw ~ $ ifconfig tun0
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1540
        inet6 fe80::240:f4ff:feb1:1c85%tun0 prefixlen 64 scopeid 0x6 
        inet 172.16.1.22 --> 172.16.1.20 netmask 0xffffffff 
        Opened by PID 89468
brian at gw ~ $ route get 172.16.1.20
   route to: 172.16.1.20
destination: 172.16.1.20
    gateway: 172.16.1.22
  interface: tun0
      flags: <UP,GATEWAY,HOST,DONE>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0        1540         0 

and on the -current box:
brian at dev ~ $ ifconfig tun0
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1540
        inet 172.16.1.20 --> 172.16.1.22 netmask 0xffffffff 
        inet6 fe80::240:f4ff:feb1:10da%tun0 prefixlen 64 scopeid 0x7 
        Opened by PID 1647
brian at dev ~ $ route get 172.16.1.22
   route to: 172.16.1.22
destination: 172.16.1.22
    gateway: 
  interface: tun0
      flags: <UP,GATEWAY,HOST,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1540         1         0 

According to ``netstat -I tun0'', Opkts is increasing when I send from
the -current box, although netstat -s doesn't seem to see any more
icmps.  Maybe this problem isn't a routing problem.  I'll
look into it further and figure out if the packet is getting to the tun
driver and if so, what it thinks it's doing with it.

Thanks.

> On Fri, Aug 21, 2009 at 16:43, Brian Somers<brian at freebsd.org> wrote:
> > Hi,
> >
> > I've been working on a fix to address an issue that came up with
> > our update of openssh-5.  The issue is that openssh-5 now uses
> > pipe() to create stdin/stdout channels between sshd and the server
> > side program where it used to use socketpair().  Because it uses
> > pipe(), stdin is no longer bi-directional and cannot be used for both
> > input and output by a child process.  This breaks the use of ssh
> > as a tunnel with ppp on either end (set device "!ssh -e none host
> > ppp -direct label")
> >
> > I talked with des@ for a while and then with the openssh folks and
> > have not been able to resolve the issues in openssh that made them
> > choose to enforce the use of pipe() over socketpair().  I now have a
> > patch to ppp that makes ppp detect that it's connected via pipe() and
> > causes it to use stdin for input and stdout for output (usually it expects
> > just one descriptor).  Although I'm happy with the patch and planned on
> > requesting permission to commit, I've bumped into a show-stopper
> > that seems unrelated, so I thought I'd ask here if anyone has seen
> > this or has any suggestions as to what the problem might be.
> >
> > The issue....
> >
> > I'm seeing a panic when I send traffic through a ppp link:
> >
> > panic string is: sin_family 18
> > Stack trace starts:
> >    in_lltable_lookup()
> >    llentry_update()
> >    flowtable_lookup()
> >    ip_output()
> >    ....
> >
> > The panic is due to a KASSERT in in_lltable_lookup() that expects the
> > sockaddr to be AF_INET.  Number 18 is AF_LINK.
> >
> > AFAICT this is happening while setting up a temporary route for the
> > first outbound packet.  I haven't been able to do much investigation
> > yet due to other patches in my tree that seem to have broken all my
> > kernel symbols, but once I get a clean rebuild I should be back in
> > business.
> >
> > If anyone has any suggestions, I'm all ears!
> >
> > Cheers.

-- 
Brian Somers                                          <brian at Awfulhak.org>
Don't _EVER_ lose your sense of humour !               <brian at FreeBSD.org>