pf not seeing inbound packets on netgraph interface

[Edward Carrel]

On Jan 24, 2012, at 2:57 AM, Andreas Longwitz wrote:

> Hi Ed,
> 
>> I am running into a roadblock getting PF to filter traffic on
>> a Netgraph interface representing an L2TP/IPSec connection.
> 
>> The problem I have is that PF only sees traffic on the outbound
>> side of the netgraph interface.
> 
> This happens because the L2TP packets are tagged with an IPSEC-flag for
> later use by ipfw, and this flag is passed to the packets coming from
> ng0. Thats done by the netgraph under control of mpd, or better: mpd
> does nothing to clear this flag.

I was exploring the netgraph sources after sending my original question, and was developing a suspicion that this is what was going on.

From my digging, it didn't look like that mpd would be involved enough in the communication channel to be able to clear the flag, even if it wanted to. It would have to be one of the netgraph nodes to do it as the packet passed by, since mpd only gets informed if a control packet hits the l2tp note.  Or perhaps I am misunderstanding what you mean, or I am misreading how the data flows around.

> 
> With net.inet.ipsec.filtertunnel=1 you can ignore this IPSEC-flag but
> only global for all interfaces in the system. Thats probably not what
> you want, especially not for the real hardware interface the
> IPSEC-tunnel is going through.

It isn't my desired alternative, having already explored that avenue a while back. I could imagine that I would end up with a severely complicated pf ruleset even if I could get things to work.

> 
> I think L2TP under control of mpd should work independent of the
> existence of an IPSEC-tunnel and therefore clear this flag:
> 
> --- ng_l2tp.c.orig       2010-04-15 14:40:02.000000000 +0200
> +++ ng_l2tp.c   2012-01-23 17:09:41.000000000 +0100
> @@ -752,6 +752,7 @@
>        hookpriv_p hpriv = NULL;
>        hook_p hook = NULL;
>        struct mbuf *m;
> +       struct m_tag *mtag;
>        u_int16_t tid, sid;
>        u_int16_t hdr;
>        u_int16_t ns, nr;
> @@ -996,6 +997,11 @@
>                ERROUT(0);
>        }
> 
> +       /* Delete an existing ipsec tag */
> +       mtag = m_tag_find(m, PACKET_TAG_IPSEC_IN_DONE, NULL);
> +       if (mtag != NULL)
> +               m_tag_delete(m, mtag);
> +
>        /* Deliver data */
>        NG_FWD_NEW_DATA(error, item, hook, m);
> 
> This patch for the l2tp netgraph node does the job and you can use pf on
> the ng0 interface without any restrictions.

Very cool. Thanks. I will give this patch a spin, and report back with my results.

A thought that comes to mind; are there other netgraph modules that would be affected similarly by a packet that comes in with this flag already set? 

From what I gathered reading the sources, the same mbuf gets used for both the L2TP data packet, and the packet encapsulated within -- it simply trims the header from the mbuf. As such, the mbuf_tags from the L2TP packet propagate across. This seems like this would not be the most obvious behavior, at least in some circumstances. I can see perhaps wanting to propagate things like PF tags from the L2TP packet to the encapsulated packet across from one interface to another, but I can also see not wanting to have that happen.

If this same sort of thing happens elsewhere, I wonder if this sort of manipulation wants to happen the moment the packet enters the netgraph system from a ksocket. Perhaps this will break netgraph behavior elsewhere that depends on this? I'm also wondering if making the encapsulated packet even less derivative of the L2TP packet would make unexpected behavior like what I was observing less likely. Again, I'm not familiar enough with all the netgraph modules to know what might depend on this behavior, or if there are historical or functional reasons to keep the behavior as it is.

Thanks,

Ed Carrel