netgraph mkpeer and connect failures with ng_ipfw and ng_nat

[Julian Elischer]

Erik Klavon wrote:
> On Thu, Jan 14, 2010 at 04:22:31PM -0800, Julian Elischer wrote:
>> Erik Klavon wrote:
>>> Here are the hooks for one ng_nat(4) node of interest. At the time I
>>> obtained this information, this node was working correctly. Everything
>>> in this output looks correct.
>>>
>>> sudo ngctl show ipfw:202182138
>>>  Name: WirelessNAT2182138 Type: nat             ID: 000062ee   Num hooks: 
>>>  2
>>>  Local hook      Peer name       Peer type    Peer ID         Peer hook   
>>>  ----------      ---------       ---------    -------         ---------   
>>>  in              ipfw            ipfw         00005a83        102182138   
>>>  out             ipfw            ipfw         00005a83        202182138   
>>>
>>> sudo ngctl msg ipfw:102182138 listredirects
>>> Rec'd response "listredirects" (10) from "[62ee]:":
>>> Args:   { total_count=1 redirects=[ { id=1 local_addr=10.10.118.43 
>>> alias_addr=136.152.182.138 proto=259 description="Static NAT" } ] }
>>>
>>> Running show on ipfw:102174202, I see that this hook is pointing to
>>> the above ng_nat(4) node WirelessNAT2182138.
>> can you show that output?
> 
> The output for ngctl show ipfw:102174202 is below.

how about the output for ipfw: ?


> 
>>> sudo ngctl show ipfw:102174202
>>>  Name: WirelessNAT2182138 Type: nat             ID: 000062ee   Num hooks: 
>>>  2
>>>  Local hook      Peer name       Peer type    Peer ID         Peer hook   
>>>  ----------      ---------       ---------    -------         ---------   
>>>  in              ipfw            ipfw         00005a83        102182138   
>>>  out             ipfw            ipfw         00005a83        202182138   
>>>
>>> But WirelessNAT2182138 has no record of a hook102174202. Somehow, two
>>> hooks used to refer to what should be two different NAT sessions are
>>> pointing to the same ng_nat(4) object (i.e. one session). If I run
>>> shutdown on ipfw:102174202, WirelessNAT2182138 goes away. Given the
>>> above calls to ngctl(8), I don't know what is causing these two separate
>>> hooks to refer to the same ng_nat(4) object.
>> you might name the ipfw nodes to make the output clearer.
> 
> ng_ipfw(4) uses a single node to pass traffic from ipfw(4) to
> netgraph(4). You associate as many hooks with this node as you like and
> then direct traffic via the single ng_ipfw(4) node to different hooks
> using ipfw(8) rules such as
> 
> 01230 netgraph tablearg ip from table(87) to any in
> 
> Suppose a packet comes in for IP address 10.10.113.226 and rule 1230
> is reached during the processing of the ipfw ruleset. According to the
> above rule, ipfw(4) will look up 10.10.113.226/32 in table 87 and if
> it finds a matching key in the table, it will tag the packet with the
> matching table entry's value and send the packet to the ng_ipfw(4)
> node. ng_ipfw(4) will then examine the tag and look for an associated
> hook with a matching name. If such a hook is found the packet is sent
> on to the node associated with the hook. In the case of 10.10.113.226,
> table 87 contains the following entry.
> 
> 10.10.113.226/32 202173114
> 
> Hook 202173114 refers to the following ng_nat node.
> 
> sudo ngctl show ipfw:202173114
>   Name: WirelessNAT2173114 Type: nat             ID: 000015f0   Num hooks: 2
>   Local hook      Peer name       Peer type    Peer ID         Peer hook      
>   ----------      ---------       ---------    -------         ---------      
>   in              ipfw            ipfw         00000001        102173114      
>   out             ipfw            ipfw         00000001        202173114      
> 
> which is configured for one to one NAT as follows.
>  
> sudo ngctl msg WirelessNAT2173114: listredirects
> Rec'd response "listredirects" (10) from "[15f0]:":
> Args:   { total_count=1 redirects=[ { id=1 local_addr=10.10.113.226 alias_addr=x.x.173.114 proto=259 description="Static NAT" } ] }
> 
> There is a similar path for traffic coming in to the globally routable
> side of these one to one NAT nodes.
> 
>> I have not looked at the ipfw node type much but it is possible
>> that is suffers from races.
>> Especially in the face of ipfw rule changes.
> 
> Can you think of an example in any other netgraph module of proper
> concurrency handling I can use as a reference when looking at
> ng_ipfw(4)?
> 
> By using tables we don't need to update the ruleset itself. The
> ruleset is (normally) loaded once after the machine boots. The table
> updates take place after all the calls to ngctl(8) and after the call
> to arp(8). I will take a look at the interface between ipfw(4) and
> ng_ipfw(4).
> 
>> have you tried adding small delays (sleep 0.5) betwenn the calls by 
>> the way?
> 
> I've now added sleep 0.5 between the calls to ngctl. I'll let you know
> if that helps at all; thanks for the suggestion.
> 
> Erik