layer2 ipfw 'fwd' support
Eduardo Meyer
dudu.meyer at gmail.com
Thu Oct 7 19:23:59 UTC 2010
On Thu, Oct 7, 2010 at 12:19 AM, Julian Elischer <julian at freebsd.org> wrote:
> On 10/6/10 12:06 PM, Eduardo Meyer wrote:
>>
>> On Tue, Oct 5, 2010 at 5:31 PM, Julian Elischer<julian at freebsd.org>
>> wrote:
>>>
>>> On 10/5/10 12:56 PM, Eduardo Meyer wrote:
>>>>
>>>> On Mon, Oct 4, 2010 at 6:23 PM, Julian Elischer<julian at freebsd.org>
>>>> wrote:
>>>>>
>>>>> On 10/4/10 12:18 PM, Eduardo Meyer wrote:
>>>>>>
>>>>>> On Mon, Oct 4, 2010 at 3:35 PM, Julian Elischer<julian at freebsd.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> On 10/4/10 10:16 AM, Eduardo Meyer wrote:
>>>>>>>>
>>>>>>>> On Mon, Oct 4, 2010 at 2:02 PM, Brandon Gooch
>>>>>>>> <jamesbrandongooch at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Oct 4, 2010 at 9:44 AM, Eduardo Meyer<dudu.meyer at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> In the past I have used this patch by Luigi Rizzo, which helped me
>>>>>>>>>> well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> http://lists.freebsd.org/pipermail/freebsd-ipfw/2003-September/000526.html
>>>>>>>>>>
>>>>>>>>>> I tried with a friend to port it to -STABLE, but we were not able
>>>>>>>>>> to
>>>>>>>>>> find out what has replaced mt_tag. Also on ip_input.c we dirty
>>>>>>>>>> hacked
>>>>>>>>>> to following piece of code:
>>>>>>>>>>
>>>>>>>>>> #ifdef IPFIREWALL_FORWARD
>>>>>>>>>> if (m->m_flags& M_FASTFWD_OURS) {
>>>>>>>>>> m->m_flags&= ~M_FASTFWD_OURS;
>>>>>>>>>> goto pass; /* XXX was 'ours' - SHOULD WE MODIFY IT
>>>>>>>>>> HERE
>>>>>>>>>> */
>>>>>>>>>> }
>>>>>>>>>> if ((dchg = (m_tag_find(m, PACKET_TAG_IPFORWARD, NULL) !=
>>>>>>>>>> NULL))
>>>>>>>>>> != 0) {
>>>>>>>>>> /*
>>>>>>>>>> * Directly ship the packet on. This allows
>>>>>>>>>> forwarding
>>>>>>>>>> * packets originally destined to us to some other
>>>>>>>>>> directly
>>>>>>>>>> * connected host.
>>>>>>>>>> */
>>>>>>>>>> ip_forward(m, dchg);
>>>>>>>>>> return;
>>>>>>>>>> }
>>>>>>>>>> #endif /* IPFIREWALL_FORWARD */
>>>>>>>>>>
>>>>>>>>>> And this is something we are not sure if its correct.
>>>>>>>>>>
>>>>>>>>>> So my very obvious question is:
>>>>>>>>>>
>>>>>>>>>> Does anyone has a recent version of this patch to share?
>>>>>>>>>>
>>>>>>>>>> Can anyone familiar with ipfw source code help me with that?
>>>>>>>>>>
>>>>>>>>> I'm certainly not an expert, but I wonder if the patch your
>>>>>>>>> referring
>>>>>>>>> to is still required? Can you provide more detail about your
>>>>>>>>> particular application?
>>>>>>>>>
>>>>>>>>> -Brandon
>>>>>>>>
>>>>>>>> Yes, its still required since ipfw fwd ignores layer2 frames.
>>>>>>>>
>>>>>>>> The application is the very same: squid. I mean, Lusca in fact
>>>>>>>> (squid
>>>>>>>> fork).
>>>>>>>>
>>>>>>>> Thank you for your interest.
>>>>>>>
>>>>>>> Cisco/Ironport have a patch that does this..
>>>>>>> I had permission to bring it back when I worked there but never got
>>>>>>> it
>>>>>>> committed.
>>>>>>>
>>>>>>> Adrian, was it part of the set I gave you?
>>>>>>
>>>>>> Hello Elischer,
>>>>>>
>>>>>> Was this made public?
>>>>>>
>>>>>> I hope Chadd has some good news. In fact I tent to use with Lusca in
>>>>>> tproxy mode. I bet this is the only missing piece of software.
>>>>>>
>>>>> I just dug up my old changes.
>>>>> do you want to fwd from a bridge? or what?
>>>>> (it makes a difference what patches are needed)
>>>>>
>>>>> If you want to fwd from a bridge to make a transparent layer 2 proxy,
>>>>> this
>>>>> may help..
>>>>>
>>>>>
>>>>> Here are parts of it that may be relevent:
>>>>> these are old (2007 I think) but may be of use still.
>>>>>
>>>>> adrian had the full set at
>>>>>
>>>>> ==quote adrian=====
>>>>> The stuff is in p4 now, but I haven't tested it out at all.
>>>>>
>>>>> //depo/projects/adrian_spoof_clientip/ I -think-.
>>>>> == end quote===
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Index: net/if_bridge.c
>>>>> ===================================================================
>>>>> RCS file: /usr/local/cvsroot/freebsd/src/sys/net/if_bridge.c,v
>>>>> retrieving revision 1.107
>>>>> diff -u -r1.107 if_bridge.c
>>>>> --- net/if_bridge.c 6 Nov 2007 23:01:42 -0000 1.107
>>>>> +++ net/if_bridge.c 28 Nov 2007 06:59:10 -0000
>>>>> @@ -2908,6 +2908,11 @@
>>>>> struct ip *ip;
>>>>> struct llc llc1;
>>>>> u_int16_t ether_type;
>>>>> + int is_ip = 0;
>>>>> +#ifdef IPFIREWALL_FORWARD
>>>>> + struct m_tag *fwd_tag;
>>>>> +#endif
>>>>> +
>>>>>
>>>>> snap = 0;
>>>>> error = -1; /* Default error if not error == 0 */
>>>>> @@ -2967,6 +2972,7 @@
>>>>> #ifdef INET6
>>>>> case ETHERTYPE_IPV6:
>>>>> #endif /* INET6 */
>>>>> + is_ip = 1;
>>>>> break;
>>>>> default:
>>>>> /*
>>>>> @@ -3024,6 +3030,30 @@
>>>>>
>>>>> if (*mp == NULL)
>>>>> return (error);
>>>>> +
>>>>> +#ifdef IPFIREWALL_FORWARD
>>>>> + /*
>>>>> + * Did the firewall want to forward it somewhere?
>>>>> + * If so, let the ip stack handle it.
>>>>> + */
>>>>> + if (i == 0&& args.next_hop != NULL&&
>>>>> + is_ip /*&& src != NULL */) {
>>>>> +
>>>>> + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD,
>>>>> + sizeof(struct sockaddr_in),
>>>>> M_NOWAIT);
>>>>> + if (fwd_tag == NULL)
>>>>> + goto drop;
>>>>> + bcopy(args.next_hop, (fwd_tag+1),
>>>>> + sizeof(struct sockaddr_in));
>>>>> + m_tag_prepend(*mp, fwd_tag);
>>>>> +
>>>>> + if (in_localip(args.next_hop->sin_addr))
>>>>> + (*mp)->m_flags |= M_FASTFWD_OURS;
>>>>> + ether_demux(src, *mp);
>>>>> + return (NULL);
>>>>> + }
>>>>> +#endif
>>>>> +
>>>>>
>>>>> if (DUMMYNET_LOADED&& (i == IP_FW_DUMMYNET)) {
>>>>>
>>>>> ==================
>>>>> Index: netinet/ip_fw2.c
>>>>> ===================================================================
>>>>> RCS file: /usr/local/cvsroot/freebsd/src/sys/netinet/ip_fw2.c,v
>>>>> retrieving revision 1.178
>>>>> diff -u -r1.178 ip_fw2.c
>>>>> --- netinet/ip_fw2.c 28 Oct 2007 17:12:47 -0000 1.178
>>>>> +++ netinet/ip_fw2.c 28 Nov 2007 06:59:10 -0000
>>>>>
>>>>> @@ -3446,8 +3507,10 @@
>>>>> case O_FORWARD_IP: {
>>>>> struct sockaddr_in *sa;
>>>>> sa =&(((ipfw_insn_sa *)cmd)->sa);
>>>>> +#if 0
>>>>> if (args->eh) /* not valid on layer2
>>>>> pkts
>>>>> */
>>>>> break;
>>>>> +#endif
>>>>> if (!q || dyn_dir == MATCH_FORWARD) {
>>>>> if (sa->sin_addr.s_addr ==
>>>>> INADDR_ANY) {
>>>>>
>>>>> bcopy(sa,&args->hopstore,
>>>>>
>>>>> =============================================
>>>>> Index: netinet/ip_output.c
>>>>
>>>> Dear Julian,
>>>>
>>>> Is anything missing from the above code? Say, like ip_output stuff?
>>>>
>>>> I have tried what you sent me, compiled fine but did not work.
>>>>
>>>> Here is my only rule (I have tried both with and without layer2 on the
>>>> rule):
>>>>
>>>> 00001 36 4338 fwd 127.0.0.1,80 tcp from any to not me
>>>> dst-port 80 layer2
>>>> 65535 32842101 2107060460 allow ip from any to any
>>>>
>>>> Here are the sysctl tunables:
>>>>
>>>> net.link.bridge.ipfw: 1
>>>> net.link.bridge.inherit_mac: 0
>>>> net.link.bridge.log_stp: 0
>>>> net.link.bridge.pfil_local_phys: 0
>>>> net.link.bridge.pfil_member: 0
>>>> net.link.bridge.pfil_bridge: 1
>>>> net.link.bridge.ipfw_arp: 0
>>>> net.link.bridge.pfil_onlyip: 0
>>>> net.link.ether.inet.log_arp_permanent_modify: 1
>>>> net.link.ether.inet.log_arp_movements: 1
>>>> net.link.ether.inet.log_arp_wrong_iface: 1
>>>> net.link.ether.inet.proxyall: 0
>>>> net.link.ether.inet.useloopback: 1
>>>> net.link.ether.inet.maxtries: 5
>>>> net.link.ether.inet.max_age: 1200
>>>> net.link.ether.ipfw: 1
>>>>
>>>> And my bridge:
>>>>
>>>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
>>>> mtu
>>>> 1500
>>>> ether 16:52:8e:91:2f:45
>>>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>>>> maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
>>>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>>>> member: vr0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>>> ifmaxaddr 0 port 5 priority 128 path cost 200000
>>>> member: sis0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>>> ifmaxaddr 0 port 1 priority 128 path cost 200000
>>>>
>>>> The ipfw counter gets increased by nothing hits by Apache. Instead I
>>>> go to Internet directly.
>>>>
>>>> sis0 is on internet, vr0 is cross-over to the laptop (customer).
>>>>
>>>> How should I debug it?
>>>>
>>>>
>>> basically I woud suggest code inspection for a start..
>>>
>>> look at where ipfw is called (just before where the patch went in) and
>>> follow the packet up into ipfw
>>> and back, and read what it would do..
>>>
>>> It's actually not a very hard path to follow.
>>>
>>> I'll try look at it after work..
>>
>> Hello Julian / Adrian.
>>
>> Thank you for your attention. A friend added some log entries so we
>> could try to find out what gets run and what doesnt.
>>
>> Here is my current patch against RELENG_8:
>>
>> --- if_bridge.c.orig 2010-09-11 22:02:36.000000000 +0000
>> +++ if_bridge.c 2010-10-05 17:59:13.000000000 +0000
>> @@ -2957,6 +2957,13 @@
>> struct ip *ip;
>> struct llc llc1;
>> u_int16_t ether_type;
>> + int is_ip = 0;
>> +#ifdef IPFIREWALL_FORWARD
>> + struct m_tag *fwd_tag;
>> +#endif
>> +
>> +
>> +
>>
>> snap = 0;
>> error = -1; /* Default error if not error == 0 */
>> @@ -3016,6 +3023,8 @@
>> #ifdef INET6
>> case ETHERTYPE_IPV6:
>> #endif /* INET6 */
>> + is_ip=1;
>> + log(LOG_NOTICE, "Entered 0: is_ip=%i\n",is_ip);
>> break;
>> default:
>> /*
>> @@ -3091,6 +3100,32 @@
>> if (*mp == NULL)
>> return (error);
>>
>> +#ifdef IPFIREWALL_FORWARD
>> + /*
>> + * Did the firewall want to forward it somewhere?
>> + * If so, let the ip stack handle it.
>> + */
>> + log(LOG_NOTICE, "Entered 1");
>> + if (i == 0&& args.next_hop != NULL&&
>> + is_ip /*&& src != NULL */) {
>> + log(LOG_NOTICE, "Entered 2");
>> +
>> + fwd_tag = m_tag_get(PACKET_TAG_IPFORWARD,
>> + sizeof(struct sockaddr_in),
>> M_NOWAIT);
>> + if (fwd_tag == NULL)
>> + goto drop;
>> + bcopy(args.next_hop, (fwd_tag+1),
>> + sizeof(struct sockaddr_in));
>> + m_tag_prepend(*mp, fwd_tag);
>> +
>> + if (in_localip(args.next_hop->sin_addr))
>> + (*mp)->m_flags |= M_FASTFWD_OURS;
>> + ether_demux(src, *mp);
>> + return (NULL);
>> + }
>> +#endif
>> +
>> +
>> if (ip_dn_io_ptr&& (i == IP_FW_DUMMYNET)) {
>>
>> /* put the Ethernet header back on */
>> --- ../netinet/ipfw/ip_fw2.c.orig 2010-09-16 15:11:17.000000000
>> +0000
>> +++ ../netinet/ipfw/ip_fw2.c 2010-10-06 12:17:12.000000000 +0000
>> @@ -2059,8 +2059,14 @@
>> break;
>>
>> case O_FORWARD_IP:
>> - if (args->eh) /* not valid on layer2
>> pkts */
>> +#if 0
>> + /* not valid on layer2 pkts */
>> + if (args->eh) {
>> + log(LOG_NOTICE, "ip_fw2.c Entered
>> 1");
>> break;
>> + }
>> + log(LOG_NOTICE, "ip_fw2.c Entered 2");
>
> these will never happen as they are in the #if 0 section.
>
> the #if 0 is to REMOVE that code from being compiled.
>
>
>> +#endif
Hello Julian,
Thank you again for your feedback. I appreciate it very much.
On my understanding this "if 0" was to really ignore this portion of
code, because as I understand what is does is to break (leave the
loop) if the packet is on layer2, and this is something we would not
want, but I guess I am wrong.
I tested now with your suggestion, and what we get is:
Oct 7 15:45:16 phoenix kernel: Entered 0: is_ip=1
Oct 7 15:45:16 phoenix kernel: ip_fw2.c Entered 1
Oct 7 15:45:16 phoenix kernel: Entered 0: is_ip=1
Oct 7 15:45:50 phoenix last message repeated 29 times
Oct 7 15:47:53 phoenix last message repeated 237 times
Oct 7 15:57:56 phoenix last message repeated 1029 times
Oct 7 16:02:51 phoenix last message repeated 655 times
Oct 7 16:02:51 phoenix kernel: ip_fw2.c Entered 1
Oct 7 16:02:51 phoenix kernel: Entered 0: is_ip=1
Oct 7 16:03:23 phoenix last message repeated 54 times
Oct 7 16:05:24 phoenix last message repeated 345 times
Oct 7 16:15:26 phoenix last message repeated 1135 times
Oct 7 16:15:33 phoenix last message repeated 8 times
So yes, we entered on ipfw code now, and executed only the instruction
before we "break".
The curious thing is that the counter did not count now with both:
00001 0 0 fwd 127.0.0.1,80 tcp from any to not me dst-port 80 layer2
00001 0 0 fwd 127.0.0.1,80 tcp from any to not me dst-port 80
How can I move forth?
--
===========
Eduardo Meyer
pessoal: dudu.meyer at gmail.com
profissional: ddm.farmaciap at saude.gov.br
More information about the freebsd-ipfw
mailing list