Deleting IPv4 iface-routes from extra FIBs

[Alan Somers]

On Thu, Apr 24, 2014 at 5:50 PM, Chris Smith <chris at nevermind.co.nz> wrote:
> On 25/04/14 11:15, Alan Somers wrote:
>>
>> On Thu, Apr 24, 2014 at 5:00 PM, Chris Smith <chris at nevermind.co.nz>
>> wrote:
>>>
>>> On 24/04/14 18:24, Alexander V. Chernikov wrote:
>>>>
>>>> On 24.04.2014 01:56, Chris Smith wrote:
>>>>>
>>>>> On 23/04/14 19:55, Julian Elischer wrote:
>>>>>>
>>>>>> On 4/23/14, 4:38 AM, Nikolay Denev wrote:
>>>>>>>
>>>>>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer
>>>>>>> <h.schmalzbauer at omnilan.de> wrote:
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895
>>>>>>>> interface route protection was added (so the following problem arose
>>>>>>>> with 9.2).
>>>>>>>>
>>>>>>>> Unfortunately, in my case, I must be able to delete these routes;
>>>>>>>> not in
>>>>>>>> the default FIB, but in jail's fibs, because:
>>>>>>>> · Host is multihomed with multiple nics in different subnets.
>>>>>>>> · Jail's IP (no vnet) is from a different subnet than host's
>>>>>>>> default-router subnet – jail has no ip in the range of host's
>>>>>>>> default-router!!!
>>>>>>>> · FIB used by jail contains valid default-router.
>>>>>>>>
>>>>>>>> Problem:
>>>>>>>> If iface-routes exist in jail's FIB, answer-packets take the
>>>>>>>> iface-shortcut, not trespassing the router (default gateway); hence
>>>>>>>> 3way-handshake never finishes and firewall terminates (half-opened)
>>>>>>>> TCP
>>>>>>>> sessions.
>>>>>>>>
>>>>>>>> Workarround:
>>>>>>>> · Abuse packet filter doing some kind of route-to…
>>>>>>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes
>>>>>>>> can
>>>>>>>> be deleted without any hack)
>>>>>>>>
>>>>>>>> Desired solution:
>>>>>>>> · Allow deletion of v4-iface-routes if FIB!=0.
>>>>>>>>
>>>>>>>> Unfortunately my C skills don't allow me to implement this myself
>>>>>>>> :-(
>>>>>>>> I can't even follow the code, I guess that was originally
>>>>>>>> considered,
>>>>>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy
>>>>>>>> way
>>>>>>>> and simply reverted r248895 instead of trying to understand
>>>>>>>> rtrequest1_fib(). I wish I had the time to learn…
>>>>>>>>
>>>>>>>> Thanks for any help,
>>>>>>>>
>>>>>>>> -Harry
>>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> As it was suggested before as immediate workaround you can set
>>>>>>> net.add_addr_allfibs=0 so that the interface routes are added only in
>>>>>>> the default FIB.
>>>>>>
>>>>>> yes, we made two behaviours.
>>>>>> Add interface routes to all active FIBS or only add them to the first
>>>>>> fib and let the user populate other fibs as needed.
>>>>>> It appears you want the second behaviour, so I suggest you use that
>>>>>> option and set up all your routes manually.
>>>>>>
>>>>> Ah, this explains a thing or two.
>>>>
>>>> There is an ongoing work to
>>>> 1) make fibs/allfibs=0 to work better
>>>> 2) Move forward to make allfibs=0 as default value.
>>>>>
>>>>> So when allfibs=0 and an interface is bought up, it's added to the
>>>>> first
>>>>> FIB automatically (and cannot be removed).
>>>>>
>>>>> Is there a way to change which fib the interface route is bought up on?
>>>>> I tried to 'setfib x ifconfig ....' which didn't work.
>>>>
>>>> This will be fixed in near future.
>>>>>
>>>>> Failing that, is there a way to change the systems global FIB without
>>>>> having to run every service with setfib? Basically, the behavour I want
>>>>> is for interface routes to be bought up on NO fibs, and manually add
>>>>> them to the fibs I need it on.
>>>>
>>>> If ifconfig_ifaceX="fib X inet 1.2.3.4/30" works as expected (changes
>>>> interface fib to chosen one and announce interface route and host route
>>>> in this particular fib) - does this sound OK to you?
>>>
>>> Yes this sounds good.
>>>
>>> If I'm not mistaken the interface FIB only makes sense when the system is
>>> routing? Because the issue I have is that SYN ACKs from services are
>>> being
>>> routed via the wrong interfaces and interface FIBs do not appear to
>>> affect
>>> that.
>>
>> The interface FIB is used when forwarding packets and when creating
>> the initial subnet and host routes when you assign an interface
>> address.  It's not used for outbound traffic (except in that it
>> determines where the host and subnet routes get created).  There are
>> several other FIB bugs that I'm actively working on.  kern/187553
>> might be related to your problem; it would be great if you could make
>> a test case.
>
> The connections I've  been testing with are TCP (SSH and Netcat)
>
> However, this:
>
> ifconfig bge0 fib 1 10.0.0.1/24
>
> Adds the interface route to FIB 0 and nothing to FIB 1. FreeBSD 10 RELEASE
> amd64

That is exactly the bug I fixed earlier today with r264887.  I'll MFC
it to stable/10 in a few weeks.

>
>
>>> Allowing interface routes on different FIBs will fix that I think. Or
>>> being
>>> able to remove interface routes from a FIB.
>>>
>>> In the mean time, I will probably use FIBs (as opposed to vnet) for my
>>> jails, but find a way to run the hosts SSHd with a specific FIB. Any easy
>>> way to do that? Or to specify a system "default FIB" other than 0?
>>
>> In FreeBSD 10 you can put "sshd_fib=1" in /etc/rc.conf to change that
>> process's fib.  That will affect the routing of sshd's outbound
>> packets.  If you also want to limit which interfaces sshd listens on,
>> you can do that with pf or by setting the ListenAddress in
>> sshd_config.
>>
>> -Alan
>>
>>>>>>> --Nikolay
>>>>>>> _______________________________________________
>>>>>>> freebsd-net at freebsd.org mailing list
>>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>>> To unsubscribe, send any mail to
>>>>>>> "freebsd-net-unsubscribe at freebsd.org"
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> freebsd-net at freebsd.org mailing list
>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>
>>>>> _______________________________________________
>>>>> freebsd-net at freebsd.org mailing list
>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>
>>>> _______________________________________________
>>>> freebsd-net at freebsd.org mailing list
>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>
>>>
>>> _______________________________________________
>>> freebsd-net at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"