Deleting IPv4 iface-routes from extra FIBs

Thu Apr 24 23:50:15 UTC 2014

On 25/04/14 11:15, Alan Somers wrote:
> On Thu, Apr 24, 2014 at 5:00 PM, Chris Smith <chris at nevermind.co.nz> wrote:
>> On 24/04/14 18:24, Alexander V. Chernikov wrote:
>>> On 24.04.2014 01:56, Chris Smith wrote:
>>>> On 23/04/14 19:55, Julian Elischer wrote:
>>>>> On 4/23/14, 4:38 AM, Nikolay Denev wrote:
>>>>>> On Tue, Apr 22, 2014 at 5:37 PM, Harald Schmalzbauer
>>>>>> <h.schmalzbauer at omnilan.de> wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> here, http://svnweb.freebsd.org/base?view=revision&revision=248895
>>>>>>> interface route protection was added (so the following problem arose
>>>>>>> with 9.2).
>>>>>>>
>>>>>>> Unfortunately, in my case, I must be able to delete these routes;
>>>>>>> not in
>>>>>>> the default FIB, but in jail's fibs, because:
>>>>>>> · Host is multihomed with multiple nics in different subnets.
>>>>>>> · Jail's IP (no vnet) is from a different subnet than host's
>>>>>>> default-router subnet – jail has no ip in the range of host's
>>>>>>> default-router!!!
>>>>>>> · FIB used by jail contains valid default-router.
>>>>>>>
>>>>>>> Problem:
>>>>>>> If iface-routes exist in jail's FIB, answer-packets take the
>>>>>>> iface-shortcut, not trespassing the router (default gateway); hence
>>>>>>> 3way-handshake never finishes and firewall terminates (half-opened)
>>>>>>> TCP
>>>>>>> sessions.
>>>>>>>
>>>>>>> Workarround:
>>>>>>> · Abuse packet filter doing some kind of route-to…
>>>>>>> · Revert r248895, to be able to delete v4-iface-routes (inet6-routes
>>>>>>> can
>>>>>>> be deleted without any hack)
>>>>>>>
>>>>>>> Desired solution:
>>>>>>> · Allow deletion of v4-iface-routes if FIB!=0.
>>>>>>>
>>>>>>> Unfortunately my C skills don't allow me to implement this myself :-(
>>>>>>> I can't even follow the code, I guess that was originally considered,
>>>>>>> but possibly doesn't work bacause of a simple bug?!? I took the lazy
>>>>>>> way
>>>>>>> and simply reverted r248895 instead of trying to understand
>>>>>>> rtrequest1_fib(). I wish I had the time to learn…
>>>>>>>
>>>>>>> Thanks for any help,
>>>>>>>
>>>>>>> -Harry
>>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> As it was suggested before as immediate workaround you can set
>>>>>> net.add_addr_allfibs=0 so that the interface routes are added only in
>>>>>> the default FIB.
>>>>> yes, we made two behaviours.
>>>>> Add interface routes to all active FIBS or only add them to the first
>>>>> fib and let the user populate other fibs as needed.
>>>>> It appears you want the second behaviour, so I suggest you use that
>>>>> option and set up all your routes manually.
>>>>>
>>>> Ah, this explains a thing or two.
>>> There is an ongoing work to
>>> 1) make fibs/allfibs=0 to work better
>>> 2) Move forward to make allfibs=0 as default value.
>>>> So when allfibs=0 and an interface is bought up, it's added to the first
>>>> FIB automatically (and cannot be removed).
>>>>
>>>> Is there a way to change which fib the interface route is bought up on?
>>>> I tried to 'setfib x ifconfig ....' which didn't work.
>>> This will be fixed in near future.
>>>> Failing that, is there a way to change the systems global FIB without
>>>> having to run every service with setfib? Basically, the behavour I want
>>>> is for interface routes to be bought up on NO fibs, and manually add
>>>> them to the fibs I need it on.
>>> If ifconfig_ifaceX="fib X inet 1.2.3.4/30" works as expected (changes
>>> interface fib to chosen one and announce interface route and host route
>>> in this particular fib) - does this sound OK to you?
>> Yes this sounds good.
>>
>> If I'm not mistaken the interface FIB only makes sense when the system is
>> routing? Because the issue I have is that SYN ACKs from services are being
>> routed via the wrong interfaces and interface FIBs do not appear to affect
>> that.
> The interface FIB is used when forwarding packets and when creating
> the initial subnet and host routes when you assign an interface
> address.  It's not used for outbound traffic (except in that it
> determines where the host and subnet routes get created).  There are
> several other FIB bugs that I'm actively working on.  kern/187553
> might be related to your problem; it would be great if you could make
> a test case.
The connections I've  been testing with are TCP (SSH and Netcat)

However, this:

ifconfig bge0 fib 1 10.0.0.1/24

Adds the interface route to FIB 0 and nothing to FIB 1. FreeBSD 10 
RELEASE amd64

>> Allowing interface routes on different FIBs will fix that I think. Or being
>> able to remove interface routes from a FIB.
>>
>> In the mean time, I will probably use FIBs (as opposed to vnet) for my
>> jails, but find a way to run the hosts SSHd with a specific FIB. Any easy
>> way to do that? Or to specify a system "default FIB" other than 0?
> In FreeBSD 10 you can put "sshd_fib=1" in /etc/rc.conf to change that
> process's fib.  That will affect the routing of sshd's outbound
> packets.  If you also want to limit which interfaces sshd listens on,
> you can do that with pf or by setting the ListenAddress in
> sshd_config.
>
> -Alan
>
>>>>>> --Nikolay
>>>>>> _______________________________________________
>>>>>> freebsd-net at freebsd.org mailing list
>>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> freebsd-net at freebsd.org mailing list
>>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>> _______________________________________________
>>>> freebsd-net at freebsd.org mailing list
>>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>>>
>>> _______________________________________________
>>> freebsd-net at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"