Default route changes unexpectedly

[Randall Stewart]

Ok

I too have been struck by this *multiple* times on my base home router.

I am not sure how its happening, but I have placed in my kernel a special
catch that if the default route is set via the normal route.c path and it
is *not* the default route to my ISP, I will crash the kernel.

My thoughts are this is *not* going to happen and its probably some other
memory corruption, but I will start with the most obvious first ;-)

Now, I have also put in a cron that checks every 10 min the default route, and 
if it finds it *not* correct, it will a) log it, and b) fix it.

So if I get a log showing up, I will know its *not* some errant program in 9.x 
causing a routing change via a routing socket, and if not I should have a crash
revealing who the guilty party is ;-)

I will keep you all informed as I try to gather more information ;-D

R
On Mar 7, 2013, at 12:07 PM, Nick Rogers wrote:

> On Wed, Mar 6, 2013 at 12:25 AM, Andre Oppermann <andre at freebsd.org> wrote:
>> On 05.03.2013 18:39, Nick Rogers wrote:
>>> 
>>> Hello,
>>> 
>>> I am attempting to create awareness of a serious issue affecting users
>>> of FreeBSD 9.x and PF. There appears to be a bug that allows the
>>> kernel's routing table to be corrupted by traffic routing through the
>>> system. Under heavy traffic load, the default route can seemingly
>>> randomly change to an IP address that is not directly connected to the
>>> network (i.e., is not configured anywhere). Dhclient is not in the
>>> mix, nor is routed, bgpd, etc. Running `route monitor` shows no
>>> evidence of the change in the default route. The one commonality
>>> between all the systems experiencing this problem seems to be the use
>>> of PF.
>>> 
>>> Obviously this is a serious problem as it causes all Internet-bound
>>> traffic to stop routing until the default route is corrected. Some
>>> users, including myself, are working around this problem by installing
>>> a script that runs multiple times a second to check if the default
>>> route is incorrect and fixing it if necessary, which mitigates the
>>> amount of downtime caused by the bug.
>> 
>> 
>> Can you describe your traffic forwarding setup in more detail?
>> Is it only pf, or do you run netgraph, or other things as well?
>> Do you use flow routing?
> 
> I use PF for NAT, filtering, and rdr rules. ALTQ for bandwidth
> management. I do not use netgraph. I use vlans. PF redirects to squid
> as a transproxy. I'm not familiar with flow routing so unless its
> enabled in 9.1 by default I do not use it.
> 
>> 
>> How frequent does this happen?
> Every other day during periods of heavier Internet-bound traffic.
> 
>> 
>> I'm trying to create a stack graph to see which parts of the network
>> stack are involved in handling your packet.
>> 
>> --
>> Andre
>> 
>>> Please refer to these past posts for more examples and evidence of
>>> other users experiencing this problem:
>>> 
>>> http://forums.freebsd.org/showthread.php?p=211610#post211610
>>> 
>>> 
>>> http://freebsd.1045724.n5.nabble.com/Default-route-quot-random-quot-gateway-modification-bug-td5750820.html
>>> 
>>> http://lists.freebsd.org/pipermail/freebsd-net/2012-March/031879.html
>>> 
>>> http://lists.freebsd.org/pipermail/freebsd-ipfw/2010-September/004361.html
>>> 
>>> There is also a PR that was incorrectly labeled as an IPFW issue.
>>> Myself and others believe this issue is not restricted to the use of
>>> IPFW and that the PR should be relabeled. I am inclined to think it is
>>> strictly a PF issue since I am not using IPFW, however there is
>>> evidence of the default route changing on people using IPFW for past
>>> versions of FreeBSD (7.x/8.x), so perhaps this is related.
>>> 
>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/174749
>>> 
>>> Another PR for the same problem but specific to IPFW and 8.2-RELEASE
>>> 
>>> http://www.freebsd.org/cgi/query-pr.cgi?pr=157796
>>> 
>>> I am hoping someone reading this can give the problem the attention it
>>> deserves. Thank you.
>>> 
>>> -Nick
>>> _______________________________________________
>>> freebsd-net at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-net
>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>>> 
>>> 
>> 
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> 

------------------------------
Randall Stewart
803-317-4952 (cell)