Default route changes unexpectedly

[Nick Rogers]

Hello,

I am attempting to create awareness of a serious issue affecting users
of FreeBSD 9.x and PF. There appears to be a bug that allows the
kernel's routing table to be corrupted by traffic routing through the
system. Under heavy traffic load, the default route can seemingly
randomly change to an IP address that is not directly connected to the
network (i.e., is not configured anywhere). Dhclient is not in the
mix, nor is routed, bgpd, etc. Running `route monitor` shows no
evidence of the change in the default route. The one commonality
between all the systems experiencing this problem seems to be the use
of PF.

Obviously this is a serious problem as it causes all Internet-bound
traffic to stop routing until the default route is corrected. Some
users, including myself, are working around this problem by installing
a script that runs multiple times a second to check if the default
route is incorrect and fixing it if necessary, which mitigates the
amount of downtime caused by the bug.

Please refer to these past posts for more examples and evidence of
other users experiencing this problem:

http://forums.freebsd.org/showthread.php?p=211610#post211610

http://freebsd.1045724.n5.nabble.com/Default-route-quot-random-quot-gateway-modification-bug-td5750820.html

http://lists.freebsd.org/pipermail/freebsd-net/2012-March/031879.html

http://lists.freebsd.org/pipermail/freebsd-ipfw/2010-September/004361.html

There is also a PR that was incorrectly labeled as an IPFW issue.
Myself and others believe this issue is not restricted to the use of
IPFW and that the PR should be relabeled. I am inclined to think it is
strictly a PF issue since I am not using IPFW, however there is
evidence of the default route changing on people using IPFW for past
versions of FreeBSD (7.x/8.x), so perhaps this is related.

http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/174749

Another PR for the same problem but specific to IPFW and 8.2-RELEASE

http://www.freebsd.org/cgi/query-pr.cgi?pr=157796

I am hoping someone reading this can give the problem the attention it
deserves. Thank you.

-Nick