routing changes from Sept 7th and possible ppp breakage (RELENG_13)

From: mike tancsa <mike_at_sentex.net>
Date: Thu, 09 Sep 2021 21:38:49 UTC
On our standard nanobsd router image we use, I noticed failover routing
broke all of a sudden.  We are not using multiple FIBs, but somehow we
now have two default routes I am guessing because of multipath.  (Note,
we just started testing RELENG_13 so its possible I am making bad
assumptions somewhere.)

default            10.255.255.1       UGS         6   1500       igb0
default            10.1.0.1           UGS        17   1500      tun10

I am pretty sure from the code base from 3+ days ago, I would not see
this condition.

In the past, my nano image would get a default route via DHCP.  If that
path was not functional or blocked, my monitoring software would detect
the failure and fire up PPP which has

 add! default HISADDR

That now seems to add a second default route instead of getting rid of
the old one.

if I add net.route.multipath=0

I get back the old behaviour

According to the ppp docs

         If the add! command is used (note the trailing "!"), then if the
         route already exists, it will be updated as with the `route change'
         command (see route(8) for further details).

Where as now, it doesnt seem to replace it, it seems to just add another
default route.

For my purposes, setting net.route.multipath=0 seems to work just fine,
but I wonder if this warrants a note in ppp's docs since it no longer
seems to work that way in RELENG_13 ?

    ---Mike