option RADIX_MPATH, RT_LINK_IS_UP() and interface routes.

[Nikolay Denev]

Hello,

A quick glance through sys/netinet/ip_output.c shows that interface routes are short-circuited and
not checked for RT_LINK_IS_UP as gateway routes are.

Consider the following scenario :

A pair of redundant routers : RTR1 and RTR2.
Each having dedicated uplink to some ISP and both run BGP, and they also have a dedicated cross-connection.
On the LAN side, they share a IP using CARP.

 Uplink1    Uplink2
    |          |
    |          |
+------+   +------+
| RTR1 |---| RTR2 |
+------+   +------+
    |          |
    |          |
+---+----------+---+
|       LAN        |
+------------------+

Now, if the cable on RTR1 connecting it to the LAN is disconnected,
RTR2 will become carp master and will start receiving packets from clients on LAN and they will be routed ok.
But form the ISP point of view the best path to the network is via RTR1, so the incoming traffic
will still be routed thru RTR1 because it's Uplink1 interface is UP and the BGP session established.
This will cause the packets destined to the LAN to be effectively blackholed, because of the interface route on RTR1.
When using kernel with RADIX_MPATH and ospf on both routers RTR1 will have two routes to the LAN, one interface and one via
the crossconnect to RTR2 but still, the interface route will be consen, regardless of link state up or down.

I'm thinking about checking for RT_LINK_IS_UP on interface routes, or clear the RTF_UP flag on those routes when interface goes link down. Any other solutions/ideas?

Cheers,
Nikolay