option RADIX_MPATH, RT_LINK_IS_UP() and interface routes.

[Nikos Vassiliadis]

On 2/12/2011 11:36 AM, Nikolay Denev wrote:
> Hello,
>
> A quick glance through sys/netinet/ip_output.c shows that interface routes are short-circuited and
> not checked for RT_LINK_IS_UP as gateway routes are.
>
> Consider the following scenario :
>
> A pair of redundant routers : RTR1 and RTR2.
> Each having dedicated uplink to some ISP and both run BGP, and they also have a dedicated cross-connection.
> On the LAN side, they share a IP using CARP.
>
>   Uplink1    Uplink2
>      |          |
>      |          |
> +------+   +------+
> | RTR1 |---| RTR2 |
> +------+   +------+
>      |          |
>      |          |
> +---+----------+---+
> |       LAN        |
> +------------------+
>
> Now, if the cable on RTR1 connecting it to the LAN is disconnected,
> RTR2 will become carp master and will start receiving packets from clients on LAN and they will be routed ok.
> But form the ISP point of view the best path to the network is via RTR1, so the incoming traffic
> will still be routed thru RTR1 because it's Uplink1 interface is UP and the BGP session established.
> This will cause the packets destined to the LAN to be effectively blackholed, because of the interface route on RTR1.
> When using kernel with RADIX_MPATH and ospf on both routers RTR1 will have two routes to the LAN, one interface and one via
> the crossconnect to RTR2 but still, the interface route will be consen, regardless of link state up or down.
>
> I'm thinking about checking for RT_LINK_IS_UP on interface routes, or clear the RTF_UP flag on those routes when interface goes link down. Any other solutions/ideas?

You could try sysutils/heartbeat which is similar in concept with
CARP, but runs in userspace and gives you the ability to run scripts
in case of a change. It can also ping any IP address to monitor a
link's health, that is, "if I cannot get replies from hosta in my LAN,
something must be wrong with me, I'll notify the backup host and change
my status to backup".

Heartbeat wikipedia article:
http://en.wikipedia.org/wiki/Linux-HA

HTH, Nikos