CARP interfaces and mastership issue

[Damien Fleuriot]

> 
> 
> 
> Quick follow-up again.
> 
> This is the code for sys/netinet/ip_carp.c on FreeBSD 8.2, OpenBSD
> 3.8, OpenBSD 3.9 in function carp_setrun(struct carp_softc *sc,
> sa_family_t af)
> 
> 
> 
> FREEBSD 8.2-PRERELEASE with init + preempt => auto MASTER bug
> Function starts at line 1371.
> ---
>         switch (sc->sc_state) {
>         case INIT:
>                 if (carp_opts[CARPCTL_PREEMPT] && !carp_suppress_preempt) {
>                         carp_send_ad_locked(sc);
>                         carp_send_arp(sc);
> #ifdef INET6
>                         carp_send_na(sc);
> #endif /* INET6 */
>                         CARP_LOG("%s: INIT -> MASTER (preempting)\n",
>                             SC2IFP(sc)->if_xname);
>                         carp_set_state(sc, MASTER);
>                         carp_setroute(sc, RTM_ADD);
>                 } else {
>                         CARP_LOG("%s: INIT -> BACKUP\n", SC2IFP(sc)->if_xname);
>                         carp_set_state(sc, BACKUP);
>                         carp_setroute(sc, RTM_DELETE);
>                         carp_setrun(sc, 0);
>                 }
>                 break;
> ---
> 
> OPENBSD 3.8 with init + preempt => auto MASTER bug
> Function starts at line 1293.
> ---
>         case INIT:
>                 if (carp_opts[CARPCTL_PREEMPT] && !carp_suppress_preempt) {
>                         carp_set_state(sc, MASTER);
>                         carp_setroute(sc, RTM_ADD);
>                         carp_send_ad(sc);
>                         carp_send_arp(sc);
> #ifdef INET6
>                         carp_send_na(sc);
> #endif /* INET6 */
>                 } else {
>                         carp_set_state(sc, BACKUP);
>                         carp_setroute(sc, RTM_DELETE);
>                         carp_setrun(sc, 0);
>                 }
>                 break;
> ---
> 
> 
> 
> OPENBSD 3.9 with bug fixed
> Function starts at line 1348.
> ---
>         switch (sc->sc_state) {
>         case INIT:
>                 carp_set_state(sc, BACKUP);
>                 carp_setroute(sc, RTM_DELETE);
>                 carp_setrun(sc, 0);
>                 break;
> ---
> 
> 
> It looks like the root cause is there.
> 
> I'll rebuild and test, keep you updated.




Find below my test results with the OpenBSD39 implementation which
forces an INIT -> BACKUP transition regardless of preempt.






# sysctl net.inet.carp.preempt
net.inet.carp.preempt: 1

# sysctl net.inet.carp.suppress_preempt
net.inet.carp.suppress_preempt: 0

# ifconfig carp17
carp17: flags=8<LOOPBACK> metric 0 mtu 1500
	inet 46.182.[snip] netmask 0xffffffff
	inet 46.182.[snip] netmask 0xffffffff
	inet 46.182.[snip] netmask 0xffffffff
	inet 46.182.[snip] netmask 0xffffffff
	inet 46.182.[snip] netmask 0xffffffff
	carp: INIT vhid 117 advbase 1 advskew 200

# ifconfig carp17 up; ./check_carp17_status.sh
count: 0	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 1	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 2	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 3	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 4	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 5	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 6	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 7	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 8	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 9	carp: BACKUP vhid 117 advbase 1 advskew 200
count: 10	carp: BACKUP vhid 117 advbase 1 advskew 200

# dmesg
carp17: INIT -> BACKUP
carp17: link state changed to DOWN



Looks like it works.

I'm afraid I cannot test actual preemption by shutting down a physical
interface on the MASTER, because they're actually used in production and
I've got users logged in to their VPN on pf1.

I see no reason this should break anything however, it only forces the
CARP interface to assume a BACKUP state right after INIT, as it normally
should, regardless of preemption.

I'm filling a PR + patch.