Vrrp/CARP/ucarp Problems

[Stefan Lambrev]

HI all,

Ross Draper wrote:
> Hi All
>  
> I was wondering if I could get some advice from those of you who have
> successfully implemented ip address failover systems such as carp and
> freevrrpd.
>  
> I am trying to set up a high availability web loadbalancer using a pair
> of freebsd 6.2 boxes. I have tried a number of ways to perform failover
> but always seem to be hitting a problem.
>  
> UCARP
> Pro's:This would be my ideal solution as the startup/shutdown scripts
> enable me to stop and start my applications and add aliases to adaptors
> easily.
> Cons: When the backup box is rebooted it always comes up advertising
> itself as the master then after a few seconds reverts to backup,
> although I was under the impression it was supposed to wait and listen
> for advertisements(it doesnt seem to). The backup boxes initial
> gratuitous arp as a master is sufficient to poison any traffic from the
> local router to the shared ip address. Only solution was to use arp-sk
> to send gratuitous arps every few secs, however, arp-sk was a bit flakey
> and it was a bodge.
>  
> CARP
> Pro's: stable and built into the kernel. Could enable acive/active arp
> load sharing at a later point.
> Cons: There is a Freebsd bug (I've seen it discussed on the lists) where
> the creation and destroyal of a carp interface causes a kernel panic.
> Also, there is no support for start/stop scripts.
>   
I do not have experience with ucarp and freevrrpd, so I can talk only 
about CARP :)
The bug you are talking is fixed in -CURRENT, and you can trigger it 
only if you have more then 1
carp interface per host.
I fetch changes from -current and made patch for -stable, that seems to 
work without problems.
There are other bugs, and I'm not sure what is their status, but you 
always can search for PR.
I do not think start/stop scripts are problem as average sysadmin can 
solve this for itself :)
>  
> Freevrrpd
> Pros: Mac address changing removes some of the arp timeout
> issues/gratuitous arp problems and it supports start/stop scripts
> Cons: I'm finding that upon rebooting the backup unit it correctly
> starts as a backup, then three seconds later syslogs that it is the
> master and changes its mac address accordingly. although a sniff of the
> network traffic indicates it is sending the right advertisements(lower
> priority), it never goes into backup mode again.
>  
> So, what am I doing wrong? Are these common problems, or something that
> appears specific to my hosts/switches? are there more suitable options?
> The loadbalancers are all single homed and I have tried a mixture of xl,
> bge and fxp cards.  
>  
> Any help/suggestions much appreciated, also, any links to a perl based
> gratuitous arp util would be great!
>  
> Many thanks
>
> Ross 
>
> PS - Apologies if you see multiple copies of this message, I seem to be
> having trouble getting mails onto the list.
>
>
>
> All correspondence, attachments and agreements remain strictly subject to fully executed contract. (c) GCap Media plc 2006. All rights remain reserved. This e-mail (and any attachments) contains information which may be confidential, subject to intellectual property protection and may be legally privileged and protected from disclosure and unauthorised use. It is intended solely for the use of the individual(s) or entity to whom it is addressed and others specifically authorised to receive it. If you are not the intended recipient of this e-mail or any parts of it please telephone 020 7054 8000 immediately upon receipt. No other person is authorised to copy, adapt, forward, disclose, distribute or retain this e-mail in any form without prior specific permission in writing from an authorised representative of GCap Media plc. We will not accept liability for any claims arising as a result of the use of the internet to transmit information by or to GCap Media plc.
>
> GCap Media plc. Registered address: 30 Leicester Square, London WC2H 7LA.  Registered in England & Wales with No. 923454
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>   
P.S. the attached patch is little old so I'm not sure it still apply 
cleanly to the latest -stable :)
I tested base functionality with patched carp, but still do not have 
server in production with it, so be careful!

-- 
Best Wishes,
Stefan Lambrev
ICQ# 24134177

-------------- next part --------------
--- src/sys/netinet/ip_carp.c.orig	Thu Feb  1 18:53:55 2007
+++ src/sys/netinet/ip_carp.c	Tue Feb  6 18:41:24 2007
@@ -191,7 +191,7 @@
 static void	carp_input_c(struct mbuf *, struct carp_header *, sa_family_t);
 static int 	carp_clone_create(struct if_clone *, int);
 static void 	carp_clone_destroy(struct ifnet *);
-static void	carpdetach(struct carp_softc *);
+static void	carpdetach(struct carp_softc *, int);
 static int	carp_prepare_ad(struct mbuf *, struct carp_softc *,
 		    struct carp_header *);
 static void	carp_send_ad_all(void);
@@ -406,9 +406,7 @@
 
 	if (sc->sc_carpdev)
 		CARP_SCLOCK(sc);
-	carpdetach(sc);	
-	if (sc->sc_carpdev)
-		CARP_SCUNLOCK(sc);
+	carpdetach(sc, 1);	/* Returns unlocked. */
 
 	mtx_lock(&carp_mtx);
 	LIST_REMOVE(sc, sc_next);
@@ -420,7 +418,7 @@
 }
 
 static void
-carpdetach(struct carp_softc *sc)
+carpdetach(struct carp_softc *sc, int unlock)	
 {
 	struct carp_if *cif;
 
@@ -450,9 +448,10 @@
 			sc->sc_carpdev->if_carp = NULL;
 			CARP_LOCK_DESTROY(cif);
 			FREE(cif, M_IFADDR);
-		}
+		} else if (unlock)
+			CARP_UNLOCK(cif);
+		sc->sc_carpdev = NULL;
 	}
-        sc->sc_carpdev = NULL;
 }
 
 /* Detach an interface from the carp. */
@@ -471,7 +470,7 @@
 	CARP_LOCK(cif);
 	for (sc = TAILQ_FIRST(&cif->vhif_vrs); sc; sc = nextsc) {
 		nextsc = TAILQ_NEXT(sc, sc_list);
-		carpdetach(sc);
+		carpdetach(sc, 0);
 	}
 }