Getting CARP to broadcast on a different interface

[Niklaas Baudet von Gersdorff]

Matthew Grooms [2016-06-08 11:02 -0500] :

> Rewriting the multicast destination would be a neat trick, but sadly no. 
> You can't rewrite a destination address on egress. Using a route-to rule 
> would only modify the destination MAC address. If you were using 
> OpenBSD, you would switch from multicast to unicast using the syncpeer 
> option. Unfortunately that's not supported on FreeBSD.
> 
> At one point I wrote a broadcast relay daemon to forward select UDP 
> broadcast traffic between two networks separated by an IPsec tunnel. It 
> had limited utility, but it worked well for what I needed it to do. I 
> wonder if someone has written a multicast relay daemon that works in a 
> similar fashion. If so, you could use it to forward CARP traffic to a 
> peer. Super ugly, but it would probably do the trick in this scenario.

Thank you for your explanation. I am far from understanding the
underlying mechanisms to really get my head around possible solutions --
your comments helped. And, as you said, maybe there is someone else who
came across this problem and has implemented a workaround.

David DeSimone [2016-06-08 15:15 +0000] :

> One of the purposes of the CARP announcements is to announce the
> location of the virtual mac address to the upstream switch fabric.
> Since CARP uses a virtual mac that floats between multiple ports, you
> need to have the CARP master continually assert that its particular
> port is the target that should be used for delivery of packets to the
> virtual MAC address.  Without this function, switches might still
> mistakenly deliver their frames to the standby node.
> 
> The CARP announcements are also helpful in detecting and routing
> around some odd failure scenarios, such as a failure within the
> upstream fabric, where the master sees link on its port, but can't
> actually send frames that reach the rest of the network.  If the
> standby can't hear the master's announcements any more, it can promote
> itself to master and hopefully keep your cluster online.  This would
> not happen without the announcement feature.
> 
> I would hope you could explain this to your provider and get them to
> white-list CARP announcements because they are defeating important
> safety features you wish to use.

Thank you too, David. I have already contacted my provider about this
and they asked me to use keepalived -- which is deprecated in ports...
I shall keep them updated about my progress in solving the issue though.
Maybe I am lucky and can convince them to white-list the packets at some
point.

At this stage I am thinking about running CARP on the additional private
virtual interface vtnet1 that I got from my provider. As I understood
it, nothing is black-listed on these interfaces so CARP packets should
go through.

Then, I could use devd to assign the public failover IP (that I actually
wanted to share with CARP on vtnet0) to the public interface vtnet0.
CARP(4) provides an example on how to use carp status change events for
additional scripting:

--------------------8<--------------------

Processing  of carp status change events can be set up by using the fol-
lowing devd.conf rule:

    notify 0 {
        match "system"      "CARP";
        match "subsystem"       "[0-9]+@[0-9a-z]+";
        match "type"        "(MASTER|BACKUP)";
        action "/root/carpcontrol.sh $subsystem $type";
    };

-------------------->8--------------------

Depending von $type, carpcontrol.sh could either

    ifconfig vtnet0 <public-ip> alias

or

    ifconfig vtnet0 <public-ip> -alias

I am not sure whether that works but I'll post my solution if I find
one.

In the meanwhile, I appreciate any further comment.

    Niklaas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-net/attachments/20160608/96b41c1e/attachment.sig>