CARP over VLAN over LAGG

[Michael Gmelin]

> On 8. Sep 2020, at 15:23, Julien Cigar <julien at perdition.city> wrote:
> 
> On Tue, Sep 01, 2020 at 10:13:23AM +0200, Julien Cigar wrote:
>>> On Mon, Aug 31, 2020 at 01:55:52PM +0200, Michael Gmelin wrote:
>>> 
>>> 
>>>> On 31. Aug 2020, at 10:37, Julien Cigar <julien at perdition.city> wrote:
>>>> 
>>>> On Fri, Aug 28, 2020 at 04:52:01PM +0200, Julien Cigar wrote:
>>>>> Hello,
>>>>> 
>>>>> I have a "highly available" router/firewall with the following
>>>>> configuration (1). Those are plugged in two 2930F (with VSF) using LACP.
>>>>> It works well, except that I have some weird issues with the CARP 
>>>>> demotion counter when I'm unplugging some interfaces involved in the 
>>>>> lagg/carp setup, for example if I unplug/replug igb0 and igb1 in this 
>>>>> case:
>>>>> 
>>>>> (dmesg):
>>>>> igb0: link state changed to DOWN
>>>>> igb1: link state changed to DOWN
>>>>> carp: demoted by 240 to 240 (send error 50 on vlan11)
>>>>> carp: 11 at vlan11: MASTER -> BACKUP (more frequent advertisement received)
>>>>> vlan11: deletion failed: 3
>>>>> igb1: link state changed to UP
>>>>> igb0: link state changed to UP
>>>>> 
>>>>> then the CARP status stays to BACKUP unless I demote the CARP demotion
>>>>> counter manually with: sudo sysctl net.inet.carp.demotion=-240:
>>>>> 
>>>>> (dmesg):
>>>>> carp: demoted by -240 to 0 (sysctl)
>>>>> carp: 11 at vlan11: BACKUP -> MASTER (preempting a slower master)
>>>>> 
>>>>> I guess this is because it takes some time for lagg/lacp to converge and
>>>>> thus carp thinks that there is a problematic condition as it experiences
>>>>> problems with sending announcements..
>>>>> 
>>>>> What it the best way to handle this?
>>>> 
>>>> I'm wondering if setting net.inet.carp.senderr_demotion_factor to "0"
>>>> could be a solution? Are there any downsides of setting this to "0"
>>>> instead of "240"?
>>>> 
>>> 
>>> Sharing your pf.conf from both hosts could be helpful analyzing the issue.
>> 
>> Here is my pf.conf (it's the same on both host):
>> https://gist.github.com/silenius/b758851f03c28ef8caaa53cfe381c455
>> 
>> However, I don't think pf is the issue here, the problem is that there
>> is a slight delay when LAGG/LACP converge and thus CARP increase the
>> demotion counter by net.inet.carp.senderr_demotion_factor (240).
> 
> I can confirm that after setting net.inet.carp.senderr_demotion_factor=0
> (instead of 240) it works as expected.
> 

Cool, thanks for posting the solution. I don’t think I ever used CARP with lagg (usually I had redundant firewalls with CARP and servers using lagg connected to both of them).

Cheers,
Michael

>> 
>>> 
>>> -m
>>> 
>>> 
>> 
>> -- 
>> Julien Cigar
>> Belgian Biodiversity Platform (http://www.biodiversity.be)
>> PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
>> No trees were killed in the creation of this message.
>> However, many electrons were terribly inconvenienced.
>> _______________________________________________
>> freebsd-net at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-net
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
> 
> -- 
> Julien Cigar
> Belgian Biodiversity Platform (http://www.biodiversity.be)
> PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
> No trees were killed in the creation of this message.
> However, many electrons were terribly inconvenienced.