Carp unpingable on vlan interfaces with lagg on emulex cards

[Phi-Phong NGUYEN]

I have two routers running freeBSD 10.1-RELEASE with 10gigaethernet
Emulex cards (OCm14104-UX-D + OCe14102‑UX-D)
On each router :
ifconfig_lagg2="laggproto lacp laggport oce0 laggport oce1 laggport oce4
laggport oce6"

Several vlans are build within the lagg :
On router1 :
[...]
cloned_interfaces=".... vlan3101 ..."
ifconfig_vlan3101="vlan 3101 vlandev lagg2 X.X.X.252/24"
ifconfig_vlan3101_alias0="vhid 121 advskew 100 pass mypass X.X.X.254/32"
[...]
vlan3101: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
metric 0 mtu 1500
    options=303<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 00:90:fa:xx:xx:xx
    inet X.X.X.252 netmask 0xffffff00 broadcast X.X.X.255
    inet X.X.X.254 netmask 0xffffffff broadcast X.X.X.254 vhid 121
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 3101 parent interface: lagg2
    carp: BACKUP vhid 121 advbase 1 advskew 100

On router2 :
[...]
cloned_interfaces=".... vlan3101 ..."
ifconfig_vlan3101="vlan 3101 vlandev lagg2 X.X.X.252/24"
ifconfig_vlan3101_alias0="vhid 121 pass mypass X.X.X.254/32"
[...]
vlan3101: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
metric 0 mtu 1500
    options=303<RXCSUM,TXCSUM,TSO4,TSO6>
    ether 00:90:fa:xx:xx:xx
    inet X.X.X.253 netmask 0xffffff00 broadcast X.X.X.255
    inet X.X.X.254 netmask 0xffffffff broadcast X.X.X.254 vhid 121
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    media: Ethernet autoselect
    status: active
    vlan: 3101 parent interface: lagg2
    carp: MASTER vhid 121 advbase 1 advskew 0

So, router2 is the master VRRP for vlan3101, ang changing advskew to 150
on router2 leaves router1 become the master.
BUT :
Router1 can ping X.X.X.253 and router2 is able to ping X.X.X.252, alas,
router1 can't ping X.X.X.254 (the virtual ip address) as a host
belonging to the 3101 vlan.

The arp request does work :
router1 # ping X.X.X.254
    <- no "Host is down", no unreachable
router1 # arp -n X.X.X.254
? (X.X.X.254) at 00:00:5e:00:01:79 on vlan3101 expires in 1129 seconds
[vlan]

Already try :
1) Reduce the lagg to one single interface -> Doesn't work
2) Create the vlans on a physical emulex interface -> Doesn't work
3) Downgrade to 9.3-RELEASE with old syntax carp -> Doesn't work
4) Upgrade all the firmwares including Emulex cards.
5) Create the lagg on a physical copper interface (Intel X540) -> Does
work !!

So, my questions are :
1) Is it related to the Emulex cards ?
2) Is there a workaround for this sort of problem ?
2) If I replace some Emulex cards with Intel ones (Say X520 items), is
the result guaranteed ? At $ 700-$ 800 each, this is a crucial question.

More information :
router1 # sysctl -a |grep carp
net.inet.carp.allow: 1
net.inet.carp.preempt: 1
net.inet.carp.log: 1
net.inet.carp.demotion: 0
net.inet.carp.senderr_demotion_factor: 0
net.inet.carp.ifdown_demotion_factor: 240

Thanks in advance.

-- 
Phi-Phong NGUYEN
Service informatique
Institut de Biologie ENS
46 rue d'Ulm
75230 PARIS CEDEX 05
Tel: 01 44 32 36 34
Fax: 01 44 32 36 30