ndp and routers with link-local addresses

[Niclas Zeising]

Hi!
It should be possible to have routing interfaces on client facing 
networks with only link-local addresses if you use SLAAC (or specify the 
default route manually).  SLAAC by default uses the link-local address 
in advertisements, and clients on the network uses the router link-local 
address as default gateway.
However, if the interface on the router facing the client network only 
has a link-local (and no global unicast) address, NDP neighbor discovery 
breaks.
I have a couple of issues:
First off, NDP from the router to the client breaks completely, unless 
the client starts the conversation.  Then the router can use existing 
NDP table entries, as long as they are valid.
Secondly, source address selection on the router side for NDP is 
strange.  Instead of using the source address on the interface directly 
connected to the client, the router uses a global unicast address, if 
any is assigned to any interface.  This confuses the client.
All this is tested on FreeBSD 12.1 (both router and client).  I also 
swapped out the router for a Linux install, using Debian 10.4.0.  All 
the cases below works out of the box on Linux.


quick setup using bhyve:

router: two interfaces, connect one interface to a bridge on the host
client: one interface, connected to the same bridge as the router

on the router (vtnet1 is connected to the bridge above)
# ifconfig vtnet1 inet6 fe80::1/64
# route -6 add -net 2001:6b8::/64 -iface vtnet1
# sysctl net.inet6.icmp6.nd6_onlink_ns_rfc4861=1
# sysctl net.inet6.ip6.forwarding=1

net.inet6.icmp6.nd6_onlink_ns_rfc4861=1 is needed, otherwise NDP won't 
work at all.
you can then add any address to vtnet1 to test the source address 
selection, as long as it's not in 2001:6b8::/64 (for instance, use 
2001:6b8:1::1/64)

on the client (vtnet0 is connected to the bridge above)
# ifconfig vtnet0 inet6 2001:6b8::2/64
# route -6 add default fe80::1%vtnet0

dump the traffic on the bridge between the machines

On the router, try to ping the client (2001:6b8::2) and see what 
happens.  This does not work.

On the client, ping any ipv6 address not in 2001:6b8::/64, and watch 
NDP.  (we don't care about the icmp echo requests, the ping is just to 
do the NDP stuff).  NDP works, using link-local address on the router.

Now on the router, ping the client again.  This works if the entry is 
still in the NDP table on the router, which was added in the previous step.

clear ndp: ndp -c

try again on the router, now it's failing again.

Add a global unicast address to the router>
# ifconfig vtnet0 inet6 2001:6b8:1::1/64

Ping the client from the router, still does not work

ping anything from the client and see which source the NDP packets are 
using.  Linux uses the link-local address of vtnet1 on the router, 
FreeBSD uses the global unicast address of vtnet0 for packets from the 
router.  This should add things to the NDP table on both router and client.

Ping the client on the router again, now it works, for a while, until 
the router tries to refresh its NDP table, then it stops working.
setting net.inet6.icmp6.nd6_onlink_ns_rfc4861=1 on the client and it 
keeps working.


More details:

With this setup, NDP works in one direction.  I can, from the client, 
get NDP working and get the ethernet address of the interface with 
fe80::1 on the router (by trying to ping6 something, for instance). 
When this is one, the router also have the client in it's ndp neighbor 
table.  Howerver, the opposite does not work.  I can't from the router 
get the information of the client.  When I try, from the router, to 
ping6 the client, all I get is

ping6: sendmsg: No buffer space available

and I don't see any NDP packets going out on the line.
If the entry for the client is already in the router NDP list, then it 
works.  To me, it looks like the router gets confused and for some 
reason won't do any neighbor discovery on its own, but it works if it is 
done already.

I also found that the source address selection for NDP is a bit strange. 
  It looks like FreeBSD prefers to use a global unicast address as 
source for ndp neighbor solicitations.  If I add an global unicast 
address to any other interface on the router, that is used as the 
source, instead of the link-local address on the interface connected to 
the client.  This sometimes confuses the FreeBSD client, at least unless 
net.inet6.icmp6.nd6_onlink_ns_rfc4861 is set to 1.



With Linux this works.  I swapped out the router for a Linux one, with 
the same setup things work out of the box.  Linux is also using the 
closest link-local address as a source for NDP NS.

I don't know if the FreeBSD behavior is expected behavior, or if there 
are bugs, but it is causing interoperability problems.

All this are with FreeBSD 12.1.  For the Linux router I used Debian 10.4.0.


Regards
-- 
Niclas