net.add_addr_allfibs=1 behaviour deprecation

[Julian Elischer]

The reason for the two behaviours is that there are two ways that the 
previous behaviour of  "add addresses to the only FIB" could be 
interpreted and extended once multiple fibs became available. The 
single fib case could be interpreted as either of:

"Add to All N fibs where N == 1"    or     "add to the first (of 1) 
fibs".
I decided to do both :-)

At Ironport where I wrote it we had a scenario where I didn't want to 
have wrong entries in all the fibs when a new interface was brought 
up. Even for a moment. An other scenarios where  for example a tunnel 
uses fib 1 but the rest of the system uses fib0 (which points through 
the tunnel) The addition of new routes into the tunnel's route when a 
new virtual interface is brought up pointing through the tunnel to the 
same address, leads in the tunnel immediately redirecting packets 
through itself which ends in tears.     SO the obvious thing to do was 
to make it possible to only add the entry in the fib that was the 
default fib or in the case of Ironport, the fib that was the default 
fib of the process adding the interface.

If you had to make a choice I think the '0' choice is the way to go. 
All other fibs need to be populated deliberately..

On 8/15/20 4:24 AM, Alexander V. Chernikov wrote:
> 18.07.2020, 14:22, "Alexander V. Chernikov" <melifaro at freebsd.org>:
>> Dear FreeBSD users,
>>
>> I would like to make net.add_addr_allfibs=0 as the default system behaviour and remove net.add_addr_allfibs.
>> To do so, I would like to collect use cases with net.add_addr_allfibs=1 and multiple fibs, to ensure they can still be supported after removal.
>>
>> Background:
>>
>> Multi-fib support was added in r178888 [1], 12 years ago. Addition of interface addresses to all fibs was a feature from day 1.
>> The `net.add_addr_allfibs` sysctl  was added in r180840 [2], 12 years ago.
>>
>> Problem:
>> The goal of the fib support is to provide multiple independent routing tables, isolated from each other.
>> `net.add_addr_allfibs` default tries to shift gears in the opposite direction, unconditionally inserting all addresses to all of the fibs.
>>
>> It complicates the logic, kernel code and makes control plane performance decrease with the number of fibs.
>> It make impossible to use the same prefixes in multiple fibs, which may be desired given shortage of IPv4 address space.
>>
>> I do understand that there are some cases where such behaviour is desired.
>> For example, it can be used to achieve VRF route leaking or binding on address from different fibs.
>> I would like to collect such cases to consider supporting them in a different way.
>>
>> The goal is to make net.add_addr_allfibs=0 default behaviour and remove net.add_addr_allfibs.
>> It will simplify kernel fib-related code and allow bringing more fib-related features. It will also improve fib scaling.
> No objections has been received.
> Next steps:
> * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 )
> * Provide an ability to use nexthops from different fibs
> * Remove net.add_addr_allfibs
>> Timeline:
>> Aug 1: summarising feedback and the usecases, decision on proceeding further
>> Aug 20 (tentative):  patches for supported usecases
>> Sep 15 (tentative):  net.add_addr_allfibs removal.
>>
>> [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=178888&view=markup)
>> [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839&r2=180840&)
>>
>> /Alexander
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>