net.add_addr_allfibs=1 behaviour deprecation

[Alexander V. Chernikov]

18.08.2020, 09:17, "Grzegorz Junka" <list1 at gjunka.com>:
> On 18/08/2020 07:54, Julian Elischer wrote:
>>  The reason for the two behaviours is that there are two ways that the
>>  previous behaviour of  "add addresses to the only FIB" could be
>>  interpreted and extended once multiple fibs became available. The
>>  single fib case could be interpreted as either of:
>>
>>  "Add to All N fibs where N == 1"    or     "add to the first (of 1)
>>  fibs".
>>  I decided to do both :-)
>>
>>  At Ironport where I wrote it we had a scenario where I didn't want to
>>  have wrong entries in all the fibs when a new interface was brought
>>  up. Even for a moment. An other scenarios where  for example a tunnel
>>  uses fib 1 but the rest of the system uses fib0 (which points through
>>  the tunnel) The addition of new routes into the tunnel's route when a
>>  new virtual interface is brought up pointing through the tunnel to the
>>  same address, leads in the tunnel immediately redirecting packets
>>  through itself which ends in tears.     SO the obvious thing to do was
>>  to make it possible to only add the entry in the fib that was the
>>  default fib or in the case of Ironport, the fib that was the default
>>  fib of the process adding the interface.
>>
>>  If you had to make a choice I think the '0' choice is the way to go.
>>  All other fibs need to be populated deliberately..
>>
>>  On 8/15/20 4:24 AM, Alexander V. Chernikov wrote:
>>>  18.07.2020, 14:22, "Alexander V. Chernikov" <melifaro at freebsd.org>:
>>>>  Dear FreeBSD users,
>>>>
>>>>  I would like to make net.add_addr_allfibs=0 as the default system
>>>>  behaviour and remove net.add_addr_allfibs.
>>>>  To do so, I would like to collect use cases with
>>>>  net.add_addr_allfibs=1 and multiple fibs, to ensure they can still
>>>>  be supported after removal.
>>>>
>>>>  Background:
>>>>
>>>>  Multi-fib support was added in r178888 [1], 12 years ago. Addition
>>>>  of interface addresses to all fibs was a feature from day 1.
>>>>  The `net.add_addr_allfibs` sysctl  was added in r180840 [2], 12
>>>>  years ago.
>>>>
>>>>  Problem:
>>>>  The goal of the fib support is to provide multiple independent
>>>>  routing tables, isolated from each other.
>>>>  `net.add_addr_allfibs` default tries to shift gears in the opposite
>>>>  direction, unconditionally inserting all addresses to all of the fibs.
>>>>
>>>>  It complicates the logic, kernel code and makes control plane
>>>>  performance decrease with the number of fibs.
>>>>  It make impossible to use the same prefixes in multiple fibs, which
>>>>  may be desired given shortage of IPv4 address space.
>>>>
>>>>  I do understand that there are some cases where such behaviour is
>>>>  desired.
>>>>  For example, it can be used to achieve VRF route leaking or binding
>>>>  on address from different fibs.
>>>>  I would like to collect such cases to consider supporting them in a
>>>>  different way.
>>>>
>>>>  The goal is to make net.add_addr_allfibs=0 default behaviour and
>>>>  remove net.add_addr_allfibs.
>>>>  It will simplify kernel fib-related code and allow bringing more
>>>>  fib-related features. It will also improve fib scaling.
>>>  No objections has been received.
>>>  Next steps:
>>>  * Switch net.add_addr_allfibs to 0 (
>>>  https://reviews.freebsd.org/D26076 )
>>>  * Provide an ability to use nexthops from different fibs
>>>  * Remove net.add_addr_allfibs
>>>>  Timeline:
>>>>  Aug 1: summarising feedback and the usecases, decision on proceeding
>>>>  further
>>>>  Aug 20 (tentative):  patches for supported usecases
>>>>  Sep 15 (tentative):  net.add_addr_allfibs removal.
>>>>
>>>>  [1]: [base Contents of
>>>>  /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=178888&view=markup)
>>>>  [2]: [base Diff of
>>>>  /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839&r2=180840&)
>>>>
>>>>  /Alexander
>
> Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing
> installations, which goes against the least surprise principle so many
> times advocated on FreeBSD lists.
>
> This is just one example:
> https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/
>
> Now, changing the default again might again break existing
> installations, which shouldn't be a reason for not doing it, but might
> be a reason to better communicate it this time around.
I plan to communicate it the following way:
1) this thread
2) GONE_IN13 in the sysctl (which will print console message when set to 1)
3) Release notes
Do you think there are other communication channels one should try to use?
>
> GrzegorzJ
>
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"