Client Networking Issues / NIC Lab

[Thomas Mueller]

from Kevin Bowling:

> I have been looking into client networking issues in FreeBSD lately.
> To summarize the situation, common NICs like Intel gigabit (e1000 aka
> lem(4)/em(4)/igb(4)), Realtek (re(4)), Aquantia, and Tehuti Networks  
> are unmaintained or not present on FreeBSD.  The purpose of this
> thread is to gauge whether that matters, and if it does what to do.  I
> believe it is important because we are losing out on a pipeline of new
> contributors by not supporting client hardware well.  We risk losing
> NAS, firewall, and other embedded users which may not be large enough
> to negotiate with these vendors for support or have the volume to do
> custom BOMs to avoid risky parts.  My opinion has been developed after
> researching the drivers, Bugzilla, and various internet forums where
> end users exchange advice or ask for help where FreeBSD is the
> underlying cause.

> e1000 is in the best shape, with recent vendor involvement, but covers
> 20 years of silicon with over 100 chipsets (of which at least 60 are
> significant variations).  Datasheets are readily available for most of
> them, as well as "specification updates" which list errata.  There are
> chipsets which have been completely broken for several years.  More 
> common, there are cases that lead to user frustration, including with
> the most recent hardware.  All of the silicon tends to have
> significant bugs around PCIe, TSO, NC-SI (IPMI sideband), arbitration
> conflicts with ME and more.  Intel doesn't patch the microcode on
> these, but many of the issues can be worked around in software.
> Performing an audit of the driver will take quite a while, and making
> and testing changes gives me concern.  When we (my previous employer
> and team) converted these drivers to iflib, we fixed some of the
> common cases for PCIe and TSO issues but only had a handful of chips
> to test against, so the driver works better for some and worse or not
> at all for others.  I have started fixing some of the bugs in
> Bugzilla, but I only have a few e1000 variants on hand to test, and I
> have an unrelated full time job so this is just occupying limited
> spare time as a hobby.
         
> re(4) is in pretty abhorrent state.  All of these chips require  
> runtime patching of the phy (which I believe is a DSP algorithm that
> gets improved over time) and mcu code.  That is totally absent in
> FreeBSD.  A vendor driver exists in net/realtek-re-kmod which contains
> the fixups and works alright for many users.  This driver cannot be
> imported into FreeBSD as is.  There is a strange use of the C
> PreProcessor which blows up compile time and driver size needlessly.
> The out of tree driver has a different set of supported adapters, so
> some kind of meld is necessary.  Realtek does not provide public chip
> documentation, I am trying to see if they will grant NDA access to
> contributors.

Some re(4) chips work in FreeBSD, some don't.  I gave up on FreeBSD 12.x because of re(4) deficiencies.

Sad to have to seek NDA access for an open-source project like FreeBSD.

NetBSD seems to work better.

OpenBSD GPT support is in such condition as to render incompatible with my system.

Haiku, maybe? 

My computer with on-motherboard AR9271 wireless chip, dating to 2013, is still waiting for FreeBSD support.
         
> Aquantia has an out of tree driver in net/aquantia-atlantic-kmod.  The  
> code is not currently in a place where I'd like to see it in the tree.
> I am not really sure how common these are, the company was acquired by
> Marvell which is still producing them as a client networking option
> while they have other IP for higher end/speed.
         
> Tehuti Networks seems to have gone out of business.  Probably not
> worth worrying about.
         
> 1) Do nothing.  This situation has gone on for a while.  Users are
> somewhat accustomed to purchasing FreeBSD-specific hardware for things
> like SOHO gateways and NAS.  A lot of people just revert back to Linux
> for client use.  OpenBSD seems to have more active contribution around
> this kind of thing and works better for common cases so that may be   
> another exit ramp.

> 2) Quantify usage data and beg the vendors for help.  This might work 
> for Intel, however these devices have transferred to a client team at
> intel that does not plan to support FreeBSD, and intel does not keep 
> test systems around long enough to meet FreeBSD user's needs. Realtek
> is a similar story, I am unsure how long they hold on to test systems 
> and would probably need technical guidance to work with the FreeBSD 
> community.  Unsure about Marvell, I've never worked with them.

> 3) Build a NIC lab and focus on building community support.  It would
> also give the vendors a place to test hardware their labs have purged 
> (due to IT asset management policies or other bureaucratic blunders).
> Set some boundaries like a 15 year window of chipsets which should
> cover practical embedded use cases.  There are backplane systems
> and/or external PCI(e) expansion systems that could be assembled to 
> house a large number of NICs.  It would probably be cheaper than this,
> but say a budget of $15000USD is enough to purchase some expansions, a
> couple managed switches, and a few dozen common NICs.  Community
> members may also send in NICs they wish to see supported or undergo
> testing.  For this to work out long term, there needs to be a quorum
> of people interested in collaborating on the issue.  There are some  
> risks around simply setting this up, depending on the configuration,
> the bus topology may introduce problems unrelated to the NICs and we'd
> probably need some semi-automated device.hints or devctl stuff to keep
> from over provisioning system resources (work on a subset of cards at
> a time).  An interesting extension of this would be a semi-automated
> validation setup for subsystem changes (significant driver changes,  
> iflib, lro, etc). 

> 4) ???  


Tom