Re: USB4/TBT3 support

Hi Joe,

Sorry for taking a long while responding to your email.

Between some changes on the personal level, really bad luck securing the
main TBT3/USB4 hosts and saving up to secure a proper set of HW for the
task, it took me that long while to come back on the beginning of the track.

I've compiled in [1]:
- All the details you've shared in your email message which includes the
notes from scottl@
- The notes left behind by hps@ (may he RIP)
- What I have in mind of how to pick up the work from here and going FWD

Any comments and/or feedback or even shouting is more than welcome

And for the rest, I wish. you and all of the FreeBSD community happy and
peaceful holidays

[1] https://wiki.freebsd.org/MohammadNoureldin/FreeBSDUSB4TBT3Support

On Fri, Jul 19, 2024 at 9:38 PM Joseph Mingrone <jrm@freebsd.org> wrote:

> Hello,
>
> Is anyone working on USB4 / Thunderbolt 3 (TBT3) support?
>
> Scott Long did a lot of work on this a few years ago, but he had to move
> on to other things, so he passed things on to Hans Petter Selasky.
> Fortunately, Hans Petter dropped the code in a public repository.
>
> https://github.com/hselasky/usb4
>
> https://github.com/hselasky/usb4/commit/dd85c216a2a6bee5361c7166595ba6ca461578b5
>
> Here is an overview of what Scott shared with me.
>
> Mostly completed work:
>
> - Debug/Trace framework
> - NHI controller driver
> - PCIe bridge driver
> - WMI driver
> - Integrated Connection Manager handshake and authentication handling
> - Router and Config Space layer handling (in progress, almost complete)
>
> Remaining work:
>
> - tbtconfig (userland tool)
> - man pages
> - DMAR/IOMMU integration, PCIe tunnelling control
> - Support for resetting and firmware flashing on the NHI via out-of-band
> control
> - Host Connection Manager
> - Cross-domain login
> - ThunderboltIP
>
> Here are the details that Scott shared.
>
>      The driver originally targeted the Thunderbolt 3 controllers that
>      were sold under the names “AlpineRidge" and “Icelake”, in the late
>      2010's, before the USB standards group publicly released the USB4
>      spec.  The driver set I wrote was complete enough to activate
>      Thunderbolt3 peripherals that otherwise would be disabled by
>      default when plugged in.  The driver also attempted to make it
>      easier to identify things like PCIe tunnels in the topology, but
>      that was mostly cosmetic.  Unfortunately, the AlpineRidge chips
>      proved to be extremely hard to work with despite their wide
>      availability, and I spent way too much time fighting them and not
>      enough time developing more useful functionality.  The WMI driver
>      was written to work around vexing problems with the Alpine Ridge
>      controller that I never figured out.
>
>      Much of the infrastructure from the TBT3 support extends to modern
>      USB4 controllers, but there are still a lot of missing pieces.  The
>      NHI driver doesn't know how to probe a USB4 controller yet, but
>      that should be easy to fix.  Even more important, though, is that
>      the code lacks a functional USB4 Connection Manager.  Most of the
>      pieces required to traverse the topology, discover routers and
>      adapters, read and write their properties, and build routes between
>      endpoints exists now, but there's no state machine yet that
>      integrates those pieces together into a real Connection Manager.
>      Without that, no attached peripherals will actually run.  The TBT3
>      controllers like AlpineRidge and IceLake have a connection manager
>      in firmware, so that's why those controllers function even with
>      minimal host OS support.  This isn't especially hard code to write,
>      but it's missing nonetheless.
>
>      Once the connection manager is written, it'll need to configure
>      connections with the USB3, PCIe, and DisplayPort devices that
>      operate over tunnels, and it'll need a cross domain handler for
>      connecting to another host.  USB3 tunnel support might require
>      significant changes in the USB3 stack in order to work with USB4.
>
>      Additionally you might need to write a USB-PD driver.  Without it,
>      negotiation on USB-C connectors for power delivery advertisements,
>      cable orientation, alt mode configuration, and USB3 vs USB4 lane
>      assignment might not work.  If those negotiations are not handled
>      then nothing that you plug into the port will even be seen by the
>      controller.
>
>      I'd totally stay away from spending time on supporting Falcon
>      Ridge, Alpine Ridge, and Titan Ridge controllers.  They're old,
>      they're extremely difficult to work with, and they're not worth the
>      headache.  In fact, just ignore all TBT3 controllers, and remove
>      the Internal Connection Manager code.  The ICM module isn't code
>      that I'm all that proud of anyways =-). Focus on writing an HCM,
>      supporting PCIe and DP tunneling, and integrating IOMMU protections
>      into both the NHI driver and the PCIe tunnel drivers.
>
>      One thing that would be pretty awesome is ThunderboltIP support.
>      It turns out that these controllers are really cheap 40Gbps
>      devices, and have the potential to perform pretty well at line rate
>      as a replacement for traditional 40Gb ethernet controllers, at
>      least in a point-to-point configuration.
>
> In short, a lot of work has been done, but a lot of work remains to be
> done.  The USB4 spec is complicated, and requires intimate knowledge
> of the USB-PD, USB-C, and USB3 specs.  There's also a spec for
> writing a Host Connection Manager that you’ll need to get familiar with.
>
> If anyone has already started or would like to continue Scott's work,
> could you please let me know?
>
> Joe
>


-- 
Thanks
- Mohammad Noureldin
--
"Life is like riding a bicycle. To keep your balance you must keep moving"
- Albert Einstein