Re: USB4/TBT3 support

On Fri, 20 Dec 2024 17:04:20 +0100
Mohammad Noureldin <mohammad@thelightbird.com> wrote:

> Hi Joe,
> 
> Sorry for taking a long while responding to your email.
> 
> Between some changes on the personal level, really bad luck securing the
> main TBT3/USB4 hosts and saving up to secure a proper set of HW for the
> task, it took me that long while to come back on the beginning of the track.
> 
> I've compiled in [1]:
> - All the details you've shared in your email message which includes the
> notes from scottl@
> - The notes left behind by hps@ (may he RIP)
> - What I have in mind of how to pick up the work from here and going FWD
> 
> Any comments and/or feedback or even shouting is more than welcome
> 
> And for the rest, I wish. you and all of the FreeBSD community happy and
> peaceful holidays
> 
> [1] https://wiki.freebsd.org/MohammadNoureldin/FreeBSDUSB4TBT3Support
> 
> On Fri, Jul 19, 2024 at 9:38 PM Joseph Mingrone <jrm@freebsd.org> wrote:
> 
> > Hello,
> >
> > Is anyone working on USB4 / Thunderbolt 3 (TBT3) support?
> >
> > Scott Long did a lot of work on this a few years ago, but he had to move
> > on to other things, so he passed things on to Hans Petter Selasky.
> > Fortunately, Hans Petter dropped the code in a public repository.
> >
> > https://github.com/hselasky/usb4
> >
> > https://github.com/hselasky/usb4/commit/dd85c216a2a6bee5361c7166595ba6ca461578b5
> >
> > Here is an overview of what Scott shared with me.
> >
> > Mostly completed work:
> >
> > - Debug/Trace framework
> > - NHI controller driver
> > - PCIe bridge driver
> > - WMI driver
> > - Integrated Connection Manager handshake and authentication handling
> > - Router and Config Space layer handling (in progress, almost complete)
> >
> > Remaining work:
> >
> > - tbtconfig (userland tool)
> > - man pages
> > - DMAR/IOMMU integration, PCIe tunnelling control
> > - Support for resetting and firmware flashing on the NHI via out-of-band
> > control
> > - Host Connection Manager
> > - Cross-domain login
> > - ThunderboltIP
> >
> > Here are the details that Scott shared.
> >
> >      The driver originally targeted the Thunderbolt 3 controllers that
> >      were sold under the names “AlpineRidge" and “Icelake”, in the late
> >      2010's, before the USB standards group publicly released the USB4
> >      spec.  The driver set I wrote was complete enough to activate
> >      Thunderbolt3 peripherals that otherwise would be disabled by
> >      default when plugged in.  The driver also attempted to make it
> >      easier to identify things like PCIe tunnels in the topology, but
> >      that was mostly cosmetic.  Unfortunately, the AlpineRidge chips
> >      proved to be extremely hard to work with despite their wide
> >      availability, and I spent way too much time fighting them and not
> >      enough time developing more useful functionality.  The WMI driver
> >      was written to work around vexing problems with the Alpine Ridge
> >      controller that I never figured out.
> >
> >      Much of the infrastructure from the TBT3 support extends to modern
> >      USB4 controllers, but there are still a lot of missing pieces.  The
> >      NHI driver doesn't know how to probe a USB4 controller yet, but
> >      that should be easy to fix.  Even more important, though, is that
> >      the code lacks a functional USB4 Connection Manager.  Most of the
> >      pieces required to traverse the topology, discover routers and
> >      adapters, read and write their properties, and build routes between
> >      endpoints exists now, but there's no state machine yet that
> >      integrates those pieces together into a real Connection Manager.
> >      Without that, no attached peripherals will actually run.  The TBT3
> >      controllers like AlpineRidge and IceLake have a connection manager
> >      in firmware, so that's why those controllers function even with
> >      minimal host OS support.  This isn't especially hard code to write,
> >      but it's missing nonetheless.
> >
> >      Once the connection manager is written, it'll need to configure
> >      connections with the USB3, PCIe, and DisplayPort devices that
> >      operate over tunnels, and it'll need a cross domain handler for
> >      connecting to another host.  USB3 tunnel support might require
> >      significant changes in the USB3 stack in order to work with USB4.
> >
> >      Additionally you might need to write a USB-PD driver.  Without it,
> >      negotiation on USB-C connectors for power delivery advertisements,
> >      cable orientation, alt mode configuration, and USB3 vs USB4 lane
> >      assignment might not work.  If those negotiations are not handled
> >      then nothing that you plug into the port will even be seen by the
> >      controller.
> >
> >      I'd totally stay away from spending time on supporting Falcon
> >      Ridge, Alpine Ridge, and Titan Ridge controllers.  They're old,
> >      they're extremely difficult to work with, and they're not worth the
> >      headache.  In fact, just ignore all TBT3 controllers, and remove
> >      the Internal Connection Manager code.  The ICM module isn't code
> >      that I'm all that proud of anyways =-). Focus on writing an HCM,
> >      supporting PCIe and DP tunneling, and integrating IOMMU protections
> >      into both the NHI driver and the PCIe tunnel drivers.
> >
> >      One thing that would be pretty awesome is ThunderboltIP support.
> >      It turns out that these controllers are really cheap 40Gbps
> >      devices, and have the potential to perform pretty well at line rate
> >      as a replacement for traditional 40Gb ethernet controllers, at
> >      least in a point-to-point configuration.
> >
> > In short, a lot of work has been done, but a lot of work remains to be
> > done.  The USB4 spec is complicated, and requires intimate knowledge
> > of the USB-PD, USB-C, and USB3 specs.  There's also a spec for
> > writing a Host Connection Manager that you’ll need to get familiar with.
> >
> > If anyone has already started or would like to continue Scott's work,
> > could you please let me know?
> >
> > Joe
> >
> 
> 
> -- 
> Thanks
> - Mohammad Noureldin
> --
> "Life is like riding a bicycle. To keep your balance you must keep moving"
> - Albert Einstein

Hi.
Thanks for the pointer.

Some notes:
  *hselasky@ (RIP, Hans...) suffered from the lack of official and open
   hardware specs for implementing his driver.

  *hselasky@ wasn't going to implement TB3's security features
   as it seemed NOT to be sufficient for actual security in spec itself.

  *I have Titan Ridge TB3 controller (falles back to Intel Cannon Lake
   USB 3.1 controller when TB3 device is not connected) in my ThinkPad
   P52, but hselasky@'s last driver didn't work for it, as his driver
   only supports older generation of controller chipset (with the lack
   of information).

  *P52 recognizes TB3-only device (Samsung X5 external SSD I've
   purchased for test) as internal PCIe drive if it was attached
   before powerint on the computer, but hot deplugging causes panic,
   with and without hselasky@'s driver installed.

  *My related info are in Bug 237666 [1].

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237666

-- 
Tomoaki AOKI    <junchoon@dec.sakura.ne.jp>