Ethernet Switch Framework, the other one

Fri Jan 20 22:36:03 UTC 2012

If you have followed a number of threads on -embedded, -mips, and -arch, you'll be aware that I've been busy attempting to come up with a switch control interface of my own.  The goal is to combine what Aleksandr has done over the past couple of months with some of what I have come up with.

The reason I did start from scratch were some architectural concerns that I had with Aleksandr's code.  Instead of idle talk, I decided to do a proof of concept implementation, and lo and behold, I did run into some interesting problems.  I'll reply to Aleksandr's post momentarily, but first I wanted to get my code out for people to review, and use the chance try and explain the problem space and why I decided to approach it the way I did.

The attached patches contain:
- etherswitchcfg.patch the userland utility
- miiproxy.patch a device that can act as a proxy between an ethernet driver and miibus
- etherswitch-complete: the above plus two switch drivers, a new kernel config and related files.

All that can also be obtained from http://gitorious.org/~stb/freebsd/stb-adrianchadd-freebsd-work in the branch work/ath.

The target systems for these switch device drivers are typical WLAN routers, which usually contain a one-chip switch controller.  All of them support port-based as well as .1q VLANs, as well as many other interesting features.  OpenWrt has a framework for configuring aspects of these switch controllers, but the original firmware of these devices usually does not expose the settings for the user.

For my proof of concept, I've put together two drivers that know how to configure an RTL8366RB and AR8x16 switches.  The RTL8366RB is configured through I2C.  The system's ethernet interface is connected directly to a switch port (back-to-back MAC connection via GMII).  The Atheros family of switch controllers is configured through an MDIO bus attachment; the switch controller uses the frame format, but reassigns phy address bits as register address bits.

Both switch drivers implement the etherswitch_if.m interface, and act as busses and automatically attach matching drivers as children.  I've written a simple cdev driver that allows the command line utility to open a control device and submit ioctls that are translated into etherswitch_if methods.

newbus attachments look like this (arrow indicates parent attachment with interface implemented)

              iicbus_if    etherswitch_if
... <--- iicbus <--- rtl833rb <--- etherswitch

        mdio_if   mdio_if    etherswitch_if
argemdio <--- mdio <--- arswitch <--- etherswitch

All switch controller chips contain PHYs (older models have external PHYs).  These are controlled in the usual manner via MDIO.  The MDIO master is in the switch controller.  Both switch drivers export the miibus_if.m methods readreg and writereg, and attach an miibus for each one of the switch ports that have a PHY.  This allows the command line utility to submit ifmedia ioctls to query and configure the individual ports.

I've gone through a number of iterations to make sure that the interdependencies of the drivers are as limited as possible, and that attachments happen automatically, without code nor hints, whereever possible.

MII Proxy

In some of the systems, one system ethernet interface is connected to a PHY inside the switch controller; the PHY acts completely independent from the switch.  However, the PHYs MDIO slave is connected not to the ethernet controllers MDIO master, but to the switch MDIO master.  This means solving two problems: allowing a driver control both an ethernet driver and use another MDIO bus driver to communicate with the PHY at the same time.

The miibus code unfortunately is extremely tailored to the one use case that was present so far; I'll go into more detail about what was stopping me from just using miibus directly in a separate post.  To leave miibus unmodified while maintaining functionality and not reimplementing miibus or the phy drivers, I've decided to create a proxy driver.  This proxy driver solves two issues: be an appropriate parent for miibus, and provide the capability to have a reference on a different device (and call methods there) than the parent.  A new bus driver, mdio, provides the MDIO readreg and writereg methods separate from the miibus driver, thus allowing automatic attachment of parent and children.

For a TL-MR3420 router, devinfo shows this device graph (filtered for interface and switch components):
nexus0
  arge0
    miiproxy0
      miibus4
        ukphy4
  arge1
  argemdio0
    mdio0
      arswitch0
        miibus0
          ukphy0
        miibus1
          ukphy1
        miibus2
          ukphy2
        miibus3
          ukphy3
        etherswitch0
        mdio1
          mdioproxy1
      mdioproxy0

arge(4) has been extended with a separate mdio host driver that has a new mdio bus driver attached.  The Atheros switch driver arswitch0 is attached to that bus; the four switch ports with PHYs on them show up as miibus0 to miibus3 and phy0 to phy3, respectively.  arge0, which has the MII connection (but not the MDIO connection) to PHY for in the switch controller, has miiproxy0 attached to it, which in turn hosts miibus4 and phy4.  miiproxy0 has a hidden connection to mdioproxy1 that was established through a hint on arge0; miiproxy0 uses mdioproxy1's parent to issue readreg and writereg calls on behalf of miibus.  Note mdioproxy0 above which was automatically attached (but is not used).

The only hint required to plug this together is
hint.arge.0.mdio=mdioproxy1

The simple and straightforward attachment follows the newbus design, and allows additional drivers to be automatically attached by newbus.  The etherswitch driver provides one possible consumer of the etherswitch_if API, but other are possible; for example, most switches allow .1d control frames to be punted to the CPU port, so our STP code could be used to construct a driver that implements spanning tree for a switch.

Stefan

-- 
Stefan Bethke <stb at lassitu.de>   Fon +49 151 14070811

-------------- next part --------------
A non-text attachment was scrubbed...
Name: etherswitchcfg.patch
Type: application/octet-stream
Size: 40486 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20120120/8b022494/etherswitchcfg-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: miiproxy.patch
Type: application/octet-stream
Size: 34705 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20120120/8b022494/miiproxy-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: etherswitch-complete.patch
Type: application/octet-stream
Size: 170338 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-net/attachments/20120120/8b022494/etherswitch-complete-0001.obj