ETA on HPS USB stack into CURRENT?

[Julian Elischer]

Anish Mistry wrote:
> 	What requirements still need to be met before the HPS USB stack is 
> committed to CURRENT?  I understand that introducing a major 
> sub-system change so close the start of a release cycle can be very 
> disruptive.  My main concern is that the RELENG_7 branch will be 
> arriving in the next couple months and imp@ mentioned in the (Re: 
> UMASS pbm at startup) thread that RELENG_7 might not see the new 
> stack.  This means that we'll have to wait until an 8.x release 
> (2009?) until we see the new stack in a stock install.  Also the 
> current stack will then need to be supported during the entire life 
> of RELENG_7 which will probably into at least late 2010.
> 	From here on out the limitations of the current USB stack not being 
> MPSAFE will only become more apparent with more systems shipping with 
> multiple processors.  Such as not allow CPUs to drop to C3 when USB 
> is loaded, performance issues, and various HID parsing problems.
> 	With all that said what would be needed so that we could see the new 
> stack in 7.1 eventhough 7.0 would have the old stack?  Though this 
> does seem to be asking for a lot of trouble since it would be a 
> STABLE branch.

This is a question we've been discussing a lot. Your public question 
probably deserves a public answer.

There are some requirements that all subsystems require. HPS's USB code 
is no different from others.

I will borrow from a talk at BSDCan that talked about project dynamics..

1/ "Bus factor" of the project must not increase.
i.e. "number of developers that may be hit by a bus before 
the project is really screwed". (bigger is better)

Currently the "bus factor of the existing USB code is about 5 (busy) people
Bus factor of new code is 1

2/ The nuclear powerplant problem.
The direct opposite from a "bikeshed".
Everyone understands a bikeshed and they can all comment on it.
Very few people understand a nuclear powerplant so sometimes they can 
get installed with almost no review and comment. When they go wrong 
however they go wrong in a big way and influence a lot.
They tend to decrease the "bus factor" of a project. (not good).


What this means:
A large module needs comprehensive review by other developers.
The module needs to be split up into chunks that can be reviewed
by humans. Ether by implementing it as a sequence of patches to get from 
"Here" to "there" in understandable steps, or by heavily documenting it.
Preferably with lots of diagrams etc. (see the papers in the "docs" 
section for some of the examples of what people have done in the past)

The code needs to reviewed, which means that it needs to be well laid 
out and commented. I beieve HPS is currently going through his code 
commenting it. It's curently "under commented". He has also been asked to
provide some sort of design document to assist the reviewers.
To some extent this means that HPS must be willing to let the control of
his code slip from his hads somewhat.


None of this of course means that it will get in if the reviewers don't
like what they see, but if they do it comes in when all issues are addressed.

Unless a really superior competitor exists, which seems doubtful at this time.
The biggest competitor th HPS's USB code is "fix the old USB code".
he needs to demonstrate that his co de is superior to this option.

BTW after "first cut reviews" (done with great pain on the undocumented code),
current issues of concern (also somewhat present in the existing code) include
Locking issues and interaction with the newbus/device framework.

When the commented and documented version of the code become available
then it wil be reviewed more thoroughly  and more issues will be raised
undoubtedly. Currently the lack of documentation has hindered review.

Implementation details:

New code modules such as this should be installed with a transition period.
In other words, for a short while both new and old code should exist 
side-by-side in some way. This allows people who need teh functionality and
cannot spare the time to debug the new code, to keep working while others
can work on the new code. This has happenned severaltimes in the past, with 
#ifdefs etc. being used to compile a kernle with new or old versions of 
various modules (e.g. pcmcia code vs pccard/newcard, or fastforward 
vs. regular forwarding)

HPS is hoping to be able to present his code to developers attending EuroBSDcon
in some way, and has agreed to assist us in reviewing it
by adding comments and other documentation. (in some cases adding back 
commenting that already existed but he removed from his versions of the files).


Until now all the work on HPS's code has been on the technical side,
and none of it has been on the project integration side of things.
This often tends to be overlooked but if Hans-Peter can get that side of his 
act together He has a chance that it could be accepted well.
(assuming that reviewing results in a well integrated result.)

Anyhow this is the basic thrust of a long discussion that I and some other 
developers had at BSDCan..

Nothing very Surprising, but it does lay down a line which needs to be reached
for integration of that code, and several of us have been communicationg these 
requirements to Hans-Peter.

Finally, We REALLY do need good USB code, so we hope that
when we can review it fully when it is documented, we discover that it
is just wonderful, and that any issues that are raised by 
various domain experts (e.g. locking, interrupts, VM, device framework)
are all addressed quickly and everyone gets what they need.

Reality rarely lives up to that standard but that's what we hope :-)