FreeBSD has serious problems with focus, longevity, and lifecycle

[Robert Watson]

On Wed, 18 Jan 2012, Andriy Gapon wrote:

> on 18/01/2012 02:16 Igor Mozolevsky said the following:
>> Seriously, WTF is the point of having a PR system that allows patches to be 
>> submitted??! When I submit a patch I fix *your* code (not yours personally, 
>> but you get my gist).
>
> Let me pretend that I don't get it.  It is as much your code as it is mine 
> if you are a user of FreeBSD.  I just happen to have a commit bit at this 
> point in time.
>
>> No other project requires a non-committer to be so ridiculously persistent 
>> in order to get a patch through.
>
> There are about 5000 open PRs for FreeBSD base system, maybe more. There are 
> only a few dozens of active FreeBSD developers.  Maybe less for any given 
> particular point in time (as opposed to a period of time). And dealing with 
> PRs is not always exciting. Need I continue?
>
> P.S. Using GNATS for the PR database doesn't help either, in some technical 
> ways.

The structural problem around the PR system for the base system is that there 
isn't a whole lot of incentive for most developers to use it.  I think we can 
reasonably categorise developers into three classes -- some move between or 
span them, of course:

(1) Volunteers.  Due to childhood trauma, they have a desperate urge to write
     operating systems.  Not much incentive to do PRs here, as most refer to
     versions of FreeBSD before their time, aren't great characterisations,
     rarely come with patches, and when they do, the patches are out of date,
     don't apply, have the wrong style, solve the wrong problem, etc.  A
     sweeping generalisation, but you see what I mean.  The only exceptions
     here are our dedicated team of bugmeisters, who get enourmous respect from
     me, but they are a tiny minority.

(2) Employees.  They work at a company using FreeBSD as a product, and
     effectively deliver their own CompanyBSD as a further product to their own
     internal customers -- to be put on a web service frontline, to ship as the
     foundation of an appliance, etc.  The key phrase here is "internal
     customers" -- they have their own bug report database, which they respond
     to in a timely way due to the incentives of the workplace, but also
     because they are relevant bug reports for their product goals.

(3) Authors of upstream code.  They don't even work on FreeBSD, but their code
     ends up in FreeBSD, so they also have their own bug report databases, fix
     bugs, and eventually the fixes trickle into FreeBSD.

With the above, the incentives to handle PRs are very weak -- and it's 
compounded by gnats being terrible for both submitters and handlers of bug 
reports.  Contrast this with ports, where the PR database is a key part of the 
workflow.

However, and I am being entirely honest when I say this: FreeBSD works anyway. 
So somehow, we end up with a pretty good OS despite largely ignoring our bug 
report database.  Why?  Well, for (1) it's because volunteers have a strong 
sense of ownership of the code they've written and care about, (2) there's a 
significant internal QA and bug management effort at downstream companies from 
FreeBSD, whose improvements are frequently upstreamed by committers on staff, 
and (3) occurs independently of bugs in our bug report database.

Don't get me wrong: it's a problem that the PR database goes so unloved.  But 
it's a symptom of the construction of *extremely large* volunteer projects in 
which the incentives are not aligned for dealing with PRs most of the time. 
If you want to see something similarly sad, try counting dropped patches on 
the linux-kernel list.  Someone once ported the entire FreeBSD kernel audit 
framework and OpenBSM to Linux, posted on the list saying "here are my 
patches", never heard anything back, and went away.  You can moralise in 
various ways and for various parties in that relationship, but at heart, 
that's pretty similar to a lot of the patches in the PR database; you'll find 
similar stuff in every open source project of scale.  I submitted patches to 
fix several bugs in KDE a decade or so ago .. after five years, the reports 
were closed as "out of date".  Yet large open source products *do* work, and 
become the foundations for amazing things.

I think shifting away from Gnats would help as it would make it easier for 
developers to find bugs they care about, users to submit higher-quality 
reports, and so on.  Gnats makes it really hard to manage reports in a useful 
way.

Another possibility is to get some combination of {The FreeBSD Foundation, iX 
Systems, ...} to trawl the bug report database in a more official capacity. 
The problem there is that this will be a high burn-out job.  I'll bring it up 
at the next Foundation board meeting, especially after a bumper year of 
fund-raising, and see what we can do.

Robert