Kwalitee

[Lowell Gilbert]

joseph.koshy at gmail.com (Joseph Koshy) writes:

> > I'm wondering if we can adopt this idea for the FDP.  Can we come up
> 
> I'm wary of metrics as I've seen way too many organizations optimize for the
> metric instead of focussing on the real thing.

Nik covered this pretty well in his introduction.  
Since you can't measure the real thing, you need to come up with
metrics where naive attempts to improve the metric value would also
improve the system.

You can't (systematically) improve anything you can't measure.  As
long as your metric and its first derivative are positively correlated
with the results you want, the metric can be useful.

> For users:
> 
> Metric:       Number of undocumented programs
> Rationale:  Every program should have a manual page.  whatis(1) should
>                   "just work".

Well, maybe.  Are there many of these that are actually useful to users?

> And for programmers trying to use FreeBSD, the following:
> 
> Metric:       Number of undocumented APIs
> Rationale:  Every visible symbol in our libraries should have a description
>                   in a manual page.

That's just silly.  Functions required by various standards should be
documented, and anything of wide use should be documented, but for any
function that was intended to be used only by a small number of
closely linked modules, forcing people to Read The Source before using
the function is a *Good* Thing.

> Metric:       Number of undocumented kernel APIs
> Rationale:  Same as above.

An appropriate metric might be "number of undocumented kernel APIs
whose use has been recommended on the freebsd-questions list this
week."  But if the suggestion is that documenting the
"debug.snapdebug" sysctl would improve FreeBSD, then I think that's
silly too.