Kwalitee

[Nik Clayton]

Thinking out loud.

One of the things we strive for is to improve the quality of our
documentation.

"quality", in this case, is quite an abstract metric.  It's hard to
measure much beyond "I'll know it when I see it".

A segment of the Perl community has adopted the term "kwality" (yes, I
know, that's the Perl sense of humour kicking in) to describe those
aspects that relate to "quality" that can be accurately measured.  The
idea being that although "kwalitee" doesn't measure "quality", a high
kwality shows a correlation with high quality.

I'm wondering if we can adopt this idea for the FDP.  Can we come up
with a series of objective metrics that let us quickly point out any
areas where the documentation might do with improvement, and mechanisms
for testing those metrics?

This would give us some 'low hanging fruit' for interested contributors
to investigate and, if necessary, fix.

So to kick off, some metrics we might use, and the rationale behind
them.  It should be possible to mechanically check each metric.

Metric:     Date of last commit > 6 months ago
Rationale:  Some documents are write-once, or change very infrequently.
            But I'd venture to suggest that the majority of them should 
	    be updated reasonably often.  A document that's not been 
	    touched in 6 months merits investigation.  Maybe it doesn't
	    need to be changed, or maybe it's slowly becoming
	    irrelevant, and needs updating or removing?

Metric:     Any spelling errors
Rationale:  Should be obvious.

Metric:     Fewer than 10% of the document is <indexterms>
Rationale:  For DocBook docs only.  And I don't know if 10% is too high
            or too low.  A document/chapter that doesn't have many 
	    indexterms in it has probably not been indexed properly.

Metric:	    Translated version is > 4 weeks behind English version
Rationale:  I have no idea how the various translation teams decide what
            order to translate things in, or which updates merit
	    attention the fastest.  But this might help point out
	    translated docs that need attention.

Metric:     Chapters with no Synopsis section
Rationale:  We've got a standard structure for chapters in most of our
            documentation, does this document follow it?

Metric:     More than x,000 words in a <chapter>
Rationale:  I don't know what an appropriate number for 'x' is, but this
            would point out a chapter that might benefit from being
	    split out in to sub-chapters.

Metric:     <sect1>s with unbalanced number of words
Rationale:  Apart from the Synopsis I suspect (and as yet have no hard 
            data to back this up) that each <sect1> in a chapter should,
	    assuming the content's been structured appropriately, have a
	    roughly similar number of words.  If they don't the content
	    may be unbalanced.

	    This might highlight chapters that go in to a lot of detail
	    about one aspect of something, but have barely touched upon
	    others.

Thoughts?  What other metrics can people come up with?  Remember, it
needs to be possible to objectively test them.

N
-- 
FreeBSD: The Power to Serve      http://www.freebsd.org/               (__)
FreeBSD Documentation Project    http://www.freebsd.org/docproj/    \\\'',)
                                                                      \/  \ ^
   --- 15B8 3FFC DDB4 34B0 AA5F  94B7 93A8 0764 2C37 E375 ---         .\._/_)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-doc/attachments/20041211/a1eada5c/attachment.sig>