dtrace: normalization of stddev

[Andriy Gapon]

It seems that currently normalization of stddev aggregation is done incorrectly.
We divide both the sum of values and the sum of their squares by the
normalization factor.  But we should divide the sum of squares by the
normalization factor squared to scale the original values properly.

--- lib/libdtrace/common/dt_consume.c
+++ lib/libdtrace/common/dt_consume.c
@@ -389,8 +389,10 @@ dt_stddev(uint64_t *data, uint64_t normal)
 	 * The standard approximation for standard deviation is
 	 * sqrt(average(x**2) - average(x)**2), i.e. the square root
 	 * of the average of the squares minus the square of the average.
+	 * When normalizing, we should divide the sum of x**2 by normal**2.
 	 */
 	dt_divide_128(data + 2, normal, avg_of_squares);
+	dt_divide_128(avg_of_squares, normal, avg_of_squares);
 	dt_divide_128(avg_of_squares, data[0], avg_of_squares);

 	norm_avg = (int64_t)data[1] / (int64_t)normal / (int64_t)data[0];


A primitive test script:

BEGIN
{
    i = 100;
    @s = avg(i);
    @v = stddev(i);

    i = 200;
    @s = avg(i);
    @v = stddev(i);

    i = 300;
    @s = avg(i);
    @v = stddev(i);

    i = 400;
    @s = avg(i);
    @v = stddev(i);

    i = 500;
    @s = avg(i);
    @v = stddev(i);

    i = 600;
    @s = avg(i);
    @v = stddev(i);

    i = 700;
    @s = avg(i);
    @v = stddev(i);

    i = 800;
    @s = avg(i);
    @v = stddev(i);

    i = 900;
    @s = avg(i);
    @v = stddev(i);

    printa("%@3d %@3d\n", @s, @v);

    normalize(@s, 10);
    normalize(@v, 10);
    printa("%@3d %@3d\n", @s, @v);

    exit(0);

}

Without the patch it produces:
500 258
 50 170

With the patch:
500 258
 50  25


-- 
Andriy Gapon