svn commit: r318783 - stable/11/cddl/contrib/opensolaris/lib/libdtrace/common

[Andriy Gapon]

Author: avg
Date: Wed May 24 11:10:55 2017
New Revision: 318783
URL: https://svnweb.freebsd.org/changeset/base/318783

Log:
  MFC r316853: dtrace: fix normalization of stddev aggregation
  
  Sponsored by:	Panzura

Modified:
  stable/11/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c
Directory Properties:
  stable/11/   (props changed)

Modified: stable/11/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c
==============================================================================
--- stable/11/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c	Wed May 24 11:09:06 2017	(r318782)
+++ stable/11/cddl/contrib/opensolaris/lib/libdtrace/common/dt_consume.c	Wed May 24 11:10:55 2017	(r318783)
@@ -389,8 +389,10 @@ dt_stddev(uint64_t *data, uint64_t norma
 	 * The standard approximation for standard deviation is
 	 * sqrt(average(x**2) - average(x)**2), i.e. the square root
 	 * of the average of the squares minus the square of the average.
+	 * When normalizing, we should divide the sum of x**2 by normal**2.
 	 */
 	dt_divide_128(data + 2, normal, avg_of_squares);
+	dt_divide_128(avg_of_squares, normal, avg_of_squares);
 	dt_divide_128(avg_of_squares, data[0], avg_of_squares);
 
 	norm_avg = (int64_t)data[1] / (int64_t)normal / (int64_t)data[0];