svn commit: r307274 - stable/10/cddl/contrib/opensolaris/cmd/ztest

[Alexander Motin]

Author: mav
Date: Fri Oct 14 07:22:44 2016
New Revision: 307274
URL: https://svnweb.freebsd.org/changeset/base/307274

Log:
  MFC r305327: MFV r303080: 6451 ztest fails due to checksum errors
  
  illumos/illumos-gate at f9eb9fdf196b6ed476e4ffc69cecd8b0da3cb7e7
  https://github.com/illumos/illumos-gate/commit/f9eb9fdf196b6ed476e4ffc69cecd8b0d
  a3cb7e7
  
  https://www.illumos.org/issues/6451
    Sometimes ztest fails because zdb detects checksum errors. e.g.:
    Traversing all blocks to verify checksums and verify nothing leaked ...
    zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 8000160> DVA0=<0:1cc2000:
    180000> [L0 other uint64[]] sha256 uncompressed LE contiguou
    s unique single size=100000L/100000P birth=271L/271P fill=1
    cksum=c5a3e27d1ed0f894:843bca3a5473c4bf:f76a19b6830a2e4:91292591613a12bf --
    skipping
    zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 800000180> DVA0=<0:ce16800:
    180000> [L0 other uint64[]] sha256 uncompressed LE contigu
    ous unique single size=100000L/100000P birth=840L/840P fill=1
    cksum=5d018f3d061e17f3:6d1584784587bf63:2805a74a0ce37369:ba68a214806c7e75
    -- skipping
    zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 1000000360> DVA0=<0:10d37400:
    180000> [L0 other uint64[]] sha256 uncompressed LE conti
    guous unique single size=100000L/100000P birth=904L/904P fill=1
    cksum=fa1e11d4138bd14b:86c9488c444473e3:f31e43c72e72e46b:e3446472d1174d
    ba -- skipping
    zdb_blkptr_cb: Got error 50 reading <71, 47, 0, 400000002c0> DVA0=<0:127ef400:
    180000> [L0 other uint64[]] sha256 uncompressed LE cont
    iguous dedup single size=100000L/100000P birth=549L/549P fill=1
    cksum=30e14955ebf13522:66dc2ff8067e6810:4607e750abb9d3b3:6582b8af909fcb
    58 -- skipping
    zdb_blkptr_cb: Got error 50 reading <657, 5, 0, 1c0> DVA0=<0:1a180400:180000>
    [L0 other uint64[]] fletcher4 uncompressed LE contiguou
    s unique single size=100000L/100000P birth=1091L/1091P fill=1 cksum=a6cf1e50:
    29b3bd01c57e5:36779b914035db9a:db61cdcf6bec56f0 -- skippin
    g
    The problem is that ztest_fault_inject() can inject multiple faults into the
    same block. It is designed such that it can inject errors on all leafs of a
    RAID-Z or mirror, but for a given range of offsets, it will only inject errors
  
  Reviewed by: George Wilson <george.wilson at delphix.com>
  Reviewed by: Prakash Surya <prakash.surya at delphix.com>
  Reviewed by: Jorgen Lundman <lundman at lundman.net>
  Approved by: Dan McDonald <danmcd at omniti.com>
  Author: Matthew Ahrens <mahrens at delphix.com>

Modified:
  stable/10/cddl/contrib/opensolaris/cmd/ztest/ztest.c
Directory Properties:
  stable/10/   (props changed)

Modified: stable/10/cddl/contrib/opensolaris/cmd/ztest/ztest.c
==============================================================================
--- stable/10/cddl/contrib/opensolaris/cmd/ztest/ztest.c	Fri Oct 14 07:21:55 2016	(r307273)
+++ stable/10/cddl/contrib/opensolaris/cmd/ztest/ztest.c	Fri Oct 14 07:22:44 2016	(r307274)
@@ -4793,7 +4793,7 @@ ztest_fault_inject(ztest_ds_t *zd, uint6
 	char path0[MAXPATHLEN];
 	char pathrand[MAXPATHLEN];
 	size_t fsize;
-	int bshift = SPA_OLD_MAXBLOCKSHIFT + 2;	/* don't scrog all labels */
+	int bshift = SPA_MAXBLOCKSHIFT + 2;	/* don't scrog all labels */
 	int iters = 1000;
 	int maxfaults;
 	int mirror_save;
@@ -4954,6 +4954,31 @@ ztest_fault_inject(ztest_ds_t *zd, uint6
 	fsize = lseek(fd, 0, SEEK_END);
 
 	while (--iters != 0) {
+		/*
+		 * The offset must be chosen carefully to ensure that
+		 * we do not inject a given logical block with errors
+		 * on two different leaf devices, because ZFS can not
+		 * tolerate that (if maxfaults==1).
+		 *
+		 * We divide each leaf into chunks of size
+		 * (# leaves * SPA_MAXBLOCKSIZE * 4).  Within each chunk
+		 * there is a series of ranges to which we can inject errors.
+		 * Each range can accept errors on only a single leaf vdev.
+		 * The error injection ranges are separated by ranges
+		 * which we will not inject errors on any device (DMZs).
+		 * Each DMZ must be large enough such that a single block
+		 * can not straddle it, so that a single block can not be
+		 * a target in two different injection ranges (on different
+		 * leaf vdevs).
+		 *
+		 * For example, with 3 leaves, each chunk looks like:
+		 *    0 to  32M: injection range for leaf 0
+		 *  32M to  64M: DMZ - no injection allowed
+		 *  64M to  96M: injection range for leaf 1
+		 *  96M to 128M: DMZ - no injection allowed
+		 * 128M to 160M: injection range for leaf 2
+		 * 160M to 192M: DMZ - no injection allowed
+		 */
 		offset = ztest_random(fsize / (leaves << bshift)) *
 		    (leaves << bshift) + (leaf << bshift) +
 		    (ztest_random(1ULL << (bshift - 1)) & -8ULL);