From nobody Fri Jun 17 18:33:52 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1504183B023 for ; Fri, 17 Jun 2022 18:34:05 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: from mail-yb1-xb2a.google.com (mail-yb1-xb2a.google.com [IPv6:2607:f8b0:4864:20::b2a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LPnkM74F6z4pjh for ; Fri, 17 Jun 2022 18:34:03 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: by mail-yb1-xb2a.google.com with SMTP id l11so8575606ybu.13 for ; Fri, 17 Jun 2022 11:34:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=v8z7sVUrPzxZDpNrj/TRNof9XqQn3Je8Ik3l+8VYy4g=; b=qSiweYoXuB3ZXlWfety9soaYqlbO0XPnu3Ynnxkx7atr82TJdktbdjELNF5Qmwwpd3 60mPEncjYI+0yFeqqs8WUOT48sF+D7HP2ZN83wEiR5t5WK31WRSwLTl8G78pGeIb2Gr3 LF9hXMNieWkrdf+JCeyHCorZjwChav1PTud7W/IBdUpB9RMj4/Jgi6MZa+KfAp/qdpK4 Kn2fTBjA/2ZY8j9JKCUwd7dMGz2k8eLRaiTM7xxB2Z22/TCdw7RPTYwvcdLxz/o3ylnv 5vAvEM0JcEkSrJCWH+fpNhLp4IDTrHAkyWh8d6vd4qEZRV2//OJRUZuhtiObDL/Ce7lu hGmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=v8z7sVUrPzxZDpNrj/TRNof9XqQn3Je8Ik3l+8VYy4g=; b=ga1YLYqMuNjxkvki7tiw3ARPbvqRILcMQvtVJBN+Hm/x4AIdBoP9Tf/m0DOiM5VtKz oQG+3t0UeRTbTED0vQZLQnaYXa+8EdfC/FKLmi3083HiVZ888XCAgP3WdH9FZnvt06rM sFydxa5mqkFQ3BsdHbcIzvwzQrEwd1Jys52aHRkQjbJDQrRWhnZrq1/tJRfQyAKfiXSn EtS0a6mFGJjIkP6j6dRH7Epm9eCXT1DytmmUYvhfxIYgFmhsVIlJexSo9zzw9vI0rplK 3Mkz6gJZsSvOH8uzoC6HIP8mQH6y//ljDght4c6cZheF8J/89SN26hwsk3uNtJXWJSTL 6Fww== X-Gm-Message-State: AJIora+DkCNYVyuW+0cA6XAddAuyhAg3nrdZh9f+jUyOqgPjSoI9XGOH sPJsxrXbC5FdIu7i9MBAxlkTBsUlPX1pjRc0tdMYvnOYdNUIiA== X-Google-Smtp-Source: AGRyM1sUAT8CPTxsPdGaSz7dZF9I3asyiKZYUKWb8FC19e2Tx3OOglWRBpbCUbSmlYxnTY9t6Hg9MyeeZNLSYqU9l7Q= X-Received: by 2002:a25:c50d:0:b0:65c:b34b:287 with SMTP id v13-20020a25c50d000000b0065cb34b0287mr12885579ybe.334.1655490843256; Fri, 17 Jun 2022 11:34:03 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Eric Borisch Date: Fri, 17 Jun 2022 13:33:52 -0500 Message-ID: Subject: Re: zpool/zfs-commands hang; ZFS filesystem/ZVOLs still running. To: freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4LPnkM74F6z4pjh X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=qSiweYoX; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of eborisch@alumni.stanford.edu designates 2607:f8b0:4864:20::b2a as permitted sender) smtp.mailfrom=eborisch@alumni.stanford.edu X-Spamd-Result: default: False [-1.78 / 15.00]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; NEURAL_HAM_MEDIUM(-0.94)[-0.939]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-0.70)[-0.703]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b2a:from]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MLMMJ_DEST(0.00)[freebsd-fs]; FORGED_SENDER(0.30)[eborisch@gmail.com,eborisch@alumni.stanford.edu]; MIME_TRACE(0.00)[0:+]; NEURAL_SPAM_SHORT(0.56)[0.558]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[eborisch@gmail.com,eborisch@alumni.stanford.edu]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 17, 2022 at 10:46 AM Eric Borisch wrote: > > Good morning, > > I have a 13.1 system where twice now (required hard restart last time > to resolve) since upgrading from 12.x I have had zfs/zpool-command > operations start to hang indefinitely, while the underlying pools are > still actively running/processing data via both filesystem and device > interfaces. I went back to some data I captured during the first instance, and it also had a zfs destroy and zfs send in perpetual D-state: 11076 - D 0:00.01 zfs destroy -d kspace/users/eborisch@zfs-auto-snap_frequent-2022-06-13-08h15 16564 - D 0:22.82 /sbin/zfs send -L -RI @MARK-mr-dewey kspace@NEW-mr-dewey 11076 190929 zfs - mi_switch _sleep rms_wlock zfs_umount dounmount zfsctl_snapshot_unmount zfs_ioc_destroy_snaps zfsdev_ioctl_common zfsdev_ioctl devfs_ioctl vn_ioctl devfs_ioctl_f kern_ioctl sys_ioctl amd64_syscall fast_syscall_common 16564 190128 zfs - mi_switch _sleep rms_wlock zfs_umount dounmount zfsctl_snapshot_unmount dsl_dataset_user_release_impl dsl_dataset_user_release_onexit zfs_onexit_destroy zfsdev_close devfs_fpdrop devfs_close_f _fdrop closef closefp amd64_syscall fast_syscall_common Note the particular @MARK- and @NEW- snapshots are different now, as they are constantly destroyed/renamed during successful backup cycles, which have occurred between these two events. Thanks, - Eric