12.1-RELEASE-p7 panic in zio_free_issue_4_6

[Christian Kratzer]

Hi,

On Thu, 29 Oct 2020, Andriy Gapon wrote:
>> I will keep this pool around for a couple of days and will try to get a crash dump
>> from the system.  After that I will have delete and recreate the pool and just
>> wait for backups to roll back in.
>
>
> Okay, let's see if we can get a vmcore.
> Otherwise, this is just a guess-work on my part.
> The problem could be very different from my initial impression.

so I added a swap device which for some reason was missing and was about to induce
the crash again.

But in order to get consistent versions of the kernel debug symbols I upgraded the system
to 12.2-RELEASE.

Now that everything was in place I was able to import the pool in readonly mode using


  zpool import -m o -readonly=on zpfra2

without the system crashing. It sits there happily now

   pool: zpfra2
  state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
         the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
    see: http://illumos.org/msg/ZFS-8000-2Q
   scan: scrub repaired 0 in 0 days 01:31:43 with 0 errors on Fri Jul 17 18:55:35 2020
config:

         NAME                     STATE     READ WRITE CKSUM
         zpfra2                   DEGRADED     0     0     0
           raidz2-0               ONLINE       0     0     0
             gpt/zfsfra1d02.eli   ONLINE       0     0     0
             gpt/zfsfra1d03.eli   ONLINE       0     0     0
             gpt/zfsfra1d04.eli   ONLINE       0     0     0
             gpt/zfsfra1d05.eli   ONLINE       0     0     0
             gpt/zfsfra1d06.eli   ONLINE       0     0     0
             gpt/zfsfra1d07.eli   ONLINE       0     0     0
             gpt/zfsfra1d08.eli   ONLINE       0     0     0
           raidz2-1               ONLINE       0     0     0
             gpt/zfsfra1d10.eli   ONLINE       0     0     0
             gpt/zfsfra1d11.eli   ONLINE       0     0     0
             gpt/zfsfra1d12.eli   ONLINE       0     0     0
             gpt/zfsfra1d13.eli   ONLINE       0     0     0
             gpt/zfsfra1d14.eli   ONLINE       0     0     0
             gpt/zfsfra1d15.eli   ONLINE       0     0     0
             gpt/zfsfra1d16.eli   ONLINE       0     0     0
         logs
           mirror-2               UNAVAIL      0     0     0
             3980362133776709100  UNAVAIL      0     0     0  was /dev/gpt/log1d0
             6670731949941654186  UNAVAIL      0     0     0  was /dev/gpt/log1d1


The pool is degraded as I had already removed the log devices.

I am pulling data of the pool as we speak and will recreate it.

In case this happens again I will be prepared with a crash dump.

I will also not enable dedup for this 15TB pool as long as the machine has only 128GB ram.

Greetings
Christian

-- 
Christian Kratzer                   CK Software GmbH
Email:   ck at cksoft.de               Wildberger Weg 24/2
Phone:   +49 7032 893 997 - 0       D-71126 Gaeufelden
Fax:     +49 7032 893 997 - 9       HRB 245288, Amtsgericht Stuttgart
Mobile:  +49 171 1947 843           Geschaeftsfuehrer: Christian Kratzer
Web:     http://www.cksoft.de/