From nobody Fri Nov 10 10:12:32 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRZQ24JWNz502cr for ; Fri, 10 Nov 2023 10:12:42 +0000 (UTC) (envelope-from SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRZQ13D4Xz3MTl for ; Fri, 10 Nov 2023 10:12:41 +0000 (UTC) (envelope-from SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=FuxE+Ipa; spf=pass (mx1.freebsd.org: domain of "SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl" designates 87.255.56.188 as permitted sender) smtp.mailfrom="SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Received: from rwvirtual374.colo.realworks.nl (rwvirtual374.colo.realworks.nl [10.0.10.74]) by mailrelayint2.colo2.realworks.nl (Postfix) with ESMTP id 4SRZPr54p8z1Z3 for ; Fri, 10 Nov 2023 11:12:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1699611152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=7GNW+a5f8V/KIpM37/vl0rJtUxqyPDjm/dTwTTW7eX4=; b=FuxE+Ipa6GP/0Vh0qvt3AsFtqZ79+Rl2jirO2OKQWCt4yHcbS7VesGO0aQmSIv8OdKh7uA hrdYy/d4F/kW6KoHrslHVAPDB1eU1V8ikNetco6C1JF+qM6q/WCWbzUyQ5ltwzdMF3Bn09 E2EjUnFIerfBZJ4eqnnzV2MXdBbPadrFbneN9v/BA2kJc6pjc3g6tUJWUF9njPgDjeI9wg 3FNyYA5TbXjnmIPu+oKrBIcqNLl9XkReG1OmjZQ4OP1bWFmY59tapmDGozj4wmCGm5er1+ vKYBRtXprTf3KPqjRb+YelcP5SgwcR9saBdEy1odA8oqGkHWIRXPYa1fyWE2xA== Received: from rwvirtual374.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual374.colo.realworks.nl (Postfix) with ESMTP id 8BE40C1188 for ; Fri, 10 Nov 2023 11:12:32 +0100 (CET) Date: Fri, 10 Nov 2023 11:12:32 +0100 (CET) From: Ronald Klop To: current@freebsd.org Message-ID: <349700057.3452.1699611152405@localhost> Subject: crash zfs_clone_range() List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_3451_1193281966.1699611152304" X-Mailer: Realworks (679.13) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual374 [10.0.10.74] with HTTP; Fri, 10 Nov 2023 11:12:32 +0100 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0 X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl]; R_SPF_ALLOW(-0.20)[+ip4:87.255.56.128/26]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[current@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL]; ARC_NA(0.00)[]; HAS_X_PRIO_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=ctA7=GX=klop.ws=ronald-lists@realworks.nl]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SRZQ13D4Xz3MTl X-Spamd-Bar: --- ------=_Part_3451_1193281966.1699611152304 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, Had this crash today on RPI4/15-CURRENT. FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 main-b0203aaa46-dirty: Sat Nov 4 11:48:33 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64 $ sysctl -a | grep bclon vfs.zfs.bclone_enabled: 1 I started a jail with poudriere to build a package. The jail uses null mounts over ZFS. [root]# cu -s 115200 -l /dev/cuaU0 Connected db> bt Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 db_trace_self() at db_trace_self db_stack_trace() at db_stack_trace+0x120 db_command() at db_command+0x2e4 db_command_loop() at db_command_loop+0x58 db_trap() at db_trap+0x100 kdb_trap() at kdb_trap+0x334 handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0xf2000000 kdb_enter() at kdb_enter+0x48 vpanic() at vpanic+0x1dc panic() at panic+0x48 data_abort() at data_abort+0x2fc handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0x96000004 rms_rlock() at rms_rlock+0x1c zfs_clone_range() at zfs_clone_range+0x68 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c null_bypass() at null_bypass+0x118 vn_copy_file_range() at vn_copy_file_range+0x18c kern_copy_file_range() at kern_copy_file_range+0x36c sys_copy_file_range() at sys_copy_file_range+0x8c do_el0_sync() at do_el0_sync+0x634 handle_el0_sync() at handle_el0_sync+0x48 --- exception, esr 0x56000000 Oh.. While typing this I rebooted the machine and it happened again. I didn't start anything in particular although the machine runs some jails. x0: 0x00000000000000e0 x1: 0xffffa00090317a48 x2: 0xffffa000f79d4f00 x3: 0xffffa000c61a44a8 x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) x5: 0xffffa001250e4c00 x6: 0xffff0000e54025b5 ($d.5 + 0xc) x7: 0x000000000000030a x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) x9: 0x0000000000000001 x10: 0x0000000000000000 x11: 0x0000000000000001 x12: 0x0000000000000002 x13: 0x0000000000000000 x14: 0x0000000000000001 x15: 0x0000000000000000 x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) x17: 0xffff0000004e0d44 (rms_rlock + 0x0) x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) x19: 0x0000000000000000 x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) x21: 0x7fffffffffffffff x22: 0xffffa00090317a48 x23: 0xffffa000f79d4f00 x24: 0xffffa001067ef910 x25: 0x00000000000000e0 x26: 0xffffa000158a8000 x27: 0x0000000000000000 x28: 0xffffa000158a8000 x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) sp: 0xffff0000deefe280 lr: 0xffff000001623564 (zfs_clone_range + 0x6c) elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) spsr: 0x00000000a0000045 far: 0x0000000000000108 esr: 0x0000000096000004 panic: data abort in critical section or under mutex cpuid = 1 time = 1699610885 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1a0 panic() at panic+0x48 data_abort() at data_abort+0x2fc handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0x96000004 rms_rlock() at rms_rlock+0x1c zfs_clone_range() at zfs_clone_range+0x68 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c null_bypass() at null_bypass+0x118 vn_copy_file_range() at vn_copy_file_range+0x18c kern_copy_file_range() at kern_copy_file_range+0x36c sys_copy_file_range() at sys_copy_file_range+0x8c do_el0_sync() at do_el0_sync+0x634 handle_el0_sync() at handle_el0_sync+0x48 --- exception, esr 0x56000000 KDB: enter: panic [ thread pid 3792 tid 100394 ] Stopped at kdb_enter+0x48: str xzr, [x19, #768] db> I'll keep the debugger open for a while. Can I type something for additional info? Regards, Ronald. ------=_Part_3451_1193281966.1699611152304 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi,

Had this crash today on RPI4/15-CURRENT.

FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 main-b0203aaa46-dirty: Sat Nov  4 11:48:33 CET 2023     ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64

$ sysctl -a | grep bclon
vfs.zfs.bclone_enabled: 1

I started a jail with poudriere to build a package. The jail uses null mounts over ZFS.

[root]# cu -s 115200 -l /dev/cuaU0
Connected

db> bt
Tracing pid 95213 tid 100438 td 0xffff0000e1e97900
db_trace_self() at db_trace_self
db_stack_trace() at db_stack_trace+0x120
db_command() at db_command+0x2e4
db_command_loop() at db_command_loop+0x58
db_trap() at db_trap+0x100
kdb_trap() at kdb_trap+0x334
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0xf2000000
kdb_enter() at kdb_enter+0x48
vpanic() at vpanic+0x1dc
panic() at panic+0x48
data_abort() at data_abort+0x2fc
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0x96000004
rms_rlock() at rms_rlock+0x1c
zfs_clone_range() at zfs_clone_range+0x68
zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
null_bypass() at null_bypass+0x118
vn_copy_file_range() at vn_copy_file_range+0x18c
kern_copy_file_range() at kern_copy_file_range+0x36c
sys_copy_file_range() at sys_copy_file_range+0x8c
do_el0_sync() at do_el0_sync+0x634
handle_el0_sync() at handle_el0_sync+0x48
--- exception, esr 0x56000000


Oh.. While typing this I rebooted the machine and it happened again. I didn't start anything in particular although the machine runs some jails.

x0: 0x00000000000000e0
  x1: 0xffffa00090317a48
  x2: 0xffffa000f79d4f00
  x3: 0xffffa000c61a44a8
  x4: 0xffff0000deefe460 ($d.2 + 0xdd776560)
  x5: 0xffffa001250e4c00
  x6: 0xffff0000e54025b5 ($d.5 + 0xc)
  x7: 0x000000000000030a
  x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100)
  x9: 0x0000000000000001
 x10: 0x0000000000000000
 x11: 0x0000000000000001
 x12: 0x0000000000000002
 x13: 0x0000000000000000
 x14: 0x0000000000000001
 x15: 0x0000000000000000
 x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310)
 x17: 0xffff0000004e0d44 (rms_rlock + 0x0)
 x18: 0xffff0000deefe280 ($d.2 + 0xdd776380)
 x19: 0x0000000000000000
 x20: 0xffff0000deefe460 ($d.2 + 0xdd776560)
 x21: 0x7fffffffffffffff
 x22: 0xffffa00090317a48
 x23: 0xffffa000f79d4f00
 x24: 0xffffa001067ef910
 x25: 0x00000000000000e0
 x26: 0xffffa000158a8000
 x27: 0x0000000000000000
 x28: 0xffffa000158a8000
 x29: 0xffff0000deefe280 ($d.2 + 0xdd776380)
  sp: 0xffff0000deefe280
  lr: 0xffff000001623564 (zfs_clone_range + 0x6c)
 elr: 0xffff0000004e0d60 (rms_rlock + 0x1c)
spsr: 0x00000000a0000045
 far: 0x0000000000000108
 esr: 0x0000000096000004
panic: data abort in critical section or under mutex
cpuid = 1
time = 1699610885
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x38
vpanic() at vpanic+0x1a0
panic() at panic+0x48
data_abort() at data_abort+0x2fc
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0x96000004
rms_rlock() at rms_rlock+0x1c
zfs_clone_range() at zfs_clone_range+0x68
zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
null_bypass() at null_bypass+0x118
vn_copy_file_range() at vn_copy_file_range+0x18c
kern_copy_file_range() at kern_copy_file_range+0x36c
sys_copy_file_range() at sys_copy_file_range+0x8c
do_el0_sync() at do_el0_sync+0x634
handle_el0_sync() at handle_el0_sync+0x48
--- exception, esr 0x56000000
KDB: enter: panic
[ thread pid 3792 tid 100394 ]
Stopped at      kdb_enter+0x48: str     xzr, [x19, #768]
db>

I'll keep the debugger open for a while. Can I type something for additional info?

Regards,
Ronald.
  ------=_Part_3451_1193281966.1699611152304--