From nobody Tue Mar 12 19:27:44 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TvNvw61Glz5FLqp for ; Tue, 12 Mar 2024 19:27:56 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TvNvv6Cbpz4MKr for ; Tue, 12 Mar 2024 19:27:55 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b=iW1RqMaF; dmarc=pass (policy=quarantine) header.from=blastwave.org; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org Received: from [172.16.35.4] (cpe8c6a8d4d360a-cm8c6a8d4d3608.cpe.net.cable.rogers.com [99.253.151.152]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.17.1) with ESMTPSA id 42CJRjF0083854 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Tue, 12 Mar 2024 15:27:52 -0400 (EDT) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1710271672; bh=L7jV0K21EzW4iZe8j49EMABke6Eg817zYf9i3MVGv6c=; h=Date:To:From:Subject; b=iW1RqMaFdmqtf3wp01mO7FwUqwil0AKfkYlooCD5GWmVMg0Y0Eo/pho6QgFWH87Aa zikaeS2KO2y0GNIpTPmnV+72d5ipxznj4jpwFtLObtzJ+QWYYhxJfx87JxSGIZ2vyd TskH/BzKv6GU724/qz5Q0iWmt1SqemLxdk4LCD7m3WGt/LJXCxfCAifftBCS9WlH6j En1F9S+Wra/Xwawsn6WPUfU3RsoVX99BsCDV2f31V7WVmuG3ZqA+xCFYtoGYCRRzcW euW0H1Ts1mFpCCcuKjH1TFiDhO9BPreDMNGWxbhHqXFlrGO2bZShiFOmYAxzhQCbaV pf+Pf7lsCMkIQ== Message-ID: <8228ca0c-85a0-4436-aaf4-d2d987e0f5a4@blastwave.org> Date: Tue, 12 Mar 2024 15:27:44 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: current@freebsd.org Content-Language: en-CA From: Dennis Clarke Subject: ZPool on iSCSI storage not available after a reboot Organization: GENUNIX Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 42CJRjF0083854 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[blastwave.org:+]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; FROM_HAS_DN(0.00)[] X-Rspamd-Queue-Id: 4TvNvv6Cbpz4MKr This is a somewhat odd problem and may have nothing to do with iSCSI config at all. Suffice it to say that I have the following in the server /etc/rc.conf : # # the iSCSI initiator iscsid_enable="YES" iscsictl_enable="YES" iscsictl_flags="-Aa" # During boot I see this on the console : cannot import 'proteus': no suchpid 55 (zpool) is attempting to use unsafe AIO requests - not logging anymore pool or dataset Destroy and re-create the pool from a backup source. cachefile import failed, retrying no pools available to import Sure enough the machine brings up a 10Gbit link with jumboframes *after* the above messages : ix0: flags=1008843 metric 0 mtu 9000 options=4e53fbb ether 8c:dc:d4:ae:18:b8 inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 media: Ethernet autoselect (10Gbase-Twinax ) status: active nd6 options=29 Then a little later I see iscsi doing its goodness : da0 at iscsi1 bus 0 scbus8 target 0 lun 0 da0: Fixed Direct Access SPC-5 SCSI device da0: Serial Number MYSERIAL0000 da0: 150.000MB/s transfers da0: Command Queueing enabled da0: 2097152MB (4294967296 512 byte sectors) add net ::0.0.0.0: gateway ::1 Starting iscsid. Starting iscsictl. The storage exists just fine and iSCSI seems to be doing its thing : root@titan:~ # root@titan:~ # camcontrol devlist at scbus0 target 0 lun 0 (pass0,ada0) at scbus1 target 0 lun 0 (pass1,ada1) at scbus2 target 0 lun 0 (ses0,pass2) at scbus6 target 0 lun 0 (ses1,pass3) at scbus7 target 0 lun 1 (pass4,nda0) at scbus8 target 0 lun 0 (da0,pass5) root@titan:~ # root@titan:~ # gpart show da0 => 40 4294967216 da0 GPT (2.0T) 40 8 - free - (4.0K) 48 4294967200 1 freebsd-zfs (2.0T) 4294967248 8 - free - (4.0K) root@titan:~ # However the zpool therein is not seen : root@titan:~ # root@titan:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT iota 7.27T 597G 6.68T - - 0% 8% 1.00x ONLINE - t0 444G 40.8G 403G - - 4% 9% 1.00x ONLINE - root@titan:~ # Of course I can manually import it : root@titan:~ # zpool import pool: proteus id: 15277728307274839698 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: proteus ONLINE da0p1 ONLINE root@titan:~ # It seems as if there is something out of sequence and the iSCSI processes should be happening earlier in the boot process? I really do not know and am wondering why that zpool proteus on the iSCSI storage needs to be manually import'ed after a reboot. Any insights would be wonderful. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken