From nobody Fri Dec 31 21:27:26 2021 X-Original-To: scsi@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 63C7019245D7 for ; Fri, 31 Dec 2021 21:27:28 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JQdX023dlz4Z8d; Fri, 31 Dec 2021 21:27:28 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qv1-xf31.google.com with SMTP id p12so1418186qvj.6; Fri, 31 Dec 2021 13:27:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=Sf1X4xDuDlaV8uKWNsrst6mGtGoVyw7O4WKKOeWVTiA=; b=qr+0s4W0/URlsmyKshHxns1gCnmihohy9tonRe++wtvuaXMHPbjuvPb3ngX1ZXbyzJ EgS2hJC/RiFQEJuQpk3obeRG1jwKkftuBHYaS8WcKjJUWICCL8YuyZSTCvJ2j+q9GaK0 w9MDkLbvc2wvsjJG1J89MNHL/A7h2L0sFAJXS+vU2hens053auMLGPbQb/6bVks0hKxW 5LEjCbfjcT7bjzftVFaNN4ASyrWPIH5Px+BFdxNnN+yiJII/bIlFLB16T4F1dlvfhNag 7jv3TDIC/PcSxWzyiIkfky14Evk6biAtGLyaJN1v6WnLnEPvP4wopmsm8x48BLHrTI4A tWIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Sf1X4xDuDlaV8uKWNsrst6mGtGoVyw7O4WKKOeWVTiA=; b=3XkSKYifxPeCgL3LjNIRcMWf9/7rRBk3qaNxdOaLd/yS8awyQwcs1v6WwKmJdYMyJO bJzumGJGqQ7WQGQJZ87mQFKBPipwiOlnTZ5NgyMCc8STiPunS86cFgdhe1jqau774Cgr jwMhkeZapqwbneoZyNqCmJg/GS4VFGLSfEGGR4P25lAtFF7aDnmS7WKpHfQ8Bvyby1dO PgalFfUcUacqwuOjaLS9AYNmvpz+NczkDlGfw7IM6q05LYT7v4sC9Rvap7lL43nNSJK9 02PIBniI4zxdmonWw84h73002NGrUlvJWY8TM+8GK1wnjM5/OLYHHyZJyICNn0H4n1dI FakQ== X-Gm-Message-State: AOAM531L5C+SONuoRN8t7NfQqcCIC486n5Gpa+JJtNhpdjQ9kB8UR9Mw D4DYQOk3QiMAIMn/U5NTFHTetp6fGG0= X-Google-Smtp-Source: ABdhPJwLddeLLiwoDwjhAgcZy0wnIsj+N3sPUa0O+Y1356FO7LCnq+lZQm0FOdIn/Wi6kEjnDUJ81A== X-Received: by 2002:a0c:edc1:: with SMTP id i1mr33649047qvr.121.1640986047524; Fri, 31 Dec 2021 13:27:27 -0800 (PST) Received: from spectre.mavhome.dp.ua (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id u17sm22632749qki.2.2021.12.31.13.27.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 31 Dec 2021 13:27:27 -0800 (PST) Subject: Re: iSCSI target: Handling in-flight requests during ctld shutdown To: John Baldwin Cc: =?UTF-8?Q?Edward_Tomasz_Napiera=c5=82a?= , scsi@FreeBSD.org, Ken Merry References: <42e175d9-1693-29e2-0b5b-3fa513aa2a2d@FreeBSD.org> <7f1a1a65-199d-858a-792c-42871d1df13e@FreeBSD.org> <68ad0ffd-79fb-973d-e5ba-92deee352d44@FreeBSD.org> <2ce80c64-6954-21d0-74eb-eeb88e289350@FreeBSD.org> <10687910-7500-008c-aed5-61f76ae90b3d@FreeBSD.org> From: Alexander Motin Message-ID: <1df3d9cf-6456-bcff-4fbe-c36136692efa@FreeBSD.org> Date: Fri, 31 Dec 2021 16:27:26 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: SCSI subsystem List-Archive: https://lists.freebsd.org/archives/freebsd-scsi List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-scsi@freebsd.org X-BeenThere: freebsd-scsi@freebsd.org MIME-Version: 1.0 In-Reply-To: <10687910-7500-008c-aed5-61f76ae90b3d@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4JQdX023dlz4Z8d X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 31.12.2021 13:41, John Baldwin wrote: > On 12/30/21 3:06 PM, Alexander Motin wrote: >> No.  cfiscsi_datamove_out() called before the new flag is set would >> still try to send R2T over the dying connection to be aborted by the >> cfiscsi_session_terminate_tasks() few milliseconds later. >> cfiscsi_data_wait_abort() would only be needed if >> cfiscsi_session_terminate_tasks() has already passed through the data >> waiters list and waiting for the last tasks completion. > > So I think what I was missing is that I had assumed in the race case > that the task was not visible when the NEXUS_I_T_RESET ran, but I think > from re-reading now that the task has to have been in the lun's OOA > queue as we don't permit queueing more tasks due to LUN_RESERVED being > cleared, so I think that means that even in the race case the task > has been aborted.  Perhaps then the code in cfiscsi_datamove_out can > just check CTL_FLAG_ABORT instead of cs_terminating?  That would > function similar to your proposed new flag I think assuming it is correct > that the task for any I/O being passed to cfiscsi_datamove_out concurrent > with cfiscsi_session_terminate_tasks must have been "visible" on the OAA > queue and thus aborted by the handler? It was looking like a good idea for few seconds, since you right that almost all commands should be visible via OAA queues and so should be aborted by cfiscsi_session_terminate_tasks() at that point. But there are few exceptions of commands that can be executed without LUNs present, see CTL_CMD_FLAG_OK_ON_NO_LUN. All 3 of them are CTL_FLAG_DATA_IN, so should not appear in cfiscsi_datamove_out(), but I am still not sure it is very good, even though it may probably work. -- Alexander Motin