From nobody Wed Jul 12 16:35:41 2023 X-Original-To: dev-commits-doc-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4R1Ndn4W3yz2tmsj for ; Wed, 12 Jul 2023 16:35:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4R1Ndn2N5jz47nH; Wed, 12 Jul 2023 16:35:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689179741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=z2kgBO+sz9gYCReZQe41VXh/mBBJyTQzQ278RHpFKeg=; b=iuOyUmUlS5h4C54OA5emMy55jBrMYHSCVHfUPmbBuraGkFFsHrdZEcOBRehNrOe+4CHeBc fnHDsT+DftGDTY0rgJ6bxPm33V6I46g6M1J/HB14qzh+h/6vEdcSGkXE0izc/qcHpqhRIZ AMcTQTV6nR8af4+8HTUna5MLavJk9ulJSeD8/jvsV5l5CbkOlSyV74NfHIXhNoGpacz4ub TVWUIvCjYmODP/g/Vhk167TdP793h7sgbailP2MAdPxL9DsO7+IID/tCspfcIvwZI/k7Vk DRUhm9h1RXMv18xwht+X576As1xe8htfPZ27V8KpGAE5V8g4VSb4/VvGjJgWGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1689179741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=z2kgBO+sz9gYCReZQe41VXh/mBBJyTQzQ278RHpFKeg=; b=Pr/kw6Ggpb2SKRi25iDNqRAbc5GqQy73c8SffrafPh+Q+zfPK2ETxBcC5mBAG4FOtkydGr FL59TwhtxCxYonNRjIqByXR3tDYCao7NOszZ6ur4lZk1SG0vL3aoSakD5NHJZo3gd60Dbb O1dVHGthNwfBPZCMCbJvAGZ5YNqH5nnBulRUTOSxtlppnpOnNi1awkrhhO6x8hf70dqMK2 2b77OUAHu4JnuDLEiBuvyESrCFCdT04Ll12bh6gt3O6ouTGaVRPHIcd5eQNow4IvsWf2+s U0jKwcQfrjypcssa7FOSFo4H35OqUSPefG28V/ad2VHK6aQmB9Fpu2kcKgmIIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1689179741; a=rsa-sha256; cv=none; b=jyrjq9CFfixkLPexsmiC5VtGeYAYz5H18mrgLEvyB8EVSngIH+v/fWWSnlD9TczCjTwWUP X6zlcY/QMlaZxxM9rlXkPZhZ1ycihK7MamdrIbpK/1m91iNwboLawoqh4GkvlmvV/w25H1 i+uDXaGgGiE3gkqbganPFqlRyyMVvA1NrwsZW8jnicAu0UVGSCqgZcrx0NuJjpZe8f+cLT OHtDy2R10/hSfrN4Uwe9lEfC+WjIru8SKJRxT5QpxToB1eST8zSG251cysNJtPL1qVx5HQ UujJs2+21LtjfKKvdjbLUTf8qa+lZqvrJ5DFWBa+2f1bzdxRuP9JX+OSjKqk6Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4R1Ndn1R6pzGHT; Wed, 12 Jul 2023 16:35:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 36CGZf51043104; Wed, 12 Jul 2023 16:35:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 36CGZfpB043103; Wed, 12 Jul 2023 16:35:41 GMT (envelope-from git) Date: Wed, 12 Jul 2023 16:35:41 GMT Message-Id: <202307121635.36CGZfpB043103@gitrepo.freebsd.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org From: Lorenzo Salvadore Subject: git: 4222032e43 - main - Status/2023Q2/nvmf.adoc: Improvements List-Id: Commit messages for all branches of the doc repository List-Archive: https://lists.freebsd.org/archives/dev-commits-doc-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-doc-all@freebsd.org X-BeenThere: dev-commits-doc-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: salvadore X-Git-Repository: doc X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4222032e43c581f284023b917dff5dc3ade0e874 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by salvadore: URL: https://cgit.FreeBSD.org/doc/commit/?id=4222032e43c581f284023b917dff5dc3ade0e874 commit 4222032e43c581f284023b917dff5dc3ade0e874 Author: Lorenzo Salvadore AuthorDate: 2023-07-12 16:28:25 +0000 Commit: Lorenzo Salvadore CommitDate: 2023-07-12 16:28:25 +0000 Status/2023Q2/nvmf.adoc: Improvements - Switch to one sentence per line. - Add one more use of the filename markup. Approved by: dbaio (mentor, implicit) --- .../en/status/report-2023-04-2023-06/nvmf.adoc | 75 +++++++--------------- 1 file changed, 22 insertions(+), 53 deletions(-) diff --git a/website/content/en/status/report-2023-04-2023-06/nvmf.adoc b/website/content/en/status/report-2023-04-2023-06/nvmf.adoc index 445119c7f9..a9ceefdaea 100644 --- a/website/content/en/status/report-2023-04-2023-06/nvmf.adoc +++ b/website/content/en/status/report-2023-04-2023-06/nvmf.adoc @@ -5,67 +5,36 @@ link:https://github.com/bsdjhb/freebsd/tree/nvmf2[nvmf2 branch] URL: link:https: Contact: John Baldwin -NVMe over Fabrics enables communication with a storage device using -the NVMe protocol over a network fabric. -This is similar to using iSCSI to export a storage device over a -network using SCSI commands. +NVMe over Fabrics enables communication with a storage device usingthe NVMe protocol over a network fabric. +This is similar to using iSCSI to export a storage device over a network using SCSI commands. -NVMe over Fabrics currently defines network transports for -Fibre Channel, RDMA, and TCP. +NVMe over Fabrics currently defines network transports for Fibre Channel, RDMA, and TCP. -The work in the nvmf2 branch includes a userland library (lib/libnvmf) -which contains an abstraction for transports and an implementation of +The work in the nvmf2 branch includes a userland library ([.filename]#lib/libnvmf#) which contains an abstraction for transports and an implementation of a TCP transport. -It also includes changes to man:nvmecontrol[8] to add 'discover', -'connect', and 'disconnect' commands to manage connections to a remote -controller. +It also includes changes to man:nvmecontrol[8] to add 'discover', 'connect', and 'disconnect' commands to manage connections to a remote controller. The branch also contains an in-kernel Fabrics implementation. -[.filename]#nvmf_transport.ko# contains a transport abstraction that -sits in between the nvmf host (initiator in SCSI terms) and the -individual transports. -[.filename]#nvmf_tcp.ko# contains an implementation of the TCP -transport layer. -[.filename]#nvmf.ko# contains an NVMe over Fabrics host (initiator) -which connects to a remote controller and exports remote namespaces as -disk devices. -Similar to the man:nvme[4] driver for NVMe over PCI-express, -namespaces are exported via [.filename]#/dev/nvmeXnsY# devices which -only support simple operations, but are also exported as ndaX disk -devices via CAM. -Unlike man:nvme[4], man:nvmf[4] does not support the man:nvd[4] disk -driver. -nvmecontrol can be used with remote namespaces and remote controllers, -for example to fetch log pages, display identify info, etc. - -Note that man:nvmf[4] is currently a bit simple and some error cases -are still a TODO. -If an error occurs, the queues (and backing network connections) are -dropped, but the devices stay around, but with I/O requests paused. -'nvmecontrol reconnect' can be used to connect a new set of network -connections to resume operation. -Unlike iSCSI which uses a persistent daemon (man:iscsid[8]) to -reconnect after an error, reconnections must be done manually. +[.filename]#nvmf_transport.ko# contains a transport abstraction that sits in between the nvmf host (initiator in SCSI terms) and the individual transports. +[.filename]#nvmf_tcp.ko# contains an implementation of the TCP transport layer. +[.filename]#nvmf.ko# contains an NVMe over Fabrics host (initiator) which connects to a remote controller and exports remote namespaces as disk devices. +Similar to the man:nvme[4] driver for NVMe over PCI-express, namespaces are exported via [.filename]#/dev/nvmeXnsY# devices which only support simple operations, but are also exported as ndaX disk devices via CAM. +Unlike man:nvme[4], man:nvmf[4] does not support the man:nvd[4] disk driver. +nvmecontrol can be used with remote namespaces and remote controllers, for example to fetch log pages, display identify info, etc. + +Note that man:nvmf[4] is currently a bit simple and some error cases are still a TODO. +If an error occurs, the queues (and backing network connections) are dropped, but the devices stay around, but with I/O requests paused. +'nvmecontrol reconnect' can be used to connect a new set of network connections to resume operation. +Unlike iSCSI which uses a persistent daemon (man:iscsid[8]) to reconnect after an error, reconnections must be done manually. The current code is very new and likely not robust. It is certainly not ready for production use. -Experienced users who do not mind all their data vanishing in a puff -of smoke after a kernel panic and who have an interest in NVMe over -Fabrics can start testing it at their own risk. +Experienced users who do not mind all their data vanishing in a puff of smoke after a kernel panic and who have an interest in NVMe over Fabrics can start testing it at their own risk. -The next main task is to implement a Fabrics controller (target in -SCSI language). -Probably a simple one in userland first followed by a "real" one that -offloads the data handling to the kernel but is somewhat integrated -with man:ctld[8] so that individual disk devices can be exported -either via iSCSI or NVMe or both using a single config file and daemon -to manage all of that. -This may require a fair bit of refactoring in ctld to make it less -iSCSI-specific. -Working on the controller side will also validate some of the -currently under-tested API design decisions in the -transport-independent layer. -I think it probably does not make sense to merge any of the NVMe over -Fabrics changes into the tree until after this step. +The next main task is to implement a Fabrics controller (target in SCSI language). +Probably a simple one in userland first followed by a "real" one that offloads the data handling to the kernel but is somewhat integrated with man:ctld[8] so that individual disk devices can be exported either via iSCSI or NVMe or both using a single config file and daemon to manage all of that. +This may require a fair bit of refactoring in ctld to make it less iSCSI-specific. +Working on the controller side will also validate some of the currently under-tested API design decisions in the transport-independent layer. +I think it probably does not make sense to merge any of the NVMe over Fabrics changes into the tree until after this step. Sponsored by: Chelsio Communications