From nobody Thu May 02 14:19:51 2024 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VVbfx1HHjz5JPGS for ; Thu, 2 May 2024 14:19:53 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VVbfx0rwPz4JfD for ; Thu, 2 May 2024 14:19:53 +0000 (UTC) (envelope-from mike@sentex.net) Authentication-Results: mx1.freebsd.org; none Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.17.1/8.16.1) with ESMTPS id 442EJqoe084754 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Thu, 2 May 2024 10:19:52 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4:bd4c:f42f:eabc:4f29] ([IPv6:2607:f3e0:0:4:bd4c:f42f:eabc:4f29]) by pyroxene2a.sentex.ca (8.18.1/8.15.2) with ESMTPS id 442EJp1N052312 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Thu, 2 May 2024 10:19:51 -0400 (EDT) (envelope-from mike@sentex.net) Content-Type: multipart/alternative; boundary="------------EL2hQbxvGZPICZTUMLYILZPQ" Message-ID: Date: Thu, 2 May 2024 10:19:51 -0400 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-stable@freebsd.org Sender: owner-freebsd-stable@FreeBSD.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: how to tell if TRIM is working To: Warner Losh Cc: Matthew Grooms , stable@freebsd.org References: <5e1b5097-c1c0-4740-a491-63c709d01c25@sentex.net> <67721332-fa1d-4b3c-aa57-64594ad5d77a@shrew.net> <77e203b3-c555-408b-9634-c452cb3a57ac@sentex.net> Content-Language: en-US From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= xsBNBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAHNHW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+wsCOBBMBCAA4FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAl+pQfkCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQeVOEFl5W rMiN6ggAk3H5vk8QnbvGbb4sinxZt/wDetgk0AOR9NRmtTnPaW+sIJEfGBOz47Xih+f7uWJS j+uvc9Ewn2Z7n8z3ZHJlLAByLVLtcNXGoRIGJ27tevfOaNqgJHBPbFOcXCBBFTx4MYMM4iAZ cDT5vsBTSaM36JZFtHZBKkuFEItbA/N8ZQSHKdTYMIA7A3OCLGbJBqloQ8SlW4MkTzKX4u7R yefAYQ0h20x9IqC5Ju8IsYRFacVZconT16KS81IBceO42vXTN0VexbVF2rZIx3v/NT75r6Vw 0FlXVB1lXOHKydRA2NeleS4NEG2vWqy/9Boj0itMfNDlOhkrA/0DcCurMpnpbM7ATQRcsMzk AQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4axtKRSG1 t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1qzAJweEt RdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6cLm0EiHPO l5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5o9KKu4O7 gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQABwsB2BBgB CAAgFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAl+pQfkCGwwACgkQeVOEFl5WrMiVqwf9GwU8 c6cylknZX8QwlsVudTC8xr/L17JA84wf03k3d4wxP7bqy5AYy7jboZMbgWXngAE/HPQU95NM aukysSnknzoIpC96XZJ0okLBXVS6Y0ylZQ+HrbIhMpuQPoDweoF5F9wKrsHRoDaUK1VR706X rwm4HUzh7Jk+auuMYfuCh0FVlFBEuiJWMLhg/5WCmcRfiuB6F59ZcUQrwLEZeNhF2XJV4KwB Tlg7HCWO/sy1foE5noaMyACjAtAQE9p5kGYaj+DuRhPdWUTsHNuqrhikzIZd2rrcMid+ktb0 NvtvswzMO059z1YGMtGSqQ4srCArju+XHIdTFdiIYbd7+jeehg== In-Reply-To: X-Scanned-By: MIMEDefang 2.86 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA] X-Rspamd-Queue-Id: 4VVbfx0rwPz4JfD This is a multi-part message in MIME format. --------------EL2hQbxvGZPICZTUMLYILZPQ Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 5/2/2024 10:16 AM, Warner Losh wrote: > > When trims are fast, you want to send them to the drive as soon as you > know the blocks are freed. UFS always does this (if trim is enabled at > all). > ZFS has a lot of knobs to control when / how / if this is done. > > vfs.zfs.vdev.trim_min_active: 1 > vfs.zfs.vdev.trim_max_active: 2 > vfs.zfs.trim.queue_limit: 10 > vfs.zfs.trim.txg_batch: 32 > vfs.zfs.trim.metaslab_skip: 0 > vfs.zfs.trim.extent_bytes_min: 32768 > vfs.zfs.trim.extent_bytes_max: 134217728 > vfs.zfs.l2arc.trim_ahead: 0 > > I've not tried to tune these in the past, but you can see how they affect things. > Thanks Warner, I will try and play around with these values to see if they impact things.  BTW, do you know what / why things would be "skipped" during trim events ? kstat.zfs.zrootoffs.misc.iostats.trim_bytes_failed: 0 kstat.zfs.zrootoffs.misc.iostats.trim_extents_failed: 0 kstat.zfs.zrootoffs.misc.iostats.trim_bytes_skipped: 5968330752 kstat.zfs.zrootoffs.misc.iostats.trim_extents_skipped: 503986 kstat.zfs.zrootoffs.misc.iostats.trim_bytes_written: 181593186304 kstat.zfs.zrootoffs.misc.iostats.trim_extents_written: 303115 --------------EL2hQbxvGZPICZTUMLYILZPQ Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 5/2/2024 10:16 AM, Warner Losh wrote:

When trims are fast, you want to send them to the drive as soon as you
know the blocks are freed. UFS always does this (if trim is enabled at all).
ZFS has a lot of knobs to control when / how / if this is done.

vfs.zfs.vdev.trim_min_active: 1
vfs.zfs.vdev.trim_max_active: 2
vfs.zfs.trim.queue_limit: 10
vfs.zfs.trim.txg_batch: 32
vfs.zfs.trim.metaslab_skip: 0
vfs.zfs.trim.extent_bytes_min: 32768
vfs.zfs.trim.extent_bytes_max: 134217728
vfs.zfs.l2arc.trim_ahead: 0

I've not tried to tune these in the past, but you can see how they affect things.

Thanks Warner, I will try and play around with these values to see if they impact things.  BTW, do you know what / why things would be "skipped" during trim events ?

kstat.zfs.zrootoffs.misc.iostats.trim_bytes_failed: 0
kstat.zfs.zrootoffs.misc.iostats.trim_extents_failed: 0
kstat.zfs.zrootoffs.misc.iostats.trim_bytes_skipped: 5968330752
kstat.zfs.zrootoffs.misc.iostats.trim_extents_skipped: 503986
kstat.zfs.zrootoffs.misc.iostats.trim_bytes_written: 181593186304
kstat.zfs.zrootoffs.misc.iostats.trim_extents_written: 303115


--------------EL2hQbxvGZPICZTUMLYILZPQ--