From nobody Sun Dec 24 21:31:37 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyvPP6PMHz54Wxt for ; Sun, 24 Dec 2023 21:31:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyvPN5xK0z3QBc for ; Sun, 24 Dec 2023 21:31:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=TF8e22oW; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703453510; bh=L5u6bHdxQ0n9y73nAeGAweGqZiZ4uFRzkdf2DDuKZ1E=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=TF8e22oWAM+011CWp0h/a7tD+scRp3KCUaawNioBuznTnibZyrpckL78moOwuWD0yTxuEa/chTH+VtnmYPXGohnLp6VfgUmbFEzbEsBS7NoULdmsurP5kAV8/dQ+2f8xEbJlfb9zlLxIrpasA1Pp9JshFW6Uh6vuLeGx1Qars9ips6HRv8gR2RMTz2KJjE4NCQT/9VfqN2GXGXASryGnIG86R07NL2c89E8tI83gkLsINdqZX+sMmxRUkJ6viNGyXKhztAJdZ0EQwBkFjOjBAewEdX9vCCH0cjZpJpi1mVp04Dg/4QOn/cmotXDByVAnY2VK6CCUER+Pyah4faZerA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703453510; bh=g3nKcj7kjqE21e6bTHhODAmSIcaR/ikLL3/TxcMpJ6O=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=AG54SNh2MhbGRrymvi8kI6Hnr1BThXXVweWJyVkGMzZPHROP+VLv9MJfknIcA49MivAxrV1WEhZqrt/98atSndgbeaCcncNJ39UOBxOTBnQBIkkxc+BTFffawG056pttKMZF2zNAlBijv7DD+2LOu2jBom5t4yAYMortdCvgWf92kpTwGXm46kaguNEaXZFpS1JVhq7Rc/FPKQdyB9ajGLo4O5fezE0MfQicPb5VmrxG8i7R5ZANq8FBx/g63zW5VT6WvUVmY0nePIk72Phry6fF/rqGGvtT2h4bgMDsHdV8GDMG1MTCwhf4eoD+G7tBwfxykwDBTRwoNOZCTBYV6g== X-YMail-OSG: iNKZzMEVM1lgcZjThdv3ZvVLkWowrZxzQayh3Nu7vOkXw6fbcFswMH5wZc7DH_b nCJgayrG3cMPA.Gfz3OeM5_GNAQyY1fxxNM2fslglddjyMYBakPkscGGkIljQGnqv709jS1Nb78Q xvDJAV06obrLRxncqB1duWXXlV64WKSMN14cAtlaGmZnyxn47slcWwqpXXImPFLTcFtv7UtU8rqQ XzlMtOgpFh9pSOeGcvzdiXOqrNsI4PzCaTwjJoevNOYc_k.E1pSZr.4UExtFsS6lfhfLcVadObzw ezlezQcRPhoPH6t_kjxbjS3O23Lewtg58rI_U0JsjOGeDw4R69efDOhVtZFXHdbks6yXPok0KJ.F gR9n5zlXuuOrsnjpzdNU7BUjJSdup9C1k42_tN18A_iEQqAqqHLwPzoXYc1_h53RxFazu8rcGrjT rCELsGjkC5fMH22yjJBVh9Uft2oPMTU7qbXzobreFqyvZ1ypCNfVqJ5E6xHkKVgsNOZiS.zHy6KV V0BnH0Q5Dp1p.KxT0dYW2xVf1B3loU1toZGZtWjqJtzkGJVI_Bk30iv415uMcNE3bIKgwbCJmx_y DyreJRypOtEm_0HBgUZK8uNPCK0e6oqsQIF_tSVUGUCorrbFkZQaAqQ6vmctK1C0oOUpiOVBW03M 8OYu53UQNGJhz787EzEFhVNCIYGZtpYbUlXyJyPhSo3EqcKuwWsdBmH6yefgK3FIJuHXHEnz9ZbU Ji5BMObc2EZx69H9ZjjiJfI1RhpqwTakv2VT0fpeiUbVU.ORKqCVZHX7_lZjnidYlpL0OyvoKEpd ApWYY0EmamyfDynAaio0MpMb55r9ZIm29aORKdOBJ_TPptQIOhkazv5dwDBM_effnR4BpM3I38z9 bLugO3lWMvxYAioTy87p6iqq7pY.lpuBH9JE3y2TL79NMcf3yd02nzlwoeIy0kBhT2_eldEuJonJ VWRdTrHOqY6Gz7omPzJ4GjNUPbxrgvtTHMDXugaSORq2XmYeXP4GMz7BlnH7Ck9jrwCuImGCWpOc RN1TT5_vbe0_SI5LOAkBHvyKEgGditPpi4JlJ1C6Flb6TMBqOSRjX.xxDUAbgWSuhvYOEn4C3uAB cZpJ.yKHJ83nCWmokHEhmxk_YtKAzAANebfP306xANM8i6p4fFrN0J6rHxT2ngVEyBNJZyUVk2ku Ocr9XTkqZYRVZqHD_xEiKlf3Xt6D1bk8TSnWE2DMTx41Jc1ZVQjnFLKRH5yvw7pBxRhpNHo233OM YbMuSUpspt2mdwnXHVjhrBdp28ZkTEH8Op7a7xAPkemaC4mqp1egME3HTvob3zq2WRLuH4va976Z jUIiPx2jZHMLY7Smv9RKkvLWGPR2W8TSk0WK0CM53aLoQfcflCrrzCiw0C7ZPte0vHKHhnkKg0wk Kglrr9POvCTue7hONy1rBCQMEbMta3wncgQ.RrbBQjO71RQMk8zgIK4eGGHrFAymnsrOMU7DgVEu 0yODzWd2wZiQ1zXYyLMpiJrSx1BUSSMHF9dh1K5497vpnHbRTPrbp49JioweJNvsJwk9bytozmWl 5csUyXWoeOlMZvixnOr.2iyjoD8Je5EomEREoHgOCQA2y44X90gW6wnt4_DBkKpnih_7.9bebQtv jhCYvbltsWn1pfFM5rrntuX3Cd1_.vxIOQtLDATzsQhNsh8hQsf96CdvERytY.F5TLdBbSKKBVOu M6D1I51nyIVUZtXDh9kPgljiYc5GMhRd7dszXRYBImULM7yVQRe4aDLeL_saxWxGft_f1R_Yf6LG tyigjckvcfYcHj8aCfV4QEVgfYbpUGo2ZsLgbtQZxg771Wgh7hA3iGQN4eghFjqMr3aue8vVL1Gt rrXd.lV0quLsK8A.IMCLoQv4qui11iju1k.sGFvc61kONKxH.mlts_ZgO9NYltP5IPp_JZzz5VAZ e0OPwFdlGPTsnscB91OSw8E33KwpV6SMBuk6biu2UgBr7dT3xLcRgc45kP__L.xTqttFm0ZSkyvI zmy5G4v33ak5e9thT1Wj0C2C9aD5ei2rIvRzx0rdcsbkg1.RVhDdaa3tzm9piVCdTJgsvixX.PRn bUSl1FEC4Q7LEbzA4wiL.xq85GetfatrNWFL7ULAr1gRpLMPz3R_I.09ULVyzOhVO1lvlr0TvZv1 UgTNrUm2lJ8HKhCmgvRKHe3r89ayGfgeoG0apDb0tq1peRBY7dXJRJZMzSAIUq5HF2L4rakBBaHW EvgreSHPFAr6spNaWRCAr09RJJLL46p4PZmOtJ2f8nqT_gwN3GA8inCHGf3Et_hf6gp_SwVnKcvJ uK5pnF.Q- X-Sonic-MF: X-Sonic-ID: 049c3491-2ee0-4967-a313-59a638fdc77a Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sun, 24 Dec 2023 21:31:50 +0000 Received: by hermes--production-gq1-6949d6d8f9-c9pk7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID fb55ae36ebc6fd2c72a35ee06edce5a4; Sun, 24 Dec 2023 21:31:48 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: <0D286183-ACD6-4818-8E27-E03D7F16D288@yahoo.com> Date: Sun, 24 Dec 2023 13:31:37 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: <0D286183-ACD6-4818-8E27-E03D7F16D288.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyvPN5xK0z3QBc X-Spamd-Bar: --- void wrote on Date: Sun, 24 Dec 2023 12:28:05 UTC : > On Sat, Dec 23, 2023 at 11:55:43AM -0800, Mark Millard wrote: >=20 > >If you had the resources to test avoiding the encrypted > >partition for your type of media, that might prove > >interesting. >=20 > The problem remains even with UFS. It remains after any non-swap i/o > load stops, even when swap is not otherwise in use.=20 >=20 > I took /dev/da0p5, formatted it with newfs -U and put it at /mnt. >=20 > then >=20 > root@pi4:/mnt# date && git clone https://git.freebsd.org/ports.git > Sun Dec 24 12:06:50 GMT 2023 > Cloning into 'ports'... > remote: Enumerating objects: 6044522, done. > remote: Counting objects: 100% (941/941), done. > remote: Compressing objects: 100% (125/125), done. > remote: Total 6044522 (delta 923), reused 821 (delta 816), pack-reused = 6043581 > Receiving objects: 100% (6044522/6044522), 1.16 GiB | 6.94 MiB/s, = done. > ^Csolving deltas: 2% (101172/3649394) >=20 > date && dd if=3D/dev/urandom of=3D/dev/da0p6 bs=3D8k conv=3Dsync = status=3Dprogress > Sun Dec 24 12:06:52 GMT 2023 > ^C203841536 bytes (204 MB, 194 MiB) transferred 205.012s, 994 kB/s=20 > 25009+0 records in > 25008+0 records out > 204865536 bytes transferred in 205.914885 secs (994904 bytes/sec) >=20 > cancelled both processes, checked swap >=20 > # date && swapinfo -h > Sun Dec 24 12:11:56 GMT 2023 > Device Size Used Avail Capacity > /dev/da0p2 2.0G 0B 2.0G 0% > /dev/da0p4 2.0G 0B 2.0G 0% > /dev/da0p6 2.0G 0B 2.0G 0% > /dev/da0p8 2.0G 0B 2.0G 0% > Total 8.0G 0B 8.0G 0% >=20 > while /dev/da0p6 is available as swap to the system >=20 > # date && dd if=3D/dev/urandom of=3D/dev/da0p6 bs=3D8k conv=3Dsync = status=3Dprogress > Sun Dec 24 12:10:38 GMT 2023 > ^C78790656 bytes (79 MB, 75 MiB) transferred 96.008s, 821 kB/s=20 > 9767+0 records in > 9766+0 records out > 80003072 bytes transferred in 96.986518 secs (824889 bytes/sec) >=20 > # swapoff /dev/da0p6 >=20 > # date && dd if=3D/dev/urandom of=3D/dev/da0p6 bs=3D8k conv=3Dsync = status=3Dprogress > Sun Dec 24 12:21:28 GMT 2023 > dd: /dev/da0p6: end of device2031 MiB) transferred 105.007s, 20 MB/s >=20 > 262145+0 records in > 262144+0 records out > 2147483648 bytes transferred in 105.837991 secs (20290291 bytes/sec) Somewhat analogous swap-active vs. swap-inactive sequences follow. I've updated to the new snaphot (that has a UFS limit increased to be big enough to again allow "poudriere bulk -a" to finish). Note that this was tried in a context with: # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 1500/-1 600/-1 dev.cpu.0.freq_levels: 1500/-1 600/-1 dev.cpu.0.freq: 600 # sysctl hw.cpufreq.sdram_freq hw.cpufreq.sdram_freq: 400000000 no powerd use or the like. I may later do some of the activity with my usual settings for such. # gpart show -p =3D> 63 468862065 da0 MBR (224G) 63 1985 - free - (993K) 2048 102400 da0s1 fat32lba [active] (50M) 104448 468757680 da0s2 freebsd (224G) =3D> 0 468757680 da0s2 BSD (224G) 0 128 - free - (64K) 128 451979136 da0s2a freebsd-ufs (216G) 451979264 16777216 da0s2b freebsd-swap (8.0G) 468756480 1200 - free - (600K) # swapinfo -h Device Size Used Avail Capacity /dev/label/growfs_swap 8.0G 0B 8.0G 0% # dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D8k conv=3Dsync = status=3Dprogress ^C506142720 bytes (506 MB, 483 MiB) transferred 31.001s, 16 MB/s 62278+0 records in 62277+0 records out 510173184 bytes transferred in 31.247613 secs (16326789 bytes/sec) # dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D8k conv=3Dsync = status=3Dprogress ^C783056896 bytes (783 MB, 747 MiB) transferred 48.001s, 16 MB/s 96653+0 records in 96652+0 records out 791773184 bytes transferred in 48.533448 secs (16313969 bytes/sec) # swapoff /dev/label/growfs_swap # dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D8k conv=3Dsync = status=3Dprogress ^C749903872 bytes (750 MB, 715 MiB) transferred 46.037s, 16 MB/s 92085+0 records in 92084+0 records out 754352128 bytes transferred in 46.308913 secs (16289567 bytes/sec) What I observe via "gstat -spod" is the likes of: dT: 1.004s w: 1.000s L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w = d/s kB kBps ms/d o/s ms/o %busy Name 1 1999 0 0 0 0.0 1999 8 15992 0.3 = 0 0 0 0.0 0 0.0 67.0| da0 Sometimes L(q) (length of queue) is 0 instead. The ops/s and w/s vary = some but 1999 looks to be the most frequent value. %busy also vary some. ms/w seems rather stable once it gets going. I'll note that ms/r counts how long it was in the queue but w/s does not involve the in-queue time. The above applies to both dd's. FYI: in my RPi4B environment: # find / -name "*.core" -mtime +2 -ls lasts a few seconds at most but there are no source trees or other significant additions to the snapshot content. The same goes for: # periodic daily So I may need to clone to populate a /usr/ports or some such. . . . # cd /usr # pkg install git-lite . . . # git clone ssh://anongit@git.FreeBSD.org/ports.git . . . Updating files: 100% (156990/156990), done. # swapinfo Device 1K-blocks Used Avail Capacity /dev/label/growfs_swap 8388604 0 8388604 0% # dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D8k conv=3Dsync = status=3Dprogress ^C554303488 bytes (554 MB, 529 MiB) transferred 34.000s, 16 MB/s 69046+0 records in 69045+0 records out 565616640 bytes transferred in 34.691537 secs (16304168 bytes/sec) # swapoff /dev/label/growfs_swap # dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D8k conv=3Dsync = status=3Dprogress ^C553779200 bytes (554 MB, 528 MiB) transferred 34.001s, 16 MB/s 69145+0 records in 69145+0 records out 566435840 bytes transferred in 34.774020 secs (16289052 bytes/sec) So I do not see the variability --including not in the "gstat -spod" display updates. I'll note that with /usr/ports/ populated: # find / -name "*.core" -mtime +2 -ls now takes more time when first tried. During it "gstat -spod" showed the likes of: dT: 1.066s w: 1.000s L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w = d/s kB kBps ms/d o/s ms/o %busy Name 0 506 506 4 2049 0.4 0 0 0 0.0 = 0 0 0 0.0 0 0.0 18.0| da0 (L(q) is 1 some of the time.) But that is only for the 1st run. Rerunning afterwards does little I/O (caching). Other forms of prior activity could produce such variations. Adjusting with swap on vs. off status does not change this behavior. Looks like a reboot or some such is required to have the find do a bunch of I/O. This likely messes up using periodic daily as well, until a reboot or some such. Note: so far I've not seen any "gstat -spod" information for any of your types of experiments. Below I instead use bs=3D4k . I also use "time -l" # swapinfo Device 1K-blocks Used Avail Capacity /dev/label/growfs_swap 8388604 0 8388604 0% # time -l dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D4k = conv=3Dsync status=3Dprogress ^C598851584 bytes (599 MB, 571 MiB) transferred 63.000s, 9506 kB/s 147514+0 records in 147513+0 records out 604213248 bytes transferred in 63.563283 secs (9505696 bytes/sec) time: command terminated abnormally 63.56 real 0.46 user 14.32 sys 2176 maximum resident set size 16 average shared memory size 4 average unshared data size 134 average unshared stack size 130 page reclaims 0 page faults 0 swaps 0 block input operations 147514 block output operations 0 messages sent 0 messages received 64 signals received 147516 voluntary context switches 14 involuntary context switches Example gstat -spod display: dT: 1.004s w: 1.000s L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w = d/s kB kBps ms/d o/s ms/o %busy Name 1 2299 0 0 0 0.0 2299 4 9195 0.3 = 0 0 0 0.0 0 0.0 76.3| da0 # swapoff /dev/label/growfs_swap # time -l dd if=3D/dev/urandom of=3D/dev/label/growfs_swap bs=3D4k = conv=3Dsync status=3Dprogress ^C579633152 bytes (580 MB, 553 MiB) transferred 61.001s, 9502 kB/s 141615+0 records in 141615+0 records out 580055040 bytes transferred in 61.045478 secs (9502015 bytes/sec) time: command terminated abnormally 61.05 real 0.48 user 13.73 sys 2176 maximum resident set size 16 average shared memory size 4 average unshared data size 135 average unshared stack size 131 page reclaims 0 page faults 0 swaps 0 block input operations 141615 block output operations 0 messages sent 0 messages received 62 signals received 141618 voluntary context switches 14 involuntary context switches dT: 1.004s w: 1.000s L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w = d/s kB kBps ms/d o/s ms/o %busy Name 1 2328 0 0 0 0.0 2328 4 9311 0.3 = 0 0 0 0.0 0 0.0 76.7| da0 Note the "time -l" "block output operations" being just a little smaller than "voluntary context switches" for each. Also, it appears that bs=3D4k is limited by whatever contributes to the ops/s being under, say, 2400 rather than being more like 2*1999 (1999 being from the 8k=3D=3D2*4k dd context). Another systematic difference in our contexts is my use of one swap partition (the default for the small arm board snapshots these days). =3D=3D=3D Mark Millard marklmi at yahoo.com