From nobody Mon May 01 19:57:01 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q9DWj4YzMz48xL8 for ; Mon, 1 May 2023 19:57:21 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q9DWh13tTz4Pry for ; Mon, 1 May 2023 19:57:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=hBJGCRje; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682971038; bh=xuLSkyPWETSS+3g7ArElxHEzmCDoZIj4wr/z1L33qJ4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=hBJGCRjesL+OhLPjvqGZipel5WepPF1987oHJ3N6sdKHmzuVS1FVF+RlMrNBZqJ3ITF1peVtRhA0nRNo4tSPTCneAB5QR+vKYXoPmXqbgh7hepqK2qRIRk8qdxF9CQi3aQwZsxGLEzXrh0uMPZPLB49zCgepdCy235P96div5/QlNxkipp7zXg3GvCnsitJBnCF5fiCr51e7OTPCc/yQqdDGBzai+H3iViWQDUI4ehvK3hINvVDf++GQdLpPrQZtwh63pEpLJ4NZeQG+K0bIv7ZQtV7jjS9xFzCIES8U0d5pD3T1M5LqT9wqe51mio1yI49BgX3oobLTJOrqvWAYeQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682971038; bh=Ra7XaJuAE/zkOrGuNBU+9mhZ9jsJmpcpFinDNIzoq78=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=MwMXQ3PGAT4NBYHfvk4EGL+XrhVjvxGYrgnOa01hIDCCduhcNSJM3jzpX7RcdmUL3uHin3zQpa/8xKbl+eiO03FXiN3WHG9DhWqXrn6OgUEKJNEhVRuMnUKq+EkkZisdkML7TmVoqwqiJKr59w1v+Tc2OHqNkyp7dBRAeDKmCpXce2LQ7Nm4ncj82B0t6gjH8/qds6IPZ63V/+k3uCLTYsAJJzFnD99ZeEyDU7yLr0VZuRmF9QgvIwhZR8yozs+TZXp3h0VairMAxleNXu2WQmwNQ9IUi1axo4DaR8i2ewflRRIMCMk+zcRwHz0X+Bv+mr2edz5guPw/mr9PhPctCA== X-YMail-OSG: PqQt0j8VM1kkdc7MA5NWMAmvhluv9gq1CFi63iMrijg423MtlF.J.icce02Z7xt gQ9zzd3a3VHJQckL4zdVW_q_TVNRGdr0MEyUhF7t1Qx7j6anSF6RUUsr.LV1joNBKBNezS1NBUt9 w7PWPV.BQd.Gb7NOShDRBRJ6mbqWHs4R9IJOMuSBX1ZJtWwKPle4Y22K6ASaPe9thnrofxuv4cfa PGPw1E9.hy1DqZwsKzeWMS1jAJeWBIk1XZoc8sGs.5YJm3McRaSAJhu_DDDWi9aZc66IMzag4lTC 0T10sB1BsOaQItYMnf5yu_nSSlZjXog8fMahtQZ_J43eHKho85WqkjD_mGrO_LqlIrj9smvCCUIq cQUtc3ASCn5nAGIKyGFfKyaZS..wQRqmVzaOLV4irgWmi.gfj4wT4WGne_pmlwDohW4N6HTVeFTZ 9UiPE8lZP.aiCZG5uv1rznnv_hLvHe7omV0Ol6e_R1rhGylS5.l0f8Pc50yNX7zqP8HPKR6H0WHp JY68ie3_IAV7nfBr5mPL8.PLh.5YxRBK_ew_jYBZ2cN09i_iRDsXEoaKYrc.dX0CbFUXQHhrdJ7S BcsQ9DLiOZRh2qTJJXnifYj30InzWbLazAz46qAOQyQk14ZuTV9zdaYJOsHOTLdchj992LUBR1mO XpOZ6T3ArLrD2il5xv4BtvHDOCDMIvW3KLDo3MxQ1cR1dZwfDfXqiK71.BtfUtgGTfzwKKcyxMAC Z5w72uEusKHKb6SC19zP2cXE1Wy6qxUV4iUihmhdTOyrm95mUrKND6IppmxG_.pKBi9qbmIabzwH fN.I80VwhzfGPdMUo8WBE5PMsbcCOhmRe97QeBeWHH1ibC8FBbhuuz4ofuOzJzCEDrcvEb.nmNyc iMVc2__9T.EefTd4MGAMWc6VWEyhy_8MqaE27QMro.f.p1sXbulu_I6ESJ6TJz15sl0yB0aRcUrp CBsuX9nPFvqqGUJvbBLkRUk60BCIYx.DbaLmD9xPH8zB8kYSIIbh7B96x9XAoHrL0f9WUUfYvymP _3NzoYVjVm6T0iUqRr86FtZTWzWSM.q3xH1ZLP0IQriIa7CbusIOWJOoEIjthMw6Wrmfldn5eX.T 6MRQ6bXhh1gVesqyPGnkd81ThftcWWdyU5qe7uprzqnF9K_iB0KUC70geHy6roDUY9mQn3eDQ8Ij TuO6zk6VxDc9xiRJy4NBZmEGMYkFw9S7aqd4Sl8pRQUYKqX_ns_47_7xv6rqDwHtK0HA6v_uczEq 7GSdP0lsCOlGuuFWCJXljIRHX1V6zD3gnr8KIvvLC_avwcnixfxbhyMWTrgfwAm8ukFEQ6XI0l2L YdAGdxqlRb_Aar8orjp2sKbkJD1cpbs4ZlONUdh2TI765L25lI3EdXaatE2SCYUgz21e5BF2_.1n GldL9xRU8e3LfI1fXMngIA38hEWVqZ7jPwJY_GsaXQN9.392N1BykpTwgWH7pkL1NpjP09MkZ6av SbDnt6mBJVElOFHM0ArglBYj1J2.7zgeKwwhOW3tF.BPJ3EjiLvdo_DK8WwEn_35fI35k2OTcNAY 6S3ZDyU3gL6DLeCHnChzQEWuur3pnv4Wl_YEIMr6mYOQfhiYtTJWu7kPmNTGphl29jnuz9NIs1HF O73m59NRzdqOInt6FHicx6EUvH_mfWoDcC5m0.hjkx70UoYgq6Ta0tsXoQu_YlUvOlkox0vVISKr hsTNnTm1egP.olCArEm51BX9gligWXgwkEuiNEZS5U5YkvFP.hqBng2ljabDgJvaMOKXLTIZ5G7r egmWPCTjLvttW6xdKP8iInW29AZPGBaCRo6avQ0GYRS5WylZ3r350xKQf1EHDv3dvI1Ke3dD7aZw 5uMUKFIs0u4WOKfaw0941gvifchDAQxY7hD.f9aibsNIswtkZPs7670QRr_fGIXrM3TbZpMHT5nA 8u9ydNztTuhcAQnJ.1bX1ViqpFY0uk7wWGECCckg.39TTBzA2bTggxPe4zKEXyTfRPIazcadg491 r.jHSAmpKl3RGcTw8zznYR6aXeCBL_17C68PIq9Z8TxMQMro6RGlKDc.xI1oNMHRHmcKGOz.Iz9R IWKWvV2uI02EKt6Gh2r6GYLPxGBbSaF_NB_IqhEEj7fkr7L_Yf23c35rWC5jVS8.BcfqrRBh2BYj sFRwHySwsOaasMS2GbaFUpzh5RU8gIoQlvksKKz2nWoTEn51yEMQmJijrwga7L4kLswAmbK5AIHc gdoqam.Vydeg4ULWQbRAXHLeVnlMu_EN3tWjyMGrZ3y0F0uBE8xpLQ1Ylw80fV55ttUwyosmFgeC m X-Sonic-MF: X-Sonic-ID: 5b84c1d3-7548-4222-aeb9-79325bd06b9f Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Mon, 1 May 2023 19:57:18 +0000 Received: by hermes--production-bf1-5f9df5c5c4-fgkgh (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7948446c661a58a6abcb434ead47fd04; Mon, 01 May 2023 19:57:13 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: armv8.2-A+ tuned FreeBSD kernels vs. poudriere bulk and USB3 media: tx->tx_quiesce_done_cv related blocking of processes? From: Mark Millard In-Reply-To: Date: Mon, 1 May 2023 12:57:01 -0700 Cc: FreeBSD Hackers , freebsd-arm Content-Transfer-Encoding: 7bit Message-Id: <8F883286-D713-4632-9575-5813E885D125@yahoo.com> References: <7AE28A5B-109E-4C26-9D70-BCA5D49CD79D@yahoo.com> <02DC03AE-E082-4FB5-AA0D-396F64CC23CB@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-2.42 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.918]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org] X-Rspamd-Queue-Id: 4Q9DWh13tTz4Pry X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Apr 30, 2023, at 18:57, Mark Millard wrote: > On Apr 30, 2023, at 17:44, Mateusz Guzik wrote: > >> can you redo zfs test with: >> sysctl vfs.zfs.per_txg_dirty_frees_percent=5 > > Sure. > > Result summary: Seems to have avoided the sustained periods > of low load average activity. Much better for the context. > > > Context: Original ZFS USB3 media. World or Kernel in use > had been built (non-debug style) using: > > -mcpu=cortex-a78C+flagm+nofp16fml > > > Steps for this test . . . > > # poudriere pkgclean -A -jmain-CA78C > . . . > > # sysctl vfs.zfs.per_txg_dirty_frees_percent=5 > vfs.zfs.per_txg_dirty_frees_percent: 30 -> 5 > > # grep USE_TMPFS= /usr/local/etc/poudriere.conf > # EXAMPLE: USE_TMPFS="wrkdir data" > USE_TMPFS="data" > #USE_TMPFS=all > > # poudriere bulk -jmain-CA78C -w -f ~/origins/CA78C-origins.txt > . . . > > At 15 minutes into the build: > 46 ports in 1st 15 minutes. Load average stayed reasonable > for the configuration. > > At 30 minutes into the build: > 102 ports in 1st 30 minutes. Load average still reasonable > for the configuration. > > Looks good compared to before. > > > I've no clue what optimal would be for the context, but > > vfs.zfs.per_txg_dirty_frees_percent=5 > > is vastly better for the context than the default 30 was. > > Thanks. > > > I'm going to stop the test and do the conversion to the > U2 960GB Optane media in the USB3 adaptor and then > compare USE_TMPFS=data vs. USE_TMPFS=all --but using your > vfs.zfs.per_txg_dirty_frees_percent=5 assignment. > Took a while to actually get around to stopping the test. It got 186 of the ports built in the 1st hour. (A from scratch build, starting with building pkg.) I finally have started the U2 960GB Optane based tests, currently USE_TMPFS=data . The initial activity looks like it might build about as many ports as the earlier USE_TMPFS=all test (for different media, vfs.zfs.per_txg_dirty_frees_percent being 30). . . . waiting . . . It got 222 of the ports built in the 1st hour, again starting with pkg. That compares to 262 for the earlier USE_TMPFS=all test. None of these ports form the first hour are large, long running port builds, none using large scale amounts of storage space for its builder. (As I build things, rust for example, uses 17GiBytes+ of file system space, more than half of the size of the RAM in the Windows Dev Kit 2023 for USE_TEMPFS=all just for file system content.) Now for a USE_TMPFS=all build test with vfs.zfs.per_txg_dirty_frees_percent being 5 . I may try letting that run to completion. (The configuration has 118 GiBytes of swap for paging activity.) === Mark Millard marklmi at yahoo.com