From nobody Sat Feb 19 17:14:16 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CD5F619CB698 for ; Sat, 19 Feb 2022 17:14:25 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-qv1-xf31.google.com (mail-qv1-xf31.google.com [IPv6:2607:f8b0:4864:20::f31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K1FXw5sFQz3Gg3 for ; Sat, 19 Feb 2022 17:14:24 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-qv1-xf31.google.com with SMTP id a19so21306832qvm.4 for ; Sat, 19 Feb 2022 09:14:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=nqlCg3AEkpmkHEA8DOBI4fYovr3txtISyiH+rHoFLG8=; b=c+dCfxWsXKooEgAEuZD1mpjyF8dSt4QdZnPPfGpDeEXO2BpMGTV1l53tFQm1WXojST UQV6/b1wgzhJPgGtawyVFTB5ycXcLpr9yD2L52izYowDgmQaVXPPaxgxlwssD88gl8Zd p1YrVZnHd8fMZDtY+jlfkfesWxfN+CYOmlB5ABPsQ96eIbfZXz84HRO2p/MsaDPvjwrx DE7vvFmE/E16U4J1dFquE8oY2/5xDE+Ly0OZ0UdHJRtgGaTzAhNWw1fFqjqehKYALP7G MIZXy5RsjWiXr6H+lD+P4xYetSQcxsV70XJ+Hc/TEFxoBL1XO1DF7muAv4MmNsLafHt0 yvGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:message-id:date:mime-version:user-agent :subject:content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=nqlCg3AEkpmkHEA8DOBI4fYovr3txtISyiH+rHoFLG8=; b=3rFo4GLHkYm4WI8GPW7nlOoYcEAN2tBPPtKAnf4W6vVu+H7Q7eu0DCVkbYbn9AtCOA MHM66DwV9yfcE67AJPD2s/+kFMP3ZeFegYwDq4re+YGIJviwvrhOQh8fBGkMaYf+ZGBc QLIiN2/aw6K3K3NGrOqoiDPx0uyT4HTGfgLtizaGrK9zp20vSkMD9bC1lSVv0hzD/73k MwGkedG/D1AZK2k56EUdHfrPqlelmFeijqQFz7JWgCtQ9X8aO1Jm4VlRL2CocMpgbXvw aTqJO4juFmYxC0NHwUI6yupMdUlwrOU/HBtEa7vp0ovnhuAIw5WJeyhBWjZDXm/sE1zu toAg== X-Gm-Message-State: AOAM532FGSkOOagXKbyUqOz8OkFunAs/Je9T69J5vlu0g4f8qXA8d2se zzwbVSAN3zHqPvpGL9+/mog= X-Google-Smtp-Source: ABdhPJxdi+nIBkmHCJzOMCVxeEaHt+a5Q/Nrsg1AUxQZlR91uqrcRW5c6Dmol+lOsAUYc4ww+HWpVQ== X-Received: by 2002:a05:622a:1349:b0:2d7:fd8c:4263 with SMTP id w9-20020a05622a134900b002d7fd8c4263mr11677289qtk.556.1645290857959; Sat, 19 Feb 2022 09:14:17 -0800 (PST) Received: from [192.168.1.66] (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id x3sm22024513qkp.54.2022.02.19.09.14.17 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Feb 2022 09:14:17 -0800 (PST) Message-ID: Date: Sat, 19 Feb 2022 12:14:16 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [Intel AlderLake] Read&Write files to FAT32 or UFS partition cause data corrupt due to P-Core&E-Core Content-Language: en-US To: Mike Karels , Tomoaki AOKI Cc: "Chen, Alvin W" , freebsd-current@freebsd.org References: <5fd2a34e-1135-4237-a028-d4566ff65c69@FreeBSD.org> <20220219115534.7db1b9f199c10894e4280b33@dec.sakura.ne.jp> <7A743668-B5AA-4679-9F56-9A6220CBBC14@karels.net> From: Alexander Motin In-Reply-To: <7A743668-B5AA-4679-9F56-9A6220CBBC14@karels.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4K1FXw5sFQz3Gg3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=c+dCfxWs; dmarc=none; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::f31 as permitted sender) smtp.mailfrom=mavbsd@gmail.com X-Spamd-Result: default: False [-3.19 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.99)[-0.994]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[FreeBSD.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f31:from]; MLMMJ_DEST(0.00)[freebsd-current]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 19.02.2022 12:02, Mike Karels wrote: > On 18 Feb 2022, at 20:55, Tomoaki AOKI wrote: >> Just a thought, but can it be the reason with timing (e.g., rendezvous >> within (i)threads, hardware controlls without using hardware timer) >> problem? >> >> On FreeBSD, IIUC, multi processor (multi core) implementation assumes >> SMP (differs only clock speed) and end up with difference of >> performance at same clock speed within P-core and E-core, possibly. > > Another possibility is that the system is confused by having hyperthreading > on the P cores but not the E cores. No, I've tried to disable SMT and different number of cores to make it look identical and uniform for the scheduler. The only thing I could not test is disabling all P cores to test only E, the motherboard does not allow that, requiring at least one P core enabled. >> BTW, how aarch64 guys implement big.Little support to avoid such a case? >> Or are they simply disable all Little cores and use big only? > > Are there supported arm64 systems with asymmetric processors yet? > > Mike > >> On Fri, 18 Feb 2022 15:36:27 -0500 >> Alexander Motin wrote: >> >>> This looks pretty weird to me, but I don't think it is specific to the >>> FAT32. Just today I've first noticed that booting TrueNAS 12.0-U8 >>> (http://download.freenas.org/12.0/STABLE/U8/x64/TrueNAS-12.0-U8.iso) >>> (based on FreeBSD 12.2 with many backports) from NVMe SSD (I don't >>> insist on NVMe so far) and ZFS almost never completes successfully, >>> ending up in hangs or random stack corruption panics in ZFS threads as >>> soon as at least one E core is enabled of my i7-12700K. Disabling all E >>> cores fixes the problem. Updated to TrueNAS 13.0-BETA1 (based on >>> FreeBSD 13.0-STABLE from few weeks ago) it does not demonstrate the >>> problem any more. The same TrueNAS 12.0-U8 kernel booted from NFS does >>> not seem to demonstrate the problem with ZFS mounting, but I haven't >>> stressed it much so far. >>> >>> There are seem to be dragons somewhere... >>> >>> On 15.02.2022 22:29, Chen, Alvin W wrote: >>>> Hi Guys, >>>> Any updates to support Intel P-core + E-core? >>>> I have filed a bug: PR 261169 >>>> , but no updates. >>>> Does anybody know the progress? >>>> >>>> For Intel Adler Lake P core + E core processor (i7-12700T), copying >>>> files to FAT32 partition, the file corrupted (50%), but ZFS is fine. >>>> After disabling E core in the code by restrict the max cpu number, this >>>> issue is gone. And No E core processor has no such issue, like i7-12400. >>>> >>>> HW ENV: >>>> CPU: Intel AlderLake 12th Gen i7-12700T >>>> Disk: NVME SSD >>>> >>>> There are 3 methods to reproduce this issue: >>>> 1. Make FreeBSD 13 USB disk installer, install FreeBSD with UFS, and >>>> select install source and ports, the txz package checking will be failed. >>>> >>>> 2. Boot to shell by USB disk installer, and mount a FAT32 partition (on >>>> SSD), and copy a 300MB file to the FAT32, compare the sha256 checksums >>>> for the source file and the dst file, the checksum are different (50%). >>>> Or if there is a 300MB file in FAT32 partition, mount the partition, and >>>> for the first time check the sha256 value by running 'sha256 file.tgz', >>>> the checksum is wrong, but the second time, the checksum is correct. >>>> >>>> 3. Install FreeBSD 13 with ZFS, and it can work well. And boot into >>>> FreeBSD, disable swap, and format the SWAP partition to FAT32. Do the >>>> testing as above. >>>> >>>> Regards, >>>> >>>> Alvin Chen >>>> >>>> Dell | Comercial Client Group >>>> >>>> office +86-10-82862506, fax +86-10-82861554, Dell Lync 8672506 >>>> weike_chen@dell.com >>>> >>>> Internal Use - Confidential >>>> >>> >>> -- >>> Alexander Motin >>> >> >> >> -- >> 青木 知明 [Tomoaki AOKI] -- Alexander Motin