From nobody Tue Sep 19 12:38:26 2023 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rqh6C2T6vz4tsRR for ; Tue, 19 Sep 2023 12:38:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rqh6B5MmQz4KBL for ; Tue, 19 Sep 2023 12:38:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1695127106; a=rsa-sha256; cv=none; b=uvgtTcUCxTcANxh88DZLXrNgh8rDwp936z0t+BoXfEk77VkVKWFqGue5T2SzjBt/eDJn6r ZGA0EL7j/TifwmwAaxWII9RLWxcedxyjcE+waqce2T1oowrp7QJMReJ7QGFGXbfLqGL3hK 214DLKujH0681mBmU550H1HysPmxHaKMUuYVO8m39EbMrMKjtz5+vHFEGzJVFiMnBnEPWx KNApdalT2mK0AXm71J93Eg7ukYOJUMugrnYW7ZQA1i3aAY32ZGNDYJ8F79b+76o43mbzrd JHI3dcCguMVwrit1QQFuL4vrQY3nKZhQUHL/t9DY6//IgoCSgpfhJAHI+pJTcA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695127106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=A/kEfR2Q+VSvGc+pYKZHim4TWgqzxOtFPCONAtyBBe8=; b=x8dOZwHA87x5rh3p29ueM6R4FmEoo/IfECq75DiP859QCOgsjeearIQwiRPwXrUh9amDqy 1ZH/J+K6NFwgxXgdeBOk+nUPON5RY8qLsmCyEZF1skGfkv6gLiB8jmaH6yn7/6TUrgLWDc rqPEy0+Wq3iTWfpbUJdpuzJxraUbAWEN94kmVbiYQgl+ZS/jsn1wVZpF6pbMG3iQO0X3Z2 TwRqxPnR+64DacctqyLFZ7QdPEwSmb6rPHME/X5wZnLjk4BfJuinIOOSmuvxarpNvXmes/ HShfhyoYE2zKLu7P6mVwXHCTO/Ihma5v9/It4L/mgB9AlPZhU77YW8jCKY/+BA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Rqh6B4Mkcz1C57 for ; Tue, 19 Sep 2023 12:38:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 38JCcQ4t092541 for ; Tue, 19 Sep 2023 12:38:26 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 38JCcQLh092540 for bugs@FreeBSD.org; Tue, 19 Sep 2023 12:38:26 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 273942] [fusefs]Read operation changes ctime on FUSE filesystems. Date: Tue, 19 Sep 2023 12:38:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.2-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: chogata@moosefs.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter attachments.mimetype attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273942 Bug ID: 273942 Summary: [fusefs]Read operation changes ctime on FUSE filesystems. Product: Base System Version: 13.2-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: chogata@moosefs.com Attachment #245021 text/plain mime type: Created attachment 245021 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D245021&action= =3Dedit Program in c to test ctime change on file read. A user reported that MooseFS client on FreeBSD 13.2 fails to complete some = git operations. We researched the problem and it all boils down to one thing: r= ead operation on a FUSE filesystem changes ctime of the read file. Steps to repeat: - install FreeBSD (13.2 RELEASE or 14.0 STABLE) - install sshfs or MooseFS (notice: I wasn't able to compile sshfs on 14.0,= but I tested this version with MooseFS) - use one of the above filesystems to run this simple python script (name it test.py): #!/usr/bin/env python3 import os import time ctime_before_read =3D os.stat('test.py').st_ctime time.sleep(1) fd =3D open('test.py','r') fd.read(10) fd.close() ctime_after_read =3D os.stat('test.py').st_ctime if ctime_before_read !=3D ctime_after_read: print("read operation has changed ctime value !!!") I'm also attaching a source written in c, that performs a similar operation= in less "dirty" way, aka it doesn't read itself and it handles all errors. In MooseFS we have a way to see what filesystem operations the kernel sends= to the filesystem via FUSE. So we can see that this happens (these are operati= ons from the c program, not from the python script): 09.19 14:11:00.100342: uid:0 gid:0 pid:31845 cmd:open (9239161,O_RDONLY) (u= sing cached data from lookup): OK (direct_io:0,keep_cache:0,append_mode:0) [handle:0C000B40] 09.19 14:11:00.107953: uid:0 gid:0 pid:31845 cmd:read (9239161,10,0) [handle:0C000B40]: OK (10) 09.19 14:11:00.108010: uid:0 gid:0 pid:31845 cmd:flush (9239161) [handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK 09.19 14:11:00.108388: uid:0 gid:0 pid:31845 cmd:setattr (9239161,0x10,[atime=3D1695125460]) [no handle]: OK (1.0,[-rw-r--r--:0100644,1,0,0,1695125460,1694769983,1695125460,10]) 09.19 14:11:00.108427: uid:0 gid:0 pid:31845 cmd:release (9239161) [handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK The setattr in between the flush and release should not happen. Setattr is a separate operation... and it will change ctime, because we don't know if it= was sent by the kernel or the user.=20 A FUSE filesystem handles atime/mtime/ctime by itself. In a network filesys= tem it should always be the job of the filesystem to keep atime/mtime/ctime updated, not the kernel's. The kernel of one client doesn't know if its tim= e is synchronized with other clients, with the filesystem's server. If data is read from cache (which is NOT what happened here, the read opera= tion was sent to the filesystem), a settattr with "now" time instead of timestamp could be used. (Linux doesn't bother, their operation log is exactly the sa= me minus the settattr operation). There are systems (first "big" example: git) that use ctime to check if a f= ile changed. All those systems will work incorrectly on a FUSE filesystem on FreeBSD. --=20 You are receiving this mail because: You are the assignee for the bug.=