[Bug 273942] [fusefs]Read operation changes ctime on FUSE filesystems.
Date: Tue, 19 Sep 2023 12:38:26 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273942 Bug ID: 273942 Summary: [fusefs]Read operation changes ctime on FUSE filesystems. Product: Base System Version: 13.2-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: chogata@moosefs.com Attachment #245021 text/plain mime type: Created attachment 245021 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=245021&action=edit Program in c to test ctime change on file read. A user reported that MooseFS client on FreeBSD 13.2 fails to complete some git operations. We researched the problem and it all boils down to one thing: read operation on a FUSE filesystem changes ctime of the read file. Steps to repeat: - install FreeBSD (13.2 RELEASE or 14.0 STABLE) - install sshfs or MooseFS (notice: I wasn't able to compile sshfs on 14.0, but I tested this version with MooseFS) - use one of the above filesystems to run this simple python script (name it test.py): #!/usr/bin/env python3 import os import time ctime_before_read = os.stat('test.py').st_ctime time.sleep(1) fd = open('test.py','r') fd.read(10) fd.close() ctime_after_read = os.stat('test.py').st_ctime if ctime_before_read != ctime_after_read: print("read operation has changed ctime value !!!") I'm also attaching a source written in c, that performs a similar operation in less "dirty" way, aka it doesn't read itself and it handles all errors. In MooseFS we have a way to see what filesystem operations the kernel sends to the filesystem via FUSE. So we can see that this happens (these are operations from the c program, not from the python script): 09.19 14:11:00.100342: uid:0 gid:0 pid:31845 cmd:open (9239161,O_RDONLY) (using cached data from lookup): OK (direct_io:0,keep_cache:0,append_mode:0) [handle:0C000B40] 09.19 14:11:00.107953: uid:0 gid:0 pid:31845 cmd:read (9239161,10,0) [handle:0C000B40]: OK (10) 09.19 14:11:00.108010: uid:0 gid:0 pid:31845 cmd:flush (9239161) [handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK 09.19 14:11:00.108388: uid:0 gid:0 pid:31845 cmd:setattr (9239161,0x10,[atime=1695125460]) [no handle]: OK (1.0,[-rw-r--r--:0100644,1,0,0,1695125460,1694769983,1695125460,10]) 09.19 14:11:00.108427: uid:0 gid:0 pid:31845 cmd:release (9239161) [handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK The setattr in between the flush and release should not happen. Setattr is a separate operation... and it will change ctime, because we don't know if it was sent by the kernel or the user. A FUSE filesystem handles atime/mtime/ctime by itself. In a network filesystem it should always be the job of the filesystem to keep atime/mtime/ctime updated, not the kernel's. The kernel of one client doesn't know if its time is synchronized with other clients, with the filesystem's server. If data is read from cache (which is NOT what happened here, the read operation was sent to the filesystem), a settattr with "now" time instead of timestamp could be used. (Linux doesn't bother, their operation log is exactly the same minus the settattr operation). There are systems (first "big" example: git) that use ctime to check if a file changed. All those systems will work incorrectly on a FUSE filesystem on FreeBSD. -- You are receiving this mail because: You are the assignee for the bug.