[Bug 273942] [fusefs]Read operation changes ctime on FUSE filesystems.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273942

            Bug ID: 273942
           Summary: [fusefs]Read operation changes ctime on FUSE
                    filesystems.
           Product: Base System
           Version: 13.2-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: chogata@moosefs.com
 Attachment #245021 text/plain
         mime type:

Created attachment 245021
  --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=245021&action=edit
Program in c to test ctime change on file read.

A user reported that MooseFS client on FreeBSD 13.2 fails to complete some git
operations. We researched the problem and it all boils down to one thing: read
operation on a FUSE filesystem changes ctime of the read file.

Steps to repeat:
- install FreeBSD (13.2 RELEASE or 14.0 STABLE)
- install sshfs or MooseFS (notice: I wasn't able to compile sshfs on 14.0, but
I tested this version with MooseFS)
- use one of the above filesystems to run this simple python script (name it
test.py):

#!/usr/bin/env python3

import os
import time

ctime_before_read = os.stat('test.py').st_ctime
time.sleep(1)
fd = open('test.py','r')
fd.read(10)
fd.close()
ctime_after_read = os.stat('test.py').st_ctime
if ctime_before_read != ctime_after_read:
    print("read operation has changed ctime value !!!")

I'm also attaching a source written in c, that performs a similar operation in
less "dirty" way, aka it doesn't read itself and it handles all errors.

In MooseFS we have a way to see what filesystem operations the kernel sends to
the filesystem via FUSE. So we can see that this happens (these are operations
from the c program, not from the python script):

09.19 14:11:00.100342: uid:0 gid:0 pid:31845 cmd:open (9239161,O_RDONLY) (using
cached data from lookup): OK (direct_io:0,keep_cache:0,append_mode:0)
[handle:0C000B40]
09.19 14:11:00.107953: uid:0 gid:0 pid:31845 cmd:read (9239161,10,0)
[handle:0C000B40]: OK (10)
09.19 14:11:00.108010: uid:0 gid:0 pid:31845 cmd:flush (9239161)
[handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK
09.19 14:11:00.108388: uid:0 gid:0 pid:31845 cmd:setattr
(9239161,0x10,[atime=1695125460]) [no handle]: OK
(1.0,[-rw-r--r--:0100644,1,0,0,1695125460,1694769983,1695125460,10])
09.19 14:11:00.108427: uid:0 gid:0 pid:31845 cmd:release (9239161)
[handle:0C000B40,uselocks:0,lock_owner:0000000000007C65]: OK

The setattr in between the flush and release should not happen. Setattr is a
separate operation... and it will change ctime, because we don't know if it was
sent by the kernel or the user. 

A FUSE filesystem handles atime/mtime/ctime by itself. In a network filesystem
it should always be the job of the filesystem to keep atime/mtime/ctime
updated, not the kernel's. The kernel of one client doesn't know if its time is
synchronized with other clients, with the filesystem's server.

If data is read from cache (which is NOT what happened here, the read operation
was sent to the filesystem), a settattr with "now" time instead of timestamp
could be used. (Linux doesn't bother, their operation log is exactly the same
minus the settattr operation).

There are systems (first "big" example: git) that use ctime to check if a file
changed. All those systems will work incorrectly on a FUSE filesystem on
FreeBSD.

-- 
You are receiving this mail because:
You are the assignee for the bug.