Re: rsync use with -tmsdosfs mounted file system? file has vanished: . . .

On Sun, Jul 16, 2023 at 1:57 PM Mark Millard <marklmi@yahoo.com> wrote:

> I used a sequence that looked like:
>
> mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /CA72optM2efi-media/ \
> && rsync -x --delete -aAUHhh --info=progress2 /boot/efi/
> /CA72optM2efi-media/
>
> that got:
>
> file has vanished: "/CA72optM2efi-media/BCM271~5.DTB"
> file has vanished: "/CA72optM2efi-media/BCM271~6.DTB"
>          73.77K   0%    1.63MB/s    0:00:00 (xfr#7, to-chk=0/493)   rsync
> warning: some files vanished before they could be transferred (code 24) at
> main.c(1357) [sender=3.2.7]
>
> After that, activity reported the likes of:
>
> rsync: [generator] delete_file: unlink(overlays/VC4-KM~8.DTB) failed:
> Read-only file system (30)
> and:
> rsync: [receiver] mkstemp "/CA72optM2efi-media/.fixup4.dat.2Wonu9" failed:
> Read-only file system (30)
>
> More than rsync was odd at that point:
>
> # ls -Tld /CA72optM2efi-media/*.DTB
> ls: /CA72optM2efi-media/BCM271~5.DTB: No such file or directory
> ls: /CA72optM2efi-media/BCM271~6.DTB: No such file or directory
>
> # rm /CA72optM2efi-media/*/*.DTB
> override rwxr-xr-x root/wheel uarch for
> /CA72optM2efi-media/overlays/SDHOST~1.DTB? y
> rm: /CA72optM2efi-media/overlays/SDHOST~1.DTB: Read-only file system
> . . .
>
> But:
>
> # mount | grep media
> /dev/gpt/CA72optM2efi on /CA72optM2efi-media (msdosfs, local, noatime)
>
> So the mount itself was not the source of the read-only status so far.
>
> I then tried:
>
> # umount /CA72optM2efi-media
> # newfs_msdos /dev/da0p1
> # mount -onoatime -tmsdosfs /dev/gpt/CA72optM2efi /mnt
> # cp -aRx /boot/efi/ /mnt/
> cp: utimensat: /mnt: Invalid argument
>
> (which is normal).
>
> # umount /mnt
>
> No more oddities , so far after that.
>
>
> For reference:
>
> # uname -apKU
> FreeBSD CA72-16Gp-ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #99
> main-n264171-2a0c0aea4209-dirty: Fri Jul 14 21:00:44 PDT 2023
>  root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm64.aarch64/sys/GENERIC-NODBG-CA72
> arm64 aarch64 1400093 1400093
>
> # pkg info rsync
> rsync-3.2.7
> Name           : rsync
> Version        : 3.2.7
> Installed on   : Sat Jul 15 14:53:48 2023 PDT
> Origin         : net/rsync
> Architecture   : FreeBSD:14:aarch64
> . . .
> Annotations    :
> FreeBSD_version: 1400092
> build_timestamp: 2023-07-02T06:57:44+0000
> built_by       : poudriere-git-3.3.99.20220831
> cpe            : cpe:2.3:a:samba:rsync:3.2.7:::::freebsd14:aarch64
> port_checkout_unclean: no
> port_git_hash  : f45cd5bd9d4b
> ports_top_checkout_unclean: yes
> ports_top_git_hash: 880f72e54deb
>
>
> ===
> Mark Millard
> marklmi at yahoo.co <http://yahoo.com>


This looks a bit like an issue I was hitting on a 4 CPU, 4 thread Alder
Lake processor and 500GB SSD running 13.2-RELEASE.

I saw several very strange corruptions, at least one "rsync warning: some
files vanished before they could be transferred". In one (the last) case,
the system crashed. The 'disc' was corrupted badly enough that fsck failed
and I could not boot up the system. The disk was UFS2 GPT format and EFI
boot. The interface is SATA, not nVME. In this case, I was installing on a
new system and copying the majority of the file system from my old system.

The 'fix' is strange and probably not one many other can use. I installed a
spinning rust drive of 500GB and installed FreeBSD and used rsync again and
it worked. I can't say whether it was a fluke that it worked, but it really
smells like some sort of race condition. Could be rsync , VFS, or device
driver. Since then I have seen one crash while backing up the system disk
using rsync. No corruption and doing another rsync after reboot worked
fine, but it was a much smaller run as the first attempt was nearly
complete when the system crashed. Maybe unrelated. I do have the core file
from the crash. Stil, something weird has been going on. Same issue on two
identical systems, so not likely hardware.
-- 
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkoberman@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683