rsync and moving files [Re: backup w/ snapshots]

[Norberto Meijome]

Svein Halvor Halvorsen wrote:
> * Norberto Meijome [2005-08-30 00:32 +1000]
> 
>> isn't that the whole point of having a backup? to have *another* copy of your
>> files?
> 
> 
> Well, yes and no.
> 
> The idea is that I have a main computer that I want to backup. I want the 
> backup to be (a) remote, (b) incremental and (c) random-accessible.
> 
> So I thought that every day my backup-server could rsync my main computer, 
> creating a mirror of the relevant directory trees. Then, as soon as the 
> rsync job completes, it takes a snapshot of the filesystem. This snapshot 
> could be mounted r/o and nfs-exported back to the original computer.
> 

yes, that sounds like it would meet your criteria.

> 
> Now: If I have a file /foo/test on my main computer. After the first 
> rsync-job this file will be copied, assigned an inode and put on the disk 
> somewhere. If I change this file, a local snapshot will be smart enough to 
> just store the changed sectors that this file now occupies.

I take your word wrt to how it works. Assuming of course that you move 
within the same filesystem.


> But: If I move the file from /foo/test to /bar/test on my main computer, 

( /foo/ and /bar/ being in the same filesystem)

> rsync will create a BRAND NEW FILE in /bar (and delete the file in /foo, 
> since I used the --delete option). Now this NEW file will have a new 
> inode, and cover new sectors on disk. The snapshot will then tak 
> considerable more diskspace. If I move a large directory tree this way, 
> this will occupy huge amounts of diskspace.

(touche). yup, that's what would happen....but tha's the nature of the 
beast :) don't keep too many snapshots ? ;)

it'd be great if you could keep a log of all local-mv operations,and 
then replay them remotely via ssh.
<hack> replace mv with your own version which does local-mv and either 
does remote-mv over ssh or sends a msg to a service to perform the 
transaction</hack> (yes, there may be other conditions that trigger the 
same effect as an mv...I just dont know which ones)

I guess the proper way to do this (if you are REALLY REALLY worried 
about that extra spaced used for snapshots in the remote site) would be 
to implement a GEOM class that knows about the remote site and the 'mv' 
condition and communicate to the remote end.

> So how can I make rsync know that the files were just moved (renamed, 
> relinked), and make rsync reflect this fact on the remote mirror?

rsync would then be the wrong tool for the job.

I would suggest that you just get more or larger drives for the remote 
site and live with the waste :) (though the GEOM class would be cool :D )

hmm...what about network operating systems like AFS or CODA (not that I 
know much about them, I just read some stuff on those being distruted,etc..)

Let us know how you solve this.

Regards,
Beto