zfs q regarding backup strategy

[tech-lists]

On Fri, Oct 01, 2021 at 10:28:16PM +0100, Steve O'Hara-Smith wrote:
>On Fri, 1 Oct 2021 15:14:08 +0100
>tech-lists <tech-lists at zyxst.net> wrote:
>
>> What I'm looking for is something (ideally) written in sh. Or even perl.
>> But even before that, need to understand incremental backups properly.
>> Basically the backup needs to be (effectively) write-only. By that, I
>> mean only added to.
>
>	With zfs send every stream is a send up to some 'target' snapshot,
>a full stream is the complete history while an incremental stream is from
>an earlier 'source' snapshot.
>
>	It is therefore necessary for any successful incremental backup
>system to know what the last snapshot sent was so that the next cycle can
>start by creating a new snapshot and sending the increment from the last
>sent to the latest. On the receiving end the options are to either store
>each send as a file named in such a way as to make it obvious what order to
>process them in should restore be needed or (more usefully) to use zfs recv
>to load them into filesystems in a pool on the receiving machine. If you do
>this there will be no pruning of the received snapshots unless you set
>something up to do it.

OK. if I think I understand correctly then if we need to keep everything then 
I'll need to keep every incremental snapshot.

Let's say there are 3 files in a zfs dir 1, 2 & 3

full backup happens on day 1, all three files are fully backed up

first snapshot happens on day 2: theres nothing added nothing removed because
nothing changed at source

second snapshot happens on day 3, file 2 disappears at source so the 
difference is this file is not there but the other two are.

third snapshot happens file 2 of the same name appears at source but has
different content.

A month (30 days) later someone looks for file 2 as it was on day 1, and
it's not the one they expected on the source. So I have to get it from
the backup somehow. In order to do this, and given that when I'm looking
for this I might not know *when* file 2 was deleted from the source, then
changed and then added, do I have to keep every single
snapshot? If this is the case, then how can I scan through the snapshots
looking for the right file? there may be thousands of snapshots.

Or can I incorporate or expire the snapshots into the full backup and
tell it that I want to look back to day 1 and retreive file 2? The
reason I'm thinking this might be possible is via zfs redundancy.
copies=3 that sort of thing. Like a time machine but for a backup, I
guess.

Or some other method that I've not understood (there's lots about zfs I
don't understand) or haven't read about yet?

thanks,
-- 
J.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20211003/d3017c86/attachment.sig>