SCSI tape data loss

[Carl Reisinger]

>Concerning the maximum buffer size: I have chosen
>the default maximum buffer size to be 64512 bytes so
>that it is smaller than 65536. In fact, 64512 bytes is
>the size (126 blocks) that I used for tar in 1982 
>and never had any problems.  

Try using the FreeBSD tar with the multi-volume flag (-M) and
your record size.

Without the flag the writes are page aligned, with the flag
the writes are offset some, either 512 or 1536 bytes (I forget
which), and the writes will be split by the kernel physio
function into a 60K and 3K write. (This is with the tar
shipped with FreeBSD up to at least 4.2. Later ones may also
do this, I have not tried them)

>
>>From what I understand the 65536 point at which 
>buffers are always split only applies to devices in
>fixed block mode, and probably older devices at that.

This magic number has nothing to do with the device. I've only
used variable block mode and newer technologies, SDLT, LTO.

>
>Though Bacula can run in fixed block mode, the
>default is variable block, so I don't see that as
>an issue here -- unless I am missing something?
>
>Can you explain why you mention 61440 bytes?  and
>why it might be a better choice than 64512?
>

61440 was mentioned since that is the largest write that can
be done without the physio function doing some surprising and
annoying things to your write. 61440 is the size that, no
matter its address alignment, can always be mapped with one
page register.

If you are careful to page align all writes then you can write
up to 65536 and have one record sent to the tape device.

(Actually, with a minor change to scsi_sa.c and limiting one
self to newer SCSI HBAs you can go as high as 128KB for
read/write)

An example:

Write 64512 bytes with a starting address of 4096. Physio will
take this, see that the address is paged aligned, check that
it can be mapped with one page register and perform one write.

Now lets write 64512 bytes but with an address of 5632. In
this case physio will notice it is not paged aligned and
adjust the starting address to be 4096. Now 66048 bytes need
to be mapped which exceeds the default size of 65536. In this
case physio will map the first 60K (64K to him because of the
starting address change), write that and then map and write
the remainder.

Now when one goes back to read 64512 bytes, the first read
returns 61440 bytes and the second 3072 instead of just one
read retuning 64512.

>On aligning the buffers on a page boundary: interesting
>idea, I'll look into it, but I'm not too keen on the
>idea. 
>

If your software has no problem with short reads and records
being split into two, then don't bother page aligning.
But, if you want to read exectly what you know you wrote then
alignment is a must.

Carl