Bacula fails on FreeBSD 10.x / "mt fsf" infinitely proceeds

[Kenneth D. Merry]

On Wed, Jul 30, 2014 at 08:03:31 +0200, Joerg Wunsch wrote:
> As Kenneth D. Merry wrote:
> 
> > Ahh.  Now I know who to send sa(4) driver patches to test.  :)
> 
> The only thing is that I don't love to reboot my main server machine
> gratuitously.  But everything I could do on old scratch hardware is
> possible.  I've also got a Tandberg SLR5 still around, I simply can't
> throw it away ...

I know what you mean. :)  (I've got 3 standalone tape drives and 2 tape
libraries that I haven't been able to part with yet...)

> (Replacing the L9 by some SAS-attached LTO-x library is already
> planned.)

Yep, you'll get much more capacity that way.

> > {black-pearl:/usr/home/kenm:!:0} dtrace -s tapetest.d
> > dtrace: script 'tapetest.d' matched 1 probe
> > CPU     ID                    FUNCTION:NAME
> >   0    401                    saerror:entry Opcode 08, Status 0xc Data: Len 524290, Resid 2, Sense: Len 252, Resid 0
> > 
> >   0    401                    saerror:entry Opcode 08, Status 0xc Data: Len 524290, Resid 2, Sense: Len 252, Resid 0
> 
> Here's the DTrace output, first a dd with a blocksize of 64 KiB (on a
> standard tar tape with 10 KiB records), then I did a rewind, followed
> by "mt fsf 32767".
> 
> root at uriah:~ # dtrace -s /tmp/tapetest.d
> dtrace: script '/tmp/tapetest.d' matched 1 probe
> CPU     ID                    FUNCTION:NAME
>   3    406                    saerror:entry Opcode 08, Status 0xc Data: Len 65536, Resid 55296, Sense: Len 252, Resid 223
> 
>   0    406                    saerror:entry Opcode 11, Status 0xc Data: Len 0, Resid 0, Sense: Len 252, Resid 223
> 
> It seems we are getting no residual for the SPACE command?!

The SPACE command doesn't send data, so it makes sense that there is no
data length and no data residual.

The sense length and residual show that we have 29 bytes of sense data.
That is plenty of length to allow for us to have a look at the info field,
which is in bytes 3-6.

I have attached an updated DTrace script.  This will print out the info
field, and we'll see what the drive is returning in the info field.

If the residual value in the info field is correct, then we may have a
problem with the way that scsi_get_sense_info() is handling it.  If the
value in the info field is incorrect, then we'll probably need to look at
the sym(4) driver.

Ken
-- 
Kenneth Merry
ken at FreeBSD.ORG
-------------- next part --------------
/*-
 * Copyright (c) 2014 Spectra Logic Corporation
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions, and the following disclaimer,
 *    without modification.
 * 2. Redistributions in binary form must reproduce at minimum a disclaimer
 *    substantially similar to the "NO WARRANTY" disclaimer below
 *    ("Disclaimer") and any redistribution must be conditioned upon
 *    including a substantially similar Disclaimer requirement for further
 *    binary redistribution.
 *
 * NO WARRANTY
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * HOLDERS OR CONTRIBUTORS BE LIABLE FOR SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
 * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
 * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGES.
 *
 * Authors: Ken Merry           (Spectra Logic Corporation)
 *
 * $FreeBSD$
 */

#pragma D option cleanrate=5000hz
#pragma D option dynvarsize=8192000

/*
 * This is a simple DTrace script to print out basic information about
 * calls to saerror().  This was originally written to help diagnose
 * problems with the sense residual, but could be used for other purposes.
 *
 * Note that CAM_STATUS_MASK == 0x3f and CAM_SCSI_STATUS_ERROR == 0x0c.
 * DTrace doesn't seem to decode the enum, so we just use hard-coded values
 * here.
 */
fbt:kernel:saerror:entry
/(args[0]->ccb_h.status & 0x3f) == 0x0c/
{
	printf("Opcode %02x, Status %#x Data: Len %u, Resid %u, Sense: Len %u, Resid %u\n",
	       args[0]->csio.cdb_io.cdb_bytes[0],
	       args[0]->ccb_h.status & 0x3f, args[0]->csio.dxfer_len,
	       args[0]->csio.resid, args[0]->csio.sense_len,
	       args[0]->csio.sense_resid);
}
fbt:kernel:scsi_get_sense_info:entry
/(arg1 >= 7)/
{
	printf("Sense info: %x,%x,%x,%x\n", args[0]->sense_buf[2],
	       args[0]->sense_buf[3], args[0]->sense_buf[4],
	       args[0]->sense_buf[5]);
}