sa(4) driver changes available for test
Dan Langille
dan at langille.org
Sat Feb 28 16:48:04 UTC 2015
> On Feb 18, 2015, at 7:13 PM, Kenneth D. Merry <ken at freebsd.org> wrote:
>
>
> I have updated the patches.
>
> I have removed the XPT_DEV_ADVINFO changes from the patches to head, since
> I committed those separately.
>
> I have (hopefully) fixed the build for the stable/10 patches by MFCing
> dependencies. (One of them mav did for me, thanks!)
>
> Rough draft commit message:
>
> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150218.1.txt
>
> The patches against FreeBSD/head as of SVN revision 278975:
>
> http://people.freebsd.org/~ken/sa_changes.20150218.1.txt
>
> And (untested) patches against FreeBSD stable/10 as of SVN revision 278974:
>
> http://people.freebsd.org/~ken/sa_changes.stable_10.20150218.1.txt
Not sure what I've done wrong here.
I've applied your patches and run make buildworld against:
[root at cuppy:/usr/src] # svn info
Path: .
Working Copy Root Path: /usr/src
URL: https://svn0.us-west.freebsd.org/base/stable/10
Relative URL: ^/stable/10
Repository Root: https://svn0.us-west.freebsd.org/base
Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Revision: 278721
Node Kind: directory
Schedule: normal
Last Changed Author: ngie
Last Changed Rev: 278721
Last Changed Date: 2015-02-13 21:46:22 +0000 (Fri, 13 Feb 2015)
But I get:
rm -f .depend GPATH GRTAGS GSYMS GTAGS
===> lib/libmp (cleandir)
rm -f Version.map libmp.3.gz libmp.3.cat.gz
rm -f a.out mpasbn.o mpasbn.o.tmp
rm -f mpasbn.po mpasbn.po.tmp
rm -f mpasbn.So mpasbn.so mpasbn.So.tmp
rm -f libmp.so
rm -f libmp.a libmp_p.a libmp.so.7
rm -f Version.map
rm -f .depend GPATH GRTAGS GSYMS GTAGS
===> lib/libmt (cleandir)
cd: /usr/src/lib/libmt: No such file or directory
*** Error code 2
Stop.
make[3]: stopped in /usr/src/lib
*** Error code 1
Stop.
make[2]: stopped in /usr/src
*** Error code 1
Stop.
make[1]: stopped in /usr/src
*** Error code 1
Stop.
make: stopped in /usr/src
>
> Thanks,
>
> Ken
>
> On Fri, Feb 13, 2015 at 17:32:32 -0700, Kenneth D. Merry wrote:
>>
>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver
>> that I'm planning to commit in the near future.
>>
>> A description of the changes is here and below in this message.
>>
>> If you have tape hardware and the inclination, I'd appreciate testing and
>> feedback.
>>
>> ============
>> Rough draft commit message:
>>
>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt
>>
>> The patches against FreeBSD/head as of SVN revision 278706:
>>
>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt
>>
>> And (untested) patches against FreeBSD stable/10 as of SVN revision 278721.
>>
>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt
>> ============
>>
>> The intent is to get the tape infrastructure more up to date, so we can
>> support LTFS and more modern tape drives:
>>
>> http://www.ibm.com/systems/storage/tape/ltfs/
>>
>> I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends
>> on the patches linked above. It isn't fully cleaned up and ready for
>> redistribution. If you're interested, though, let me know and I'll tell
>> you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or
>> TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older
>> drives don't have the necessary features to support LTFS.
>>
>> The commit message below outlines most of the changes.
>>
>> A few comments:
>>
>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately.
>>
>> 2. The XML output is similar to what GEOM and CTL do. It would be nice to
>> figure out how to put a standard schema on it so that standard tools
>> could read it. I don't know how feasible that is, since I haven't
>> time to dig into it. If anyone has suggestions on whether that is
>> feasible or advisable, I'd appreciate feedback.
>>
>> 3. I have tested with a reasonable amount of tape hardware (see below for a
>> list), but more testing and feedback would be good.
>>
>> 4. Standard 'mt status' output looks like this:
>>
>> # mt -f /dev/nsa3 status -v
>> Drive: sa3: <IBM ULTRIUM-HH6 E4J1> Serial Number: 101500520A
>> ---------------------------------
>> Mode Density Blocksize bpi Compression
>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff)
>> ---------------------------------
>> Current Driver State: at rest.
>> ---------------------------------
>> Partition: 0 Calc File Number: 0 Calc Record Number: 0
>> Residual: 0 Reported File Number: 0 Reported Record Number: 0
>> Flags: BOP
>>
>> 5. 'mt status -v' looks like this:
>>
>> # mt -f /dev/nsa3 status -v
>> Drive: sa3: <IBM ULTRIUM-HH6 E4J1> Serial Number: 101500520A
>> ---------------------------------
>> Mode Density Blocksize bpi Compression
>> Current: 0x5a:LTO-6 variable 384607 enabled (0xff)
>> ---------------------------------
>> Current Driver State: at rest.
>> ---------------------------------
>> Partition: 0 Calc File Number: 0 Calc Record Number: 0
>> Residual: 0 Reported File Number: 0 Reported Record Number: 0
>> Flags: BOP
>> ---------------------------------
>> Tape I/O parameters:
>> Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes
>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes
>> Maximum block size supported by tape drive and media (max_blk): 8388608 bytes
>> Minimum block size supported by tape drive and media (min_blk): 1 bytes
>> Block granularity supported by tape drive and media (blk_gran): 0 bytes
>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes
>>
>> 6. Existing applications should work without changes. If not, please let
>> me know. Hopefully they will move over time to the new interfaces.
>>
>> 7. There are lots of additional features that could be added later.
>> Append-only support, encryption, more log pages, etc.
>>
>> 8. I have SCSI READ ATTRIBUTE changes for camcontrol(8) that will go in
>> separately. These changes allow displaying the contents of the MAM
>> (Medium Auxiliary Memory) chips on LTO, TS and other modern tape drives.
>> These are good, and a future possible direction is adding attributes
>> to the status XML from the sa(4) driver.
>>
>> ============
>> Significant upgrades to sa(4) and mt(1).
>>
>> The primary focus of these changes is to modernize FreeBSD's
>> tape infrastructure so that we can take advantage of some of the
>> features of modern tape drives and allow support for LTFS.
>>
>> Significant changes and new features include:
>>
>> o sa(4) driver status and parameter information is now exported via an
>> XML structure. This will allow for changes and improvements later
>> on that will not break userland applications. The old MTIOCGET
>> status ioctl remains, so applications using the existing interface
>> will not break.
>>
>> o 'mt status' now reports drive-reported tape position information
>> as well as the previously available calculated tape position
>> information. These numbers will be different at times, because
>> the drive-reported block numbers are relative to BOP (Beginning
>> of Partition), but the block numbers calculated previously via
>> sa(4) (and still provided) are relative to the last filemark.
>> Both numbers are now provided. 'mt status' now also shows the
>> drive INQUIRY information, serial number and any position flags
>> (BOP, EOT, etc.) provided with the tape position information.
>> 'mt status -v' adds information on the maximum possible I/O size,
>> and the underlying values used to calculate it.
>>
>> o The extra sa(4) /dev entries (/dev/saN.[0-3]) have been removed.
>>
>> The extra devices were originally added as place holders for
>> density-specific device nodes. Some OSes (NetBSD, NetApp's OnTap
>> and Solaris) have had device nodes that, when you write to them,
>> will automatically select a given density for particular tape drives.
>>
>> This is a convenient way of switching densities, but it was never
>> implemented in FreeBSD. Only the device nodes were there, and that
>> sometimes confused users.
>>
>> For modern tape devices, the density is generally not selectable
>> (e.g. with LTO) or defaults to the highest availble density when
>> the tape is rewritten from BOT (e.g. TS11X0). So, for most users,
>> density selection won't be necessary. If they do need to select
>> the density, it is easy enough to use 'mt density' to change it.
>>
>> o Protection information is now supported. This is either a
>> Reed-Solomon CRC or CRC32 that is included at the end of each block
>> read and written. On write, the tape drive verifies the CRC, and
>> on read, the tape drive provides a CRC for the userland application
>> to verify.
>>
>> o New, extensible tape driver parameter get/set interface.
>>
>> o Density reporting information. For drives that support it,
>> 'mt getdensity' will show detailed information on what formats the
>> tape drive supports, and what formats the tape drive supports.
>>
>> o Some mt(1) functionality moved into a new mt(3) library so that
>> external applications can reuse the code.
>>
>> o The new mt(3) library includes helper routines to aid in parsing
>> the XML output of the sa(4) driver, and build a tree of driver
>> metadata.
>>
>> o Support for the MTLOAD (load a tape in the drive) and MTWEOFI
>> (write filemark immediate) ioctls needed by IBM's LTFS
>> implementation.
>>
>> o Improve device departure behavior for the sa(4) driver. The previous
>> implementation led to hangs when the device was open.
>>
>> o This has been tested on the following types of drives:
>> IBM TS1150
>> IBM TS1140
>> IBM LTO-6
>> IBM LTO-5
>> HP LTO-2
>> Seagate DDS-4
>> Quantum DLT-4000
>> Exabyte 8505
>> Sony DDS-2
>>
>> contrib/groff/tmac/doc-syms,
>> share/mk/bsd.libnames.mk,
>> lib/Makefile,
>> Add libmt.
>>
>> lib/libmt/Makefile,
>> lib/libmt/mt.3,
>> lib/libmt/mtlib.c,
>> lib/libmt/mtlib.h,
>> New mt(3) library that contains functions moved from mt(1) and
>> new functions needed to interact with the updated sa(4) driver.
>>
>> This includes XML parser helper functions that application writers
>> can use when writing code to query tape parameters.
>>
>> rescue/rescue/Makefile:
>> Add -lmt to CRUNCH_LIBS.
>>
>> sys/cam/cam_ccb.h
>> Add a new flag value for the XPT_DEV_ADVINFO CCB, CDAI_FLAG_NONE.
>>
>> sbin/camcontrol/camcontrol.c,
>> sys/cam/scsi/scsi_da.c,
>> sys/cam/scsi/scsi_enc_ses.c,
>> sys/dev/mps/mps_sas.c:
>> Make sure the flags for the XPT_DEV_ADVINFO CCB are set correctly.
>> This prevents unintended attempts to set advanced information
>> values when XPT_DEV_ADVINFO CCBs are not pre-zeroed.
>>
>> src/share/man/man4/mtio.4
>> Clarify this man page a bit, and since it contains what is
>> essentially the mtio.h header file, add new ioctls and structure
>> definitions from mtio.h.
>>
>> src/share/man/man4/sa.4
>> Update BUGS and maintainer section.
>>
>> sys/cam/scsi/scsi_all.c,
>> sys/cam/scsi/scsi_all.h:
>> Add SCSI SECURITY PROTOCOL IN/OUT CDB definitions and CDB building
>> functions.
>>
>> sys/cam/scsi/scsi_sa.c
>> sys/cam/scsi/scsi_sa.h
>> Many tape driver changes, largely outlined above.
>>
>> Increase the sa(4) driver read/write timeout from 4 to 32
>> minutes. This is based on the recommended values for IBM LTO
>> 5/6 drives. This may also avoid timeouts for other tape
>> hardware that can take a long time to do retries and error
>> recovery. Longer term, a better way to handle this is to ask
>> the drive for recommended timeout values using the REPORT
>> SUPPORTED OPCODES command. Modern IBM and Oracle tape drives
>> at least support that command, and it would allow for more
>> accurate timeout values.
>>
>> Add XML status generation. This is done with a series of
>> macros to eliminate as much duplicate code as possible. The
>> new XML-based status values are reported through the new
>> MTIOCEXTGET ioctl.
>>
>> Add XML driver parameter reporting, using the new MTIOCPARAMGET
>> ioctl.
>>
>> Add a new driver parameter setting interface, using the new
>> MTIOCPARAMSET and MTIOCSETLIST ioctls.
>>
>> Add a new MTIOCRBLIM ioctl to get block limits information.
>>
>> Add CCB/CDB building routines scsi_locate_16, scsi_locate_10,
>> and scsi_read_position_10().
>>
>> scsi_locate_10 implements the LOCATE command, as does the
>> existing scsi_set_position() command. It just supports
>> additional arguments and features. If/when we figure out a
>> good way to provide backward compatibility for older
>> applications using the old function API, we can just revamp
>> scsi_set_position(). The same goes for
>> scsi_read_position_10() and the existing scsi_read_position()
>> function.
>>
>> Revamp sasetpos() to take the new mtlocate structure as an
>> argument. It now will use either scsi_locate_10() or
>> scsi_locate_16(), depending upon the arguments the user
>> supplies. As before, once we change position we don't have a
>> clear idea of what the current logical position of the tape
>> drive is.
>>
>> For tape drives that support long form position data, we
>> read the current position and store that for later reporting
>> after changing the position. This should help applications
>> like Bacula speed tape access under FreeBSD once they are
>> modified to support the new ioctls.
>>
>> Add a new quirk, SA_QUIRK_NO_LONG_POS, that is set for all
>> drives that report SCSI-2 or older, as well as drives that
>> report an Illegal Request type error for READ POSITION with
>> the long format. So we should automatically detect drives
>> that don't support the long form and stop asking for it after
>> an initial try.
>>
>> Add a partition number to the sa(4) softc.
>>
>> Improve device departure handling. The previous implementation
>> led to hangs when the device was open.
>>
>> If an application had the sa(4) driver open, and attempted to
>> close it after it went away, the cam_periph_release() call in
>> saclose() would cause the periph to get destroyed because that
>> was the last reference to it. Because destroy_dev() was
>> called from the sa(4) driver's cleanup routine (sacleanup()),
>> and would block waiting for the close to happen, a deadlock
>> would result.
>>
>> So instead of calling destroy_dev() from the cleanup routine,
>> call destroy_dev_sched_cb() from saoninvalidate() and wait for
>> the callback.
>>
>> Acquire a reference for devfs in saregister(), and release it
>> in the new sadevgonecb() routine when all devfs devices for
>> the particular sa(4) driver instance are gone.
>>
>> Add a new function, sasetupdev(), to centralize setting
>> per-instance devfs device parameters instead of repeating the
>> code in saregister().
>>
>> Add an open count to the softc, so we know how many
>> peripheral driver references are a result of open
>> sessions.
>>
>> Add the D_TRACKCLOSE flag to the cdevsw flags so
>> that we get a 1:1 mapping of open to close calls
>> instead of a N:1 mapping.
>>
>> This should be a no-op for everything except the
>> control device, since we don't allow more than one
>> open on non-control devices.
>>
>> However, since we do allow multiple opens on the
>> control device, the combination of the open count
>> and the D_TRACKCLOSE flag should result in an
>> accurate peripheral driver reference count, and an
>> accurate open count.
>>
>> The accurate open count allows us to release all
>> peripheral driver references that are the result
>> of open contexts once we get the callback from devfs.
>>
>> sys/sys/mtio.h:
>> Add a number of new mt(4) ioctls and the requisite data
>> structures. None of the existing interfaces been removed
>> or changed.
>>
>> This includes definitions for the following new ioctls:
>>
>> MTIOCRBLIM /* get block limits */
>> MTIOCEXTLOCATE /* seek to position */
>> MTIOCEXTGET /* get tape status */
>> MTIOCPARAMGET /* get tape params */
>> MTIOCPARAMSET /* set tape params */
>> MTIOCSETLIST /* set N params */
>>
>> usr.bin/mt/Makefile:
>> mt(1) now depends on libmt, libsbuf and libbsdxml.
>>
>> usr.bin/mt/mt.1:
>> Document new mt(1) features and subcommands.
>>
>> usr.bin/mt/mt.c:
>> Implement support for mt(1) subcommands that need to
>> use getopt(3) for their arguments.
>>
>> Implement a new 'mt status' command to replace the old
>> 'mt status' command. The old status command has been
>> renamed 'ostatus'.
>>
>> The new status function uses the MTIOCEXTGET ioctl, and
>> therefore parses the XML data to determine drive status.
>> The -x argument to 'mt status' allows the user to dump out
>> the raw XML reported by the kernel.
>>
>> The new status display is mostly the same as the old status
>> display, except that it doesn't print the redundant density
>> mode information, and it does print the current partition
>> number and position flags.
>>
>> Add a new command, 'mt locate', that will supersede the
>> old 'mt setspos' and 'mt sethpos' commands. 'mt locate'
>> implements all of the functionality of the MTIOCEXTLOCATE
>> ioctl, and allows the user to change the logical position
>> of the tape drive in a number of ways. (Partition,
>> block number, file number, set mark number, end of data.)
>> The immediate bit and the explicit address bits are
>> implemented, but not documented in the man page.
>>
>> Add a new 'mt weofi' command to use the new MTWEOFI ioctl.
>> This allows the user to ask the drive to write a filemark
>> without waiting around for the operation to complete.
>>
>> Add a new 'mt getdensity' command that gets the XML-based
>> tape drive density report from the sa(4) driver and displays
>> it. This uses the SCSI REPORT DENSITY SUPPORT command
>> to get comprehensive information from the tape drive about
>> what formats it is able to read and write.
>>
>> Add a new 'mt protect' command that allows getting and setting
>> tape drive protection information. The protection information
>> is a CRC tacked on to the end of every read/write from and to
>> the tape drive.
>>
>> Sponsored by: Spectra Logic
>> MFC after: 1 month
>>
>> Thanks,
>>
>> Ken
>> --
>> Kenneth Merry
>> ken at FreeBSD.ORG
>
> --
> Kenneth Merry
> ken at FreeBSD.ORG
> _______________________________________________
> freebsd-scsi at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-scsi
> To unsubscribe, send any mail to "freebsd-scsi-unsubscribe at freebsd.org"
—
Dan Langille
http://langille.org/
More information about the freebsd-scsi
mailing list