Tape drive handling on FreeBSD

[Kern Sibbald]

On Monday 19 December 2005 20:12, Matthew Jacob wrote:
> On 12/19/05, Kern Sibbald <kern at sibbald.com> wrote:
> > On Sunday 18 December 2005 23:48, Matthew Jacob wrote:
> > > The first paragraph of mtio(4) describes the usage of a control device.
> >
> > Yes, thanks. I've already read that and other sections of the man pages,
> > and
> > it is true that the .ctl device allows one to figure out if a tape drive
> > really exists or not.  The problem is:
> >
> > 1. It is system dependent -- I would like to see some standardization of
> > the
> > Unix API from system to system where it is reasonable, practical, and
> > doesn't
> > break everything.
>
> .....
>
> My problem is that if I change the FreeBSD API, it breaks current usage.

I'm proposing changes that I do not believe will break current usage.  

>
> Otherwise, there is no true tape device API in Unix. Look at any existing
> backup package that is multi-platform. It breaks each platform into
> different vectors that abstract the differences.

What a waste of programming time ...

>
> Don't get me wrong- I'm not promoting that there should differences. I'm
> observing that nearly thirty years of Unix shows that there are and that
> the way that those differences have been handled successfully by dozens of
> tape backup packages (albeit with grumbling).

Well, I'm one of the ones grumbling. Maybe after 30 years someone will listen.

>
> > 2. The errno that is currently returned by an empt tape drive during an
> > open()
> > is "incorrect" or at the least misleading.
>
> The value that Solaris returns (EIO) for trying to open a tape
> (!O_NONBLOCK) when no tape is loaded (i.e., fails TUR) is no less
> misleading.

Yes, I agree.  Linux returns ENOMEDIUM.  Too bad this isn't standard, but I 
can deal with that.

>
> Your complaint here is that ERRNO is not sufficient. It isn't.

Yes, I agree.

>
> > 3. The API that you currently have doesn't work correctly with Bacula,
> > which
> > was originally designed on Linux and Solaris -- it penalizes FreeBSD
> > Bacula
> > users.  Since the FreeBSD way of doing things is the minority in my
> > little Unix world, it is unlikely (though possible) that I will rewrite
> > my code.
>
> Well, that is your privilege.

I don't consider it a privilege just spending what little time I have on the 
high priority items.

>
> > 4. There are other shortcomings of the SCSI driver that penalize FreeBSD
> > users
> > compared to Solaris and Linux that I would like to discuss as well
> > (ability
> > to have the driver efficiently space to the end of the medium and keep
> > track
> > of the file position).
>
> Despite the fact that I'm the nominal maintainer, I probably won't do that
> much more. I put in a fair amount of time some years back, spent around
> 7500$ of my own personal money to buy tape drives to do testing with. 

Yes, I understand. I've been there too. Take a look at the Bacula 
"contribution" page some day when you are bored.

> That 
> said, the return value for this was more than what I got for writing the
> Solaris tape driver (which is shockingly and pathetically bad, but has also
> remained relatively unchanged for over 10 years). 

Maybe the Solaris drive is shockingly bad from your point of view, but from my 
point of view, it works, is efficient, and allows the user to control the 
drive in a reasonable way.

> Still, the things that 
> penalize FreeBSD are many- the tape driver is hardly a leading candidate.

I'm not a FreeBSD expert, but for me the tape drive is a good place to start. 
It doesn't need much ...

>
> > Actually, I am really shocked how poor the "Unix" tape interface is (this
> > has
> > nothing to do with FreeBSD in particular). There is no standard way to
> > set tape drive options, there is no standard way to figure out if a tape
> > drive exists and what options it has set for it (blocking, two eof,
> > density, compression, ...). Probably most annoying, there is no standard
> > way to know
> > if a tape is mounted or not.
>
> Yes.
>
> > My main interest is to try to discuss what possibilities exist for
> > changing
> > (unifying) the API (or there is also a small chance that I am missing
> > something ...).  If no possibility of discussion/changes exists, so be
> > it.
>
> a) If you can get a consensus amongs more FreeBSD tape clients than you
> that there *should* be convergence to, e.g., Linux, or Solaris, I certainly
> would be interested. Get the Amanda and Arkeia folks on board, and we might
> be able to get enough consensus to override the concerns of people even
> more conservative than I am.

It seems to me that the Amanda project is dead.  I seen no activity and their 
developers receive but do not respond to my emails.  I'd be willing to try, 
but I think the things I am proposing at this stage would change nothing for 
them -- see below.

>
> b) If you rewrite Bacula to actually be more platform independent you might
> serve yourself better. Having maintained some of the nsrmmd code for
> NetWorker/Legato, I *am* sympathetic to your concerns here. However, the
> other thing I've learned, very very very very painfully, is that making
> changes to existing tape interfaces (no matter how broken) will almost
> certainly enrage and break more than it fixes as everyone who uses said
> interfaces usually is caught by surprise when things change.

I cannot answer this because I already consider Bacula to be extremely 
platform independent simply because it uses very elementary features so it 
doesn't really need any platform dependent code.

>
> I suppose that we could have a separate SA device entry point that mandated
> different tape semantics- that might help minimize breakage at the risk of
> complexity. Part of the problem is that you probably are asking for a
> different state machine in the driver as well.

Not knowing the driver, I may be wrong here, but I hope I am not asking for a 
different state machine.  I am asking for some tweaks that would bring 
FreeBSD forward and more in line with some other platforms without breaking 
anything (I may be wrong though ...).

>
> Look- dialogue is good, but temper it with some realities please.

OK, here are a couple of real possibilities (at least from my point of view):

1. I believe that one can modify the FreeBSD tape API to become more 
compatible with Solaris and Linux without breaking any programs -- simply use 
their "trick" of allowing an open() in all cases to succeed if O_NONBLOCK is 
set.  I think this is an ugly kludge, but that is how they do it. If there is 
no tape in the drive, then return a file descriptor with no read and no write 
permission, and modify them when a tape is mounted.

2. Certainly someone can certainly figure out some magic ioctl() that allows 
the user to set a mode so that the SCSI driver will do a forward space file 
(big number) when an ioctl( EOM ) issued.  The driver can do it *much* more 
efficiently and quickly than a program can.  The penalty is a bit of speed, 
but the gain is that the correct file number can be returned.

Am I wrong?  Yes, when things work, I know it is hard to get excited about 
making changes.


-- 
Best regards,

Kern

  (">
  /\
  V_V