add BIO_NORETRY flag, implement support in ata_da, use in ZFS vdev_geom

[Warner Losh]

On Fri, Nov 24, 2017 at 3:30 AM, Andriy Gapon <avg at freebsd.org> wrote:

>
> https://reviews.freebsd.org/D13224
>
> Anyone interested is welcome to join the review.
>

I think it's a really bad idea. It introduces a 'one-size-fits-all' notion
of QoS that seems misguided. It conflates a shorter timeout with don't
retry. And why is retrying bad? It seems more a notion of 'fail fast' or so
other concept. There's so many other ways you'd want to use it. And it uses
the same return code (EIO) to mean something new. It's generally meant 'The
lower layers have retried this, and it failed, do not submit it again as it
will not succeed' with 'I gave it a half-assed attempt, and that failed,
but resubmission might work'. This breaks a number of assumptions in the
BUF/BIO layer as well as parts of CAM even more than they are broken now.

So let's step back a bit: what problem is it trying to solve?

Warner