CAM tags / reset problem

[Scott Long]

Ade Lovett wrote:
> The observed behaviour is this: there is a bus reset during device
> probing / negotiation during boot, and after the system is up many of
> the scsi devices are found to be running with dev_openings = 1, which
> is less than the mintags = 2 setting, even though tag queueing is
> supposedly enabled. Performance is impaired as a result.
> 
> The cause appears to be as follows: in cam_xpt.c, the routine
> xpt_dev_ccbq_resize saves, in dev->tag_saved_openings, the requested
> queue size _if and only if_ tag queueing is either enabled (SID_CmdQue
> is set) or is _scheduled_ to be enabled (CAM_DEV_TAG_AFTER_COUNT is
> set). However, the following sequence can occur:
> 
> 1) Device transfer settings are negotiated. The device can support
> tags, so CAM_DEV_TAG_AFTER_COUNT is set (tag queueing is _not_
> immediately enabled).
> 
> 2) Before enough commands have been sent to cause tag queueing to be
> started, the bus is reset and all transfer settings are reset. This
> results in xpt_set_transfer_settings being called, and in turn this
> clears SID_CmdQue and calls xpt_dev_ccbq_resize (to resize the queue
> down to its non-tagged size, typically 1), and only then clears
> CAM_DEV_TAG_AFTER_COUNT (which was of course set) and its associated
> count. This causes xpt_dev_ccbq_resize to save the requested queue
> size even though it does not relate to the desired size with tag
> queueing enabled.
> 
> 3) If tag-queueing is enabled again, e.g. after another negotiation,
> then the saved value (1) of tag_saved_openings is used rather than
> the correct value.
> 
> The fix seems to be to clear CAM_DEV_TAG_AFTER_COUNT _before_ resizing
> the queue, as per the following patch:
> 
> http://people.FreeBSD.org/~ade/sys-dev-cam-xpt.c
> 
> I'd like permission to commit this to HEAD, followed by a relatively  
> quick MFC to RELENG_6 (I know I've missed BETA5, but would like to  get 
> it into -RELEASE)
> 
> -aDe
> 

It's a bit late tonight for me to think coherently about this and give
the patch a good review, but I promise that it will happen.  Maybe
Justin, Ken, Matt, or Nate can also step in and look at it.  Are you 
saying that there are multiple bus resets happening in relatively quick
succession on each bus during the inital probe?  Are you observing this
with only a particular controller family, or with multiple ones?

Scott