aac(4) resource FIB starvation on BUS scan revisited

[Jung-uk Kim]

On Monday 07 December 2009 07:47 pm, Scott Long wrote:
> On Dec 7, 2009, at 5:31 PM, Jung-uk Kim wrote:
> > On Monday 07 December 2009 05:30 pm, Alexander Sack wrote:
> >> On Mon, Dec 7, 2009 at 4:42 PM, Alexander Sack
> >> <pisymbol at gmail.com>
> >
> > wrote:
> >>> Folks:
> >>>
> >>> I posted a similar thread on freebsd-scsi only to realize that
> >>> scottl had fixed my first issue during some MP CAM cleanup with
> >>> respect to a race during resource allocation issues on a later
> >>> version of the driver we are using (I believe we did the same
> >>> thing to resolve a lock issue on bootup).
> >>>
> >>> However on my RELENG_8 box with (2) Adaptec 5085s connected to
> >>> some JBODs (9TB each) I still have a FIB starvation issue
> >>> during the LUN scan:
> >>>
> >>> The number of FIBs allocated to this card is 512 (older cards
> >>> are 256).  The max_target per bus is 287.  On a six channel
> >>> controller with a BUS scan done in parallel I see a lot of
> >>> this:
> >>>
> >>> ...
> >>> (probe501:aacp1:0:214:0): Request Requeued
> >>> (probe501:aacp1:0:214:0): Retrying Command
> >>> (probe520:aacp1:0:233:0): Request Requeued
> >>> (probe520:aacp1:0:233:0): Retrying Command
> >>> (probe528:aacp1:0:241:0): Request Requeued
> >>> (probe528:aacp1:0:241:0): Retrying Command
> >>> (probe540:aacp1:0:253:0): Request Requeued
> >>> (probe540:aacp1:0:253:0): Retrying Command
> >>> (probe541:aacp1:0:254:0): Request Requeued
> >>> (probe541:aacp1:0:254:0): Retrying Command
> >>> ....
> >>>
> >>> I think the driver is much happier with the following attached
> >>> patch (with dmesg).
> >>
> >> Patch again but this time not base-64 encoded:
> >
> > [SNIP!]
> >
> > I want it to be little conservative here, i.e., pre-allocating
> > half of max_fibs.  Will the attached patch work for you?
>
> The FIB allocation scheme was written when it was common for
> machines to only have 64MB of RAM and proportionally less KVA, so
> 256KB or 512KB was a lot of RAM to wire down.  Those days have
> probably passed.

So, what would do if you were hypothetically rewriting it today? :-)

Jung-uk Kim