aac(4) resource FIB starvation on BUS scan revisited

[Jung-uk Kim]

On Monday 07 December 2009 05:30 pm, Alexander Sack wrote:
> On Mon, Dec 7, 2009 at 4:42 PM, Alexander Sack <pisymbol at gmail.com> 
wrote:
> > Folks:
> >
> > I posted a similar thread on freebsd-scsi only to realize that
> > scottl had fixed my first issue during some MP CAM cleanup with
> > respect to a race during resource allocation issues on a later
> > version of the driver we are using (I believe we did the same
> > thing to resolve a lock issue on bootup).
> >
> > However on my RELENG_8 box with (2) Adaptec 5085s connected to
> > some JBODs (9TB each) I still have a FIB starvation issue during
> > the LUN scan:
> >
> > The number of FIBs allocated to this card is 512 (older cards are
> > 256).  The max_target per bus is 287.  On a six channel
> > controller with a BUS scan done in parallel I see a lot of this:
> >
> > ...
> > (probe501:aacp1:0:214:0): Request Requeued
> > (probe501:aacp1:0:214:0): Retrying Command
> > (probe520:aacp1:0:233:0): Request Requeued
> > (probe520:aacp1:0:233:0): Retrying Command
> > (probe528:aacp1:0:241:0): Request Requeued
> > (probe528:aacp1:0:241:0): Retrying Command
> > (probe540:aacp1:0:253:0): Request Requeued
> > (probe540:aacp1:0:253:0): Retrying Command
> > (probe541:aacp1:0:254:0): Request Requeued
> > (probe541:aacp1:0:254:0): Retrying Command
> > ....
> >
> > I think the driver is much happier with the following attached
> > patch (with dmesg).
>
> Patch again but this time not base-64 encoded:

[SNIP!]

I want it to be little conservative here, i.e., pre-allocating half of 
max_fibs.  Will the attached patch work for you?

Thanks,

Jung-uk Kim
-------------- next part --------------
Index: sys/dev/aac/aacvar.h
===================================================================
--- sys/dev/aac/aacvar.h	(revision 200237)
+++ sys/dev/aac/aacvar.h	(working copy)
@@ -57,13 +57,6 @@
 #define AAC_ADAPTER_FIBS	8
 
 /*
- * FIBs are allocated in page-size chunks and can grow up to the 512
- * limit imposed by the hardware.
- */
-#define AAC_PREALLOCATE_FIBS	128
-#define AAC_NUM_MGT_FIB		8
-
-/*
  * The controller reports status events in AIFs.  We hang on to a number of
  * these in order to pass them out to user-space management tools.
  */
Index: sys/dev/aac/aac.c
===================================================================
--- sys/dev/aac/aac.c	(revision 200237)
+++ sys/dev/aac/aac.c	(working copy)
@@ -604,7 +604,7 @@ aac_alloc(struct aac_softc *sc)
 	TAILQ_INIT(&sc->aac_fibmap_tqh);
 	sc->aac_commands = malloc(sc->aac_max_fibs * sizeof(struct aac_command),
 				  M_AACBUF, M_WAITOK|M_ZERO);
-	while (sc->total_fibs < AAC_PREALLOCATE_FIBS) {
+	while (sc->total_fibs < sc->aac_max_fibs / 2) {
 		if (aac_alloc_commands(sc) != 0)
 			break;
 	}
Index: sys/dev/aac/aac_cam.c
===================================================================
--- sys/dev/aac/aac_cam.c	(revision 200237)
+++ sys/dev/aac/aac_cam.c	(working copy)
@@ -260,8 +260,11 @@ aac_cam_action(struct cam_sim *sim, union ccb *ccb
 		cpi->hba_inquiry = PI_WIDE_16;
 		cpi->target_sprt = 0;
 
-		/* Resetting via the passthrough causes problems. */
-		cpi->hba_misc = PIM_NOBUSRESET;
+		/*
+		 * Resetting via the passthrough or parallel bus scan
+		 * causes problems.
+		 */
+		cpi->hba_misc = PIM_NOBUSRESET | PIM_SEQSCAN;
 		cpi->hba_eng_cnt = 0;
 		cpi->max_target = camsc->inf->TargetsPerBus;
 		cpi->max_lun = 8;	/* Per the controller spec */