bus_dmamap_create() breakage (alias 3Ware driver problems)

[Petr Lampa]

I have got similar problems with the new 3Ware driver like others, 
but on the second controller. After some debugging and playing with 
bus_dma_tag_create() etc. arguments (new driver is using 3 busdma_tags
instead 1 in old driver), I have located source of failure 
in bus_dmamap_create(). Here is the trouble spot:

$FreeBSD: src/sys/i386/i386/busdma_machdep.c,v 1.59.2.3 2004/12/04 05:
55:10 scottl Exp $

bus_dmamap_create(bus_dma_tag_t dmat, int flags, bus_dmamap_t *mapp)
{
...
	maxpages = MIN(MAX_BPAGES, Maxmem - atop(dmat->lowaddr));
	if ((dmat->flags & BUS_DMA_MIN_ALLOC_COMP) == 0 
	    || (dmat->map_count > 0 && total_bpages < maxpages)) {
		int pages;
		...
		pages = MAX(atop(dmat->maxsize), 1);

At this location maxpages=512, total_bpages=513, dmat->maxsize=131072, pages=32

		pages = MIN(maxpages - total_bpages, pages);

Here pages=-1!

		if (alloc_bounce_pages(dmat, pages) < pages)
			error = ENOMEM;

Here all kernel virtual memory is lost (or something like that),
the result was spurious page fault 12 or other panic after while.

I've loooked in CVS and it seems to me, that bus_dmamap_create() was
not updated after introduction of bounce zones. The first
test that leads to alloc_bounce_pages() is wrong in any case.
I've tried to fixed this and my solution works for me, but it needs some
inspection.

fix #1: change parenthesis here (I'am no sure, but the next
code only add pages when there are not sufficient pages allocated,
so it shouldn't be called if total_bpages>=maxpages at all):

	if (((dmat->flags & BUS_DMA_MIN_ALLOC_COMP) == 0 
	    || dmat->map_count > 0) && total_bpages < maxpages)) {

fix #2: don't check total_bpages, but bounce zone's bz->total_bpages

	bz = dmat->bounce_zone;
	if (((dmat->flags & BUS_DMA_MIN_ALLOC_COMP) == 0 
	    || dmat->map_count > 0) && bz->total_bpages < maxpages)) {

	...
	            pages = MIN(maxpages - bz->total_bpages, pages);

With this fix, my two 3Ware controllers are running again. Attached
diff is for 5.3-STABLE branch, but it should work also for 6-CURRENT branch
(busdma_machdep.c 1.70).

Petr Lampa

-- 
Computer Centre                             E-mail: lampa at fit.vutbr.cz
Faculty of Information Technology           Web: http://www.fit.vutbr.cz/
Brno University of Technology               Fax:  +420 54114-1270
Bozetechova 2, 612 66 Brno, Czech Republic  Phone: +420 54114-1225
-------------- next part --------------
*** busdma_machdep.c	Sat Dec  4 06:55:10 2004
--- /tmp/busdma_machdep.c	Fri Jan 28 10:37:35 2005
***************
*** 393,403 ****
--- 393,405 ----
  
  		/* Must bounce */
  		int maxpages;
+ 		struct bounce_zone *bz;
  
  		if (dmat->bounce_zone == NULL) {
  			if ((error = alloc_bounce_zone(dmat)) != 0)
  				return (error);
  		}
+ 		bz = dmat->bounce_zone;
  
  		*mapp = (bus_dmamap_t)malloc(sizeof(**mapp), M_DEVBUF,
  					     M_NOWAIT | M_ZERO);
***************
*** 415,422 ****
  		 * basis up to a sane limit.
  		 */
  		maxpages = MIN(MAX_BPAGES, Maxmem - atop(dmat->lowaddr));
! 		if ((dmat->flags & BUS_DMA_MIN_ALLOC_COMP) == 0
! 		 || (dmat->map_count > 0 && total_bpages < maxpages)) {
  			int pages;
  
  			if (dmat->lowaddr > bounce_lowaddr) {
--- 417,424 ----
  		 * basis up to a sane limit.
  		 */
  		maxpages = MIN(MAX_BPAGES, Maxmem - atop(dmat->lowaddr));
! 		if (((dmat->flags & BUS_DMA_MIN_ALLOC_COMP) == 0
! 		 || dmat->map_count > 0) && bz->total_bpages < maxpages) {
  			int pages;
  
  			if (dmat->lowaddr > bounce_lowaddr) {
***************
*** 428,434 ****
  				      "not implemented");
  			}
  			pages = MAX(atop(dmat->maxsize), 1);
! 			pages = MIN(maxpages - total_bpages, pages);
  			if (alloc_bounce_pages(dmat, pages) < pages)
  				error = ENOMEM;
  
--- 430,436 ----
  				      "not implemented");
  			}
  			pages = MAX(atop(dmat->maxsize), 1);
! 			pages = MIN(maxpages - bz->total_bpages, pages);
  			if (alloc_bounce_pages(dmat, pages) < pages)
  				error = ENOMEM;