svn commit: r200088 - head/sys/dev/bge

[Pyun YongHyeon]

Author: yongari
Date: Thu Dec  3 23:57:06 2009
New Revision: 200088
URL: http://svn.freebsd.org/changeset/base/200088

Log:
  Add workaround to overcome hardware limitation which allows only a
  single outstanding DMA read operation. Most controllers targeted to
  client with PCIe bus interface(e.g. BCM5761) may have this
  limitation. All controllers for servers does not have this
  limitation.
  Collapsing mbuf chains to reduce number of memory reads before
  transmitting was most effective way to workaround this. I got about
  940Mbps from 850Mbps with mbuf collapsing on BCM5761. However it
  takes a lot of CPU cycles to collapse mbuf chains so add tunable to
  control the number of allowed TX buffers before collapsing. The
  default value is 0 which effectively disables the forced collapsing.
  For most cases 2 would yield best performance(about 930Mbps)
  without much sacrificing CPU cycles.
  Note the collapsing is only activated when the controller is on
  PCIe bus and the frame does not need TSO operation. TSO does not
  seem to suffer from the hardware limitation because the payload
  size is much bigger than normal IP datagram.
  Thanks to davidch@ who told me the limitation of client controllers
  and actually gave possible workarounds to mitigate the limitation.
  
  Reviewed by:	davidch, marius

Modified:
  head/sys/dev/bge/if_bge.c

Modified: head/sys/dev/bge/if_bge.c
==============================================================================
--- head/sys/dev/bge/if_bge.c	Thu Dec  3 23:24:12 2009	(r200087)
+++ head/sys/dev/bge/if_bge.c	Thu Dec  3 23:57:06 2009	(r200088)
@@ -483,12 +483,29 @@ DRIVER_MODULE(bge, pci, bge_driver, bge_
 DRIVER_MODULE(miibus, bge, miibus_driver, miibus_devclass, 0, 0);
 
 static int bge_allow_asf = 1;
+/*
+ * A common design characteristic for many Broadcom client controllers
+ * is that they only support a single outstanding DMA read operation
+ * on the PCIe bus. This means that it will take twice as long to fetch
+ * a TX frame that is split into header and payload buffers as it does
+ * to fetch a single, contiguous TX frame (2 reads vs. 1 read). For
+ * these controllers, coalescing buffers to reduce the number of memory
+ * reads is effective way to get maximum performance(about 940Mbps).
+ * Without collapsing TX buffers the maximum TCP bulk transfer
+ * performance is about 850Mbps. However forcing coalescing mbufs
+ * consumes a lot of CPU cycles, so leave it off by default.
+ */
+static int bge_forced_collapse = 0;
 
 TUNABLE_INT("hw.bge.allow_asf", &bge_allow_asf);
+TUNABLE_INT("hw.bge.forced_collapse", &bge_forced_collapse);
 
 SYSCTL_NODE(_hw, OID_AUTO, bge, CTLFLAG_RD, 0, "BGE driver parameters");
 SYSCTL_INT(_hw_bge, OID_AUTO, allow_asf, CTLFLAG_RD, &bge_allow_asf, 0,
 	"Allow ASF mode if available");
+SYSCTL_INT(_hw_bge, OID_AUTO, forced_collapse, CTLFLAG_RD, &bge_forced_collapse,
+	0, "Number of fragmented TX buffers of a frame allowed before "
+	"forced collapsing");
 
 #define	SPARC64_BLADE_1500_MODEL	"SUNW,Sun-Blade-1500"
 #define	SPARC64_BLADE_1500_PATH_BGE	"/pci at 1f,700000/network at 2"
@@ -3915,6 +3932,26 @@ bge_encap(struct bge_softc *sc, struct m
 			csum_flags |= BGE_TXBDFLAG_IP_FRAG;
 	}
 
+	if ((m->m_pkthdr.csum_flags & CSUM_TSO) == 0 &&
+	    bge_forced_collapse > 0 && (sc->bge_flags & BGE_FLAG_PCIE) != 0 &&
+	    m->m_next != NULL) {
+		/*
+		 * Forcedly collapse mbuf chains to overcome hardware
+		 * limitation which only support a single outstanding
+		 * DMA read operation.
+		 */
+		if (bge_forced_collapse == 1)
+			m = m_defrag(m, M_DONTWAIT);
+		else
+			m = m_collapse(m, M_DONTWAIT, bge_forced_collapse);
+		if (m == NULL) {
+			m_freem(*m_head);
+			*m_head = NULL;
+			return (ENOBUFS);
+		}
+		*m_head = m;
+	}
+
 	map = sc->bge_cdata.bge_tx_dmamap[idx];
 	error = bus_dmamap_load_mbuf_sg(sc->bge_cdata.bge_tx_mtag, map, m, segs,
 	    &nsegs, BUS_DMA_NOWAIT);