svn commit: r294062 - in head: share/man/man4 sys/dev/ioat

[Conrad E. Meyer]

Author: cem
Date: Fri Jan 15 01:34:43 2016
New Revision: 294062
URL: https://svnweb.freebsd.org/changeset/base/294062

Log:
  ioat(4): Add support for 'fence' bit with DMA_FENCE flag
  
  Some classes of IOAT hardware prefetch reads.  DMA operations that
  depend on the result of prior DMA operations must use the DMA_FENCE flag
  to prevent stale reads.
  
  (E.g., I've hit this personally on Broadwell-EP.  The Broadwell-DE has a
  different IOAT unit that is documented to not pipeline DMA operations.)
  
  Sponsored by:	EMC / Isilon Storage Division

Modified:
  head/share/man/man4/ioat.4
  head/sys/dev/ioat/ioat.c
  head/sys/dev/ioat/ioat.h

Modified: head/share/man/man4/ioat.4
==============================================================================
--- head/share/man/man4/ioat.4	Fri Jan 15 01:26:32 2016	(r294061)
+++ head/share/man/man4/ioat.4	Fri Jan 15 01:34:43 2016	(r294062)
@@ -24,7 +24,7 @@
 .\"
 .\" $FreeBSD$
 .\"
-.Dd January 7, 2016
+.Dd January 14, 2016
 .Dt IOAT 4
 .Os
 .Sh NAME
@@ -134,7 +134,7 @@ Null operations do nothing, but may be u
 mechanism.
 .Pp
 All operations can optionally trigger an interrupt at completion with the
-.Ar DMA_EN_INT
+.Ar DMA_INT_EN
 flag.
 For example, a user might submit multiple operations to the same channel and
 only enable an interrupt and callback for the last operation.
@@ -160,6 +160,17 @@ flag.
 .Ar DMA_NO_WAIT
 may return NULL.)
 .Pp
+Operations that depend on the result of prior operations should use
+.Ar DMA_FENCE .
+For example, such a scenario can happen when two related DMA operations are
+queued.
+First, a DMA copy to one location (A), followed directly by a DMA copy
+from A to B.
+In this scenario, some classes of I/OAT hardware may prefetch A for the second
+operation before it is written by the first operation.
+To avoid reading a stale value in sequences of dependent operations, use
+.Ar DMA_FENCE .
+.Pp
 All operations, as well as
 .Fn ioat_get_dmaengine ,
 can return NULL in special circumstances.

Modified: head/sys/dev/ioat/ioat.c
==============================================================================
--- head/sys/dev/ioat/ioat.c	Fri Jan 15 01:26:32 2016	(r294061)
+++ head/sys/dev/ioat/ioat.c	Fri Jan 15 01:34:43 2016	(r294062)
@@ -852,6 +852,8 @@ ioat_op_generic(struct ioat_softc *ioat,
 
 	if ((flags & DMA_INT_EN) != 0)
 		hw_desc->u.control_generic.int_enable = 1;
+	if ((flags & DMA_FENCE) != 0)
+		hw_desc->u.control_generic.fence = 1;
 
 	hw_desc->size = size;
 	hw_desc->src_addr = src;

Modified: head/sys/dev/ioat/ioat.h
==============================================================================
--- head/sys/dev/ioat/ioat.h	Fri Jan 15 01:26:32 2016	(r294061)
+++ head/sys/dev/ioat/ioat.h	Fri Jan 15 01:34:43 2016	(r294062)
@@ -46,7 +46,13 @@ __FBSDID("$FreeBSD$");
  * descriptor without blocking.
  */
 #define	DMA_NO_WAIT	0x2
-#define	DMA_ALL_FLAGS	(DMA_INT_EN | DMA_NO_WAIT)
+/*
+ * Disallow prefetching the source of the following operation.  Ordinarily, DMA
+ * operations can be pipelined on some hardware.  E.g., operation 2's source
+ * may be prefetched before operation 1 completes.
+ */
+#define	DMA_FENCE	0x4
+#define	DMA_ALL_FLAGS	(DMA_INT_EN | DMA_NO_WAIT | DMA_FENCE)
 
 /*
  * Hardware revision number.  Different hardware revisions support different