svn commit: r359367 - head/usr.sbin/bhyve

[Chuck Tuffli]

Author: chuck
Date: Fri Mar 27 15:28:27 2020
New Revision: 359367
URL: https://svnweb.freebsd.org/changeset/base/359367

Log:
  bhyve: fix NVMe emulation update of SQHD
  
  The SQHD field of a Completion Queue entry indicates the current
  Submission Queue head pointer value. The head pointer represents the
  next entry to be consumed and is updated after consuming the current
  entry.
  
  In the Admin queue processing, the current code updates the head pointer
  after reporting the value to the host via the SQHD. This gives the
  impression that the Controller is perpetually one command behind in its
  processing of the Admin SQ. And while this doesn't appear to bother some
  initiators, it is wrong.
  
  Fix is to update the SQ head pointer prior to writing the SQHD value in
  the completion.
  
  While here, fix missed update of dword 0 (cdw0) in the completion
  message.
  
  Reported by:	khng300
  Reviewed by:	jhb, imp
  Approved by:	jhb (maintainer)
  MFC after:	2 weeks
  Differential Revision: https://reviews.freebsd.org/D24083

Modified:
  head/usr.sbin/bhyve/pci_nvme.c

Modified: head/usr.sbin/bhyve/pci_nvme.c
==============================================================================
--- head/usr.sbin/bhyve/pci_nvme.c	Fri Mar 27 15:28:22 2020	(r359366)
+++ head/usr.sbin/bhyve/pci_nvme.c	Fri Mar 27 15:28:27 2020	(r359367)
@@ -1153,7 +1153,8 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u
 			    cmd->opc));
 			pci_nvme_status_genc(&compl.status, NVME_SC_INVALID_OPCODE);
 		}
-	
+		sqhead = (sqhead + 1) % sq->size;
+
 		if (NVME_COMPLETION_VALID(compl)) {
 			struct nvme_completion *cp;
 			int phase;
@@ -1170,7 +1171,6 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u
 
 			cq->tail = (cq->tail + 1) % cq->size;
 		}
-		sqhead = (sqhead + 1) % sq->size;
 	}
 
 	DPRINTF(("setting sqhead %u", sqhead));
@@ -1285,8 +1285,9 @@ pci_nvme_set_completion(struct pci_nvme_softc *sc,
 
 	compl = &cq->qbase[cq->tail];
 
-	compl->sqhd = atomic_load_acq_short(&sq->head);
+	compl->cdw0 = cdw0;
 	compl->sqid = sqid;
+	compl->sqhd = atomic_load_acq_short(&sq->head);
 	compl->cid = cid;
 
 	// toggle phase