svn commit: r364193 - head/sys/riscv/riscv

[Mitchell Horne]

Author: mhorne
Date: Thu Aug 13 14:21:05 2020
New Revision: 364193
URL: https://svnweb.freebsd.org/changeset/base/364193

Log:
  Enable interrupts while handling traps
  
  I observed hangs post-r362977 in QEMU with -smp 2, in which one thread
  would acquire write access to an rm_lock (sysctllock) and get stuck
  waiting in smp_rendezvous_cpus while the other CPU was servicing a trap.
  The other thread was waiting for read access to the same lock, thus
  causing deadlock.
  
  It's clear that this is just one symptom of a larger problem. The
  general expectation of MI kernel code is that interrupts are enabled.
  Violating this assumption will at best create some additional latency,
  but otherwise might cause locking or other unforeseen issues. All other
  architectures do so for some subset of trap values, but this somehow got
  missed in the RISC-V port. Enable interrupts now during kernel page
  faults and for all user trap types.
  
  The code in exception.S already knows to disable interrupts while
  handling the return from exception, so there are no changes required
  there.
  
  Reviewed by:	jhb, markj
  MFC after:	2 weeks
  Differential Revision:	https://reviews.freebsd.org/D26017

Modified:
  head/sys/riscv/riscv/trap.c

Modified: head/sys/riscv/riscv/trap.c
==============================================================================
--- head/sys/riscv/riscv/trap.c	Thu Aug 13 14:17:36 2020	(r364192)
+++ head/sys/riscv/riscv/trap.c	Thu Aug 13 14:21:05 2020	(r364193)
@@ -198,14 +198,22 @@ data_abort(struct trapframe *frame, int usermode)
 	    "Kernel page fault") != 0)
 		goto fatal;
 
-	if (usermode)
+	if (usermode) {
 		map = &td->td_proc->p_vmspace->vm_map;
-	else if (stval >= VM_MAX_USER_ADDRESS)
-		map = kernel_map;
-	else {
-		if (pcb->pcb_onfault == 0)
-			goto fatal;
-		map = &td->td_proc->p_vmspace->vm_map;
+	} else {
+		/*
+		 * Enable interrupts for the duration of the page fault. For
+		 * user faults this was done already in do_trap_user().
+		 */
+		intr_enable();
+
+		if (stval >= VM_MAX_USER_ADDRESS) {
+			map = kernel_map;
+		} else {
+			if (pcb->pcb_onfault == 0)
+				goto fatal;
+			map = &td->td_proc->p_vmspace->vm_map;
+		}
 	}
 
 	va = trunc_page(stval);
@@ -324,6 +332,7 @@ do_trap_user(struct trapframe *frame)
 		riscv_cpu_intr(frame);
 		return;
 	}
+	intr_enable();
 
 	CTR3(KTR_TRAP, "do_trap_user: curthread: %p, sepc: %lx, frame: %p",
 	    curthread, frame->tf_sepc, frame);