svn commit: r295563 - head/sys/arm64/arm64

[Andrew Turner]

Author: andrew
Date: Fri Feb 12 12:38:04 2016
New Revision: 295563
URL: https://svnweb.freebsd.org/changeset/base/295563

Log:
  Only update curthread and curpcb after we have finished using the old
  values.
  
  If switching from a thread that used floating-point registers to a thread
  that is still running, but holding the blocked_lock lock we would switch
  the curthread to the new (running) thread, then call critical_enter. This
  will non-atomically increment td_critnest, and later call critical_exit to
  non-atomically decrement this value.
  
  This can happen at the same time as the new thread is still running on the
  old core, also calling these functions. In this case there will be a race
  between these non-atomic operations. This can be an issue as we could loose
  one of these operations leading to the value to not return to zero.
  
  If, later on, we then hit a data abort we check if the td_critnest is zero.
  If this check fails we will panic the kernel.
  
  This has been observed when running pcmstat on a Cavium ThunderX. The pcm
  thread will use the blocked_lock lock and there is a high chance userspace
  will use the floating-point registers. When, later on, pmcstat triggers a
  data abort we will hit this panic.
  
  The fix is to update these values after storing the floating-point state.
  This means we use the correct curthread while storing the state so it will
  not be an issue that the changes to td_critnest are non-atomic.
  
  Sponsored by:	ABT Systems Ltd

Modified:
  head/sys/arm64/arm64/swtch.S

Modified: head/sys/arm64/arm64/swtch.S
==============================================================================
--- head/sys/arm64/arm64/swtch.S	Fri Feb 12 10:58:13 2016	(r295562)
+++ head/sys/arm64/arm64/swtch.S	Fri Feb 12 12:38:04 2016	(r295563)
@@ -129,12 +129,6 @@ END(cpu_throw)
  * x3 to x7, x16 and x17 are caller saved
  */
 ENTRY(cpu_switch)
-	/* Store the new curthread */
-	str	x1, [x18, #PC_CURTHREAD]
-	/* And the new pcb */
-	ldr	x4, [x1, #TD_PCB]
-	str	x4, [x18, #PC_CURPCB]
-
 	/*
 	 * Save the old context.
 	 */
@@ -174,10 +168,14 @@ ENTRY(cpu_switch)
 	mov	x0, x19
 #endif
 
+	/* Store the new curthread */
+	str	x1, [x18, #PC_CURTHREAD]
+
 	/*
-	 * Restore the saved context.
+	 * Restore the saved context and set it as curpcb.
 	 */
 	ldr	x4, [x1, #TD_PCB]
+	str	x4, [x18, #PC_CURPCB]
 
 	/*
 	 * TODO: We may need to flush the cache here if switching