svn commit: r352505 - head/sys/dev/vt

[Andriy Gapon]

Author: avg
Date: Thu Sep 19 09:22:45 2019
New Revision: 352505
URL: https://svnweb.freebsd.org/changeset/base/352505

Log:
  vt: fix problems with trying to switch to a closed VT
  
  If there is an attempt to switch from a process-owned VT to a closed VT,
  then vt(4) first requests the process to release its VT and only then
  realizes that the target VT is closed and, so, the switch is not
  possible.  So, the driver does not actually do any switch, but at the
  same time the owning process is not notified about that and it does not
  re-acquire the VT.
  
  This change adds an early check for the target VT state, so that the
  switch can be refused before the process coordination dance.
  On top of that, the code now checks for a failure of vt_window_switch()
  and calls vt_window_postswitch() for the current VT if it is in the
  process mode.
  
  Test Plan:
  - configure VT1 - VT8 (ttyv0 - ttyv7) to be text consoles (run getty)
  - configure VT9 (ttyv8) to rn X server
  - make sure that the X server configuration allows VT switching
  - leave VT10 - VT12 unconfigured
  - while in the X server press Ctrl+Alt+F10
  - without the patch, observe strange screen content and problems with
    keyboard input
  - with the patch, observe that nothing happens
  
  The problem has been observed and the fix has been tested with an nVidia
  graphics card and the proprietary nvidia driver.
  Not sure if that matters.
  
  Reviewed by:	ray
  MFC after:	2 weeks
  Differential Revision: https://reviews.freebsd.org/D21704

Modified:
  head/sys/dev/vt/vt_core.c

Modified: head/sys/dev/vt/vt_core.c
==============================================================================
--- head/sys/dev/vt/vt_core.c	Thu Sep 19 07:28:24 2019	(r352504)
+++ head/sys/dev/vt/vt_core.c	Thu Sep 19 09:22:45 2019	(r352505)
@@ -335,7 +335,7 @@ static void
 vt_switch_timer(void *arg)
 {
 
-	vt_late_window_switch((struct vt_window *)arg);
+	(void)vt_late_window_switch((struct vt_window *)arg);
 }
 
 static int
@@ -457,13 +457,22 @@ vt_window_postswitch(struct vt_window *vw)
 static int
 vt_late_window_switch(struct vt_window *vw)
 {
+	struct vt_window *curvw;
 	int ret;
 
 	callout_stop(&vw->vw_proc_dead_timer);
 
 	ret = vt_window_switch(vw);
-	if (ret)
+	if (ret != 0) {
+		/*
+		 * If the switch hasn't happened, then return the VT
+		 * to the current owner, if any.
+		 */
+		curvw = vw->vw_device->vd_curwindow;
+		if (curvw->vw_smode.mode == VT_PROCESS)
+			(void)vt_window_postswitch(curvw);
 		return (ret);
+	}
 
 	/* Notify owner process about terminal availability. */
 	if (vw->vw_smode.mode == VT_PROCESS) {
@@ -509,6 +518,19 @@ vt_proc_window_switch(struct vt_window *vw)
 		return (0);	/* success */
 	}
 
+	/*
+	 * Early check for an attempt to switch to a non-functional VT.
+	 * The same check is done in vt_window_switch(), but it's better
+	 * to fail as early as possible to avoid needless pre-switch
+	 * actions.
+	 */
+	VT_LOCK(vd);
+	if ((vw->vw_flags & (VWF_OPENED|VWF_CONSOLE)) == 0) {
+		VT_UNLOCK(vd);
+		return (EINVAL);
+	}
+	VT_UNLOCK(vd);
+
 	/* Ask current process permission to switch away. */
 	if (curvw->vw_smode.mode == VT_PROCESS) {
 		DPRINTF(30, "%s: VT_PROCESS ", __func__);
@@ -1792,7 +1814,7 @@ finish_vt_rel(struct vt_window *vw, int release, int *
 		vw->vw_flags &= ~VWF_SWWAIT_REL;
 		if (release) {
 			callout_drain(&vw->vw_proc_dead_timer);
-			vt_late_window_switch(vw->vw_switch_to);
+			(void)vt_late_window_switch(vw->vw_switch_to);
 		}
 		return (0);
 	}