kern/81867: ioctl collision between pcvt(4) and vga(4) after recent vesa changes

[Antoine Brodin]

>Number:         81867
>Category:       kern
>Synopsis:       ioctl collision between pcvt(4) and vga(4) after recent vesa changes
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Jun 03 16:00:21 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     Antoine Brodin
>Release:        FreeBSD 6.0-CURRENT i386
>Organization:
none
>Environment:
System: FreeBSD barton.dreadbsd.org 6.0-CURRENT FreeBSD 6.0-CURRENT #1: Sun May 29 17:26:31 CEST 2005 antoine at barton.dreadbsd.org:/usr/obj/usr/src/sys/BARTON i386
>Description:
I've already sent a description to current@ without feedback:
http://docs.freebsd.org/cgi/mid.cgi?20050602171012.61b4568e.antoine.brodin

The recent vesa changes cause a problem on my laptop.
During startup, when ispcvt is called by syscons_precmd
(in /etc/rc.d/syscons), it clears my screen and the screen keeps
cleared.  My kernel doesn't have pcvt compiled in so it's strange.

I investigated a bit, and chmod'ing -x ispcvt solves the problem.

vidcontrol -i mode shows that mode 369 is supported but mode 369 doesn't
work so I think that there is an ioctl collision between:
. this mode: _IO('V', 369 - 256) (from vga(4))
. and VGAPCVTID: _IOWR('V',113, struct pcvtid) (form pcvt(4), used by ispcvt(8))

>How-To-Repeat:
Run ispcvt on a computer where mode 369 is claimed to be supported but doesn't work.
The screen is cleared and it stays cleared.

>Fix:
Collisions must be avoided.

The attached patch works:

--- scvesactl.diff begins here ---
Index: scvesactl.c
===================================================================
RCS file: /home/ncvs/src/sys/dev/syscons/scvesactl.c,v
retrieving revision 1.21
diff -u -p -r1.21 scvesactl.c
--- scvesactl.c	29 May 2005 08:43:43 -0000	1.21
+++ scvesactl.c	2 Jun 2005 14:56:09 -0000
@@ -115,7 +115,9 @@ vesa_ioctl(struct cdev *dev, u_long cmd,
 
 			mode = (cmd & 0xff) + M_VESA_BASE;
 
-			if ((mode > M_VESA_FULL_1280) &&
+			/* Avoid collisions with pcvt. */
+			if (((cmd & IOC_DIRMASK) == IOC_VOID) &&
+			    (mode > M_VESA_FULL_1280) &&
 			    (mode < M_VESA_MODE_MAX))
 				return sc_set_graphics_mode(scp, tp, mode);
 		}
--- scvesactl.diff ends here ---


>Release-Note:
>Audit-Trail:
>Unformatted: