[PATCH:] psm(4) IntelliMouse Explorer KVM hack breaks my mouse

[Juergen Lock]

(..and older vbox versions.)

Hi!

 I just saw this vbox ticket:

	http://www.virtualbox.org/ticket/6488

(`Mouse wheel scrolling interpredted as click events in guest -> fixed
  after the 3.1.6 release')

 ..which sounded just like what a physical mouse I have (MS `IntelliMouse
Optical 1.1A' according to whats printed on the bottom) did outside of
a VM too, so I got curious and patched my psm driver to disable the KVM
hack mentioned in that ticket which was introduced back in Apr 2000 by
this commit:

	http://svn.freebsd.org/viewvc/base?view=revision&revision=58923

(`Add temporary workaround to fool some "clever" KVM switch products
  which think they know the IntelliMouse 4-byte packet and believe,
  wrongly, that any other protocols use 3-byte packets.')

 ..and indeed, now the stray click events are gone for me too! :)

 So now I made a patch that allows disabling that KVM hack via device
hints, appended below.  (hint.psm.0.flags="0x10000" - or do you guys
think the hack should be disabled by default instead?)

 Cheers,
	Juergen

Index: src/sys/dev/atkbdc/psm.c
===================================================================
RCS file: /home/scvs/src/sys/dev/atkbdc/psm.c,v
retrieving revision 1.104.2.2
diff -u -p -r1.104.2.2 psm.c
--- src/sys/dev/atkbdc/psm.c	20 Aug 2009 20:23:28 -0000	1.104.2.2
+++ src/sys/dev/atkbdc/psm.c	11 May 2010 18:06:01 -0000
@@ -326,6 +326,7 @@ static devclass_t psm_devclass;
 #define	PSM_CONFIG_HOOKRESUME	0x2000	/* hook the system resume event */
 #define	PSM_CONFIG_INITAFTERSUSPEND 0x4000 /* init the device at the resume event */
 #define	PSM_CONFIG_SYNCHACK	0x8000	/* enable `out-of-sync' hack */
+#define	PSM_CONFIG_NOKVMHACK	0x10000	/* disable IntelliMouse Explorer KVM hack */
 
 #define	PSM_CONFIG_FLAGS	\
     (PSM_CONFIG_RESOLUTION |	\
@@ -337,7 +338,8 @@ static devclass_t psm_devclass;
     PSM_CONFIG_FORCETAP |	\
     PSM_CONFIG_IGNPORTERROR |	\
     PSM_CONFIG_HOOKRESUME |	\
-    PSM_CONFIG_INITAFTERSUSPEND)
+    PSM_CONFIG_INITAFTERSUSPEND | \
+    PSM_CONFIG_NOKVMHACK)
 
 /* other flags (flags) */
 #define	PSM_FLAGS_FINGERDOWN	0x0001	/* VersaPad finger down */
@@ -3779,20 +3781,23 @@ enable_msexplorer(struct psm_softc *sc)
 	sc->hw.hwid = id;
 	sc->hw.buttons = 5;		/* IntelliMouse Explorer XXX */
 
-	/*
-	 * XXX: this is a kludge to fool some KVM switch products
-	 * which think they are clever enough to know the 4-byte IntelliMouse
-	 * protocol, and assume any other protocols use 3-byte packets.
-	 * They don't convey 4-byte data packets from the IntelliMouse Explorer
-	 * correctly to the host computer because of this!
-	 * The following sequence is actually IntelliMouse's "wake up"
-	 * sequence; it will make the KVM think the mouse is IntelliMouse
-	 * when it is in fact IntelliMouse Explorer.
-	 */
-	for (i = 0; i < sizeof(rate0)/sizeof(rate0[0]); ++i)
-		if (set_mouse_sampling_rate(kbdc, rate0[i]) != rate0[i])
-			break;
-	id = get_aux_id(kbdc);
+	if (!(sc->config & PSM_CONFIG_NOKVMHACK)) {
+		/*
+		 * XXX: this is a kludge to fool some KVM switch products
+		 * which think they are clever enough to know the 4-byte
+		 * IntelliMouse protocol, and assume any other protocols
+		 * use 3-byte packets.
+		 * They don't convey 4-byte data packets from the IntelliMouse
+		 * Explorer correctly to the host computer because of this!
+		 * The following sequence is actually IntelliMouse's "wake up"
+		 * sequence; it will make the KVM think the mouse is
+		 * IntelliMouse when it is in fact IntelliMouse Explorer.
+		 */
+		for (i = 0; i < sizeof(rate0)/sizeof(rate0[0]); ++i)
+			if (set_mouse_sampling_rate(kbdc, rate0[i]) != rate0[i])
+				break;
+		id = get_aux_id(kbdc);
+	}
 
 	return (TRUE);
 }
Index: src/share/man/man4/psm.4
===================================================================
RCS file: /home/scvs/src/share/man/man4/psm.4,v
retrieving revision 1.49.2.1
diff -u -p -r1.49.2.1 psm.4
--- src/share/man/man4/psm.4	3 Aug 2009 08:13:06 -0000	1.49.2.1
+++ src/share/man/man4/psm.4	11 May 2010 18:04:16 -0000
@@ -349,6 +349,11 @@ after the `resume' event.
 It has no effect unless the
 .Em HOOKRESUME
 flag is set as well.
+.It bit 16 NOKVMHACK
+This flag disables the IntelliMouse Explorer protocol KVM switch
+workaround that makes some virtual machine's mouse emulations as well
+as at least one physical IntelliMouse Optical model misbehave
+(causing the scroll wheel to produce stray click events.)
 .El
 .Sh LOADER TUNABLES
 Extended support for Synaptics touchpads can be enabled by setting