ports/156386: [patch] x11/nvidia-driver: EnableMSI doesn't work

[Pan Tsu]

>Number:         156386
>Category:       ports
>Synopsis:       [patch] x11/nvidia-driver: EnableMSI doesn't work
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-ports-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu Apr 14 02:10:07 UTC 2011
>Closed-Date:
>Last-Modified:
>Originator:     Pan Tsu
>Release:        FreeBSD 9.0-CURRENT amd64
>Organization:
>Environment:
$ sysctl -n hw.nvidia.registry.EnableMSI
1

$ sudo pciconf -lvcb | sed -n '/vgapci/,/@pci/p' | sed '$d'
vgapci0 at pci0:1:0:0:     class=0x030000 card=0x34811458 chip=0x062210de rev=0xa1 hdr=0x00
    vendor     = 'NVIDIA Corporation'
    device     = 'NVIDIA GeForce 9600 GT (G94)'
    class      = display
    subclass   = VGA
    bar   [10] = type Memory, range 32, base 0xf6000000, size 16777216, enabled
    bar   [14] = type Prefetchable Memory, range 64, base 0xe0000000, size 268435456, enabled
    bar   [1c] = type Memory, range 64, base 0xf4000000, size 33554432, enabled
    bar   [24] = type I/O Port, range 32, base 0xb000, size 128, enabled
    cap 01[60] = powerspec 3  supports D0 D3  current D0
    cap 05[68] = MSI supports 1 message, 64 bit
    cap 10[78] = PCI-Express 2 endpoint max data 128(128) link x16(x16)
    cap 09[b4] = vendor (length 20)
ecap 0002[100] = VC 1 max VC0
ecap 0004[128] = unknown 1
ecap 000b[600] = unknown 1
>Description:
Since 256.29 nvidia driver allows setting EnableMSI via sysctl which was
long available on linux before. However, sysctls are initialized at the
last stage of module loading when the device is already attached.

Linux uses NVreg_EnableMSI as a module parameter via modprobe(8) but
it doesn't seem to work on FreeBSD when set via kenv(1) or loader(8).
>How-To-Repeat:
1. load nvidia.ko
2. set hw.nvidia.registry.EnableMSI via sysctl
3. start X11 # ???
4. check pciconf -lc and/or vmstat -i

1. apply the patch
2. kenv hint.nvidia.0.msi=1
3. kldload nvidia
4. check pciconf -lc and/or vmstat -i

`pciconf -lc' diff
  -    cap 05[68] = MSI supports 1 message, 64 bit 
  +    cap 05[68] = MSI supports 1 message, 64 bit enabled with 1 message

`vmstat -i' diff
   interrupt                          total       rate
  -irq16: vgapci0++                    3639         13
   irq18: uhci2 ehci0*                 XXXX          X
   irq20: fxp0                         XXXX          X
   irq21: uhci1                        XXXX          X
   irq22: hpet0                        XXXX          X
   irq23: uhci3 ehci1                  XXXX          X
  +irq256: vgapci0                     5167         20
   irq257: hdac0                       XXXX          X
   irq258: ahci1:ch0                   XXXX          X
   irq259: ahci1:ch1                   XXXX          X
>Fix:
Below workaround allows one to enable MSI via device.hints(5)

  hint.nvidia.%d.msi=1

not sure if it'll work with different settings on each card with
a multicard setup.

--- a.diff begins here ---
Index: x11/nvidia-driver/files/patch-src-nvidia_subr.c
===================================================================
RCS file: x11/nvidia-driver/files/patch-src-nvidia_subr.c
diff -N x11/nvidia-driver/files/patch-src-nvidia_subr.c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ x11/nvidia-driver/files/patch-src-nvidia_subr.c	14 Apr 2011 00:47:22 -0000
@@ -0,0 +1,13 @@
+--- src/nvidia_subr.c~
++++ src/nvidia_subr.c
+@@ -252,8 +252,8 @@ int nvidia_alloc_hardware(device_t dev)
+         sc->BAR_recs[i] = res;
+     }
+ 
+-    if ((rm_read_registry_dword(sp, NULL, "NVreg",
+-            "EnableMSI", &enable_msi) == RM_OK) && (enable_msi != 0)) {
++    if ((resource_int_value(device_get_name(dev), device_get_unit(dev),
++            "msi", &enable_msi) == 0) && (enable_msi != 0)) {
+         count = pci_msi_count(dev);
+         if ((count == 1) && (pci_alloc_msi(dev, &count) == 0))
+             sc->irq_rid = 1;
--- a.diff ends here ---
>Release-Note:
>Audit-Trail:
>Unformatted: