NVidia + run X apps from jail

Wed Oct 5 12:31:08 UTC 2016

Petr Fischer kirjoitti 05.10.2016 06:45:
>> Petr Fischer kirjoitti 03.10.2016 05:15:
>> > Hello, I have kodi installed in a jail and running it remotely to the
>> > main host X server with DISPLAY=server:0 kodi...
>> >
>> > With vesa driver, it's slow but everything works from jail.
>> >
>> > Just bought new NVidia 210 card, installed "nvidia-driver" package,
>> > but now, every OpenGL application cant't start (when started inside
>> > jail):
>> >
>> > 1) DISPLAY=server:0 xclock (from jail)
>> > OK
>> >
>> > 2) DISPLAY=server:0 glxgears (from jail)
>> > libGL error: No matching fbConfigs or visuals found
>> > libGL error: failed to load driver: swrast
>> > X Error of failed request:  BadValue (integer parameter out of range
>> > for operation)
>> >   Major opcode of failed request:  153 (GLX)
>> >   Minor opcode of failed request:  3 (X_GLXCreateContext)
>> >   Value in failed request:  0x0
>> >   Serial number of failed request:  33
>> >   Current serial number in output stream:  35
>> >
>> > What's the problem? Thanks! pf
>> > _______________________________________________
>> > freebsd-x11 at freebsd.org mailing list
>> > https://lists.freebsd.org/mailman/listinfo/freebsd-x11
>> > To unsubscribe, send any mail to "freebsd-x11-unsubscribe at freebsd.org"
>> 
>> Yes, first verify that you can get accelerated GPU support working on 
>> the
>> actual system where you run the X-server. If the nvidia-driver 
>> -package does
>> not work, install one of the other nvidia-driver-* package and try 
>> again.
>> Test the graphics with OpenGL applications.
> 
> Yes, I installed correct nvidia-driver-340 (old graphics card, older 
> driver).
> That is OK. Works fine (media acceleration + opengl).
> 
>> After you can get OpenGL working locally on the system where X11 
>> server is
>> running in, next you need to try if you can get AIGLX working. To do 
>> this I
>> would suggest installing the exact nvidia-driver or nvidia-driver-* 
>> package
>> (which you installed in system with X11 server) inside the jail from 
>> where
>> you want to run X11 apps. Aftern this you should try starting some 
>> OpenGL
>> using app from inside the jail with DISPLAY=server:0 and see what 
>> happens.
> 
> Installed exactly this older nvidia driver also inside jail - but no
> luck. No working OpenGL (AIGLX) to server:0.
> Still this errors:
> libGL error: No matching fbConfigs or visuals found
> libGL error: failed to load driver: swrast
> ...

The error messages imply that the nVidia specific libGL failed to 
initialize AIGLX on the server end. BUT the message also clearly states 
that libGL at least tried AIGLX, and this is good!

I would try to figure out first if the X.org server enabled AIGLX 
support or not. This can be figured out by looking the 
/var/log/Xorg.0.log -logfile

If the server enabled AIGLX _with_ the correct nVidia driver installed, 
there should be a line such as:
[    74.014] (II) LoadModule: "glx"
[    74.015] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[    74.017] (II) Module glx: vendor="X.Org Foundation"
[    74.017]    compiled for 1.16.4, module version = 1.0.0
[    74.017]    ABI class: X.Org Server Extension, version 8.0
[    74.017] (==) AIGLX enabled

Please note that the above example is obviously different in your case, 
but there should be "AIGLX enabled" somewhere if AIGLX was enabled.

If you do not find any such message, then the next step would be to 
figure out how to force loading of AIGLX on the device. This certainly 
means you will have to write somekind of X.org configuration file to 
/usr/local/etc/X11/xorg.conf.d or similar location.

I've been googling for some ideas on this but nothing come up thus far. 
It is assumed that if the nVidia driver is loaded by the X.org server, 
then it should also enable AIGLX.

But there is still the VirtualGL option if this does not work.

>> If the AIGLX option did not work, you have another solution that 
>> requires
>> more work but is way more optimal: VirtualGL. I actually have used 
>> this
>> succesfully on several occassions. To get it working you just need to 
>> read
>> the documentation at 
>> http://www.virtualgl.org/Documentation/Documentation
>> and improvise. Requires shell scripting, but very doable. You can 
>> install
>> VirtualGL via packages or ports.
> 
> I tried hard this virtualgl thing, started vglclient, used vglrun on
> other side, and ended with this error message:
> 
> $ DISPLAY=server:0 vglrun glxgears
> [VGL] ERROR: in fconfig_instance--
> [VGL]    97: Function not implemented
> 
> It's too much for me, absolutely no info on google about this error.
> To much layers for me (and maybe some of the layer not implemented
> some function, or this is just bogus error).

Don't give up yet. I had no idea VirtualGL required SYSV shared memory, 
because I had no idea why would it even need that API ...

https://sourceforge.net/p/virtualgl/mailman/message/35007472/
http://forum.directadmin.com/showthread.php?t=30537

In short: SHM aka shared memory is not enabled by default in jails for 
security reasons.

You must enable shared memory in jails either system wide via sysctl or 
jail specific via jail configuration.

To enable per-jail, put "allow.sysvipc = true" in jail configuration:
jail {
...
allow.sysvipc = true;
...
}

To enable systemwide for all jails, do as root:
sysctl security.jail.sysvipc_allowed=1

Try enabling per jail first, if that don't help then set also the system 
wide sysctl. This must be tried in the order I described, because the 
documentation is vague on whether the sysctl is required even if 
per-jail shm is actually wanted.

ALSO on older FreeBSD you might need to do as root: kldload shm

> So I gave up OpenGL in a jail and installed everything (whole desktops
> + kodi + all deps) on the base host (server:0).
> 
> A also tried to run whole Xorg in a jail, configured many devfs rules
> (nvidia devs, mixer devs etc etc), but also no luck - there was some
> error like: "Can't get extended write mode on /dev/io" (or something
> like this).

This is by design. You see, X.org requires direct access to many 
insecure device facilities, which cannot be allowed in a secure jail.

Without modifying FreeBSD kernel sources and recompiling the kernel you 
cannot run X-server inside the jail. In my opinion a jail with access to 
insecure device nodes is pointless.

-- 
Arto Pekkanen