XBMC crash with vaapi vdpau

Mon Jan 10 00:42:31 UTC 2011

Mickaël Maillot <mickael.maillot at gmail.com> writes:

> i recompiled libva & vdpau-video WITH_DEBUG
> but no more usefull information in bt:
> (gdb) bt
> #0  0x0000000812e14349 in __vaDriverInit_0_31 () from
> /usr/local/lib/va/nvidia_drv_video.so
> #1  0x0000000812e11794 in __vaDriverInit_0_31 () from
> /usr/local/lib/va/nvidia_drv_video.so
> #2  0x0000000812e119a7 in __vaDriverInit_0_31 () from
> /usr/local/lib/va/nvidia_drv_video.so
> #3  0x0000000812e11ad2 in __vaDriverInit_0_31 () from
> /usr/local/lib/va/nvidia_drv_video.so
> #4  0x0000000801344b63 in vaCopySurfaceGLX () from /usr/local/lib/libva-glx.so.1
> #5  0x000000080134315f in vaCopySurfaceGLX () from /usr/local/lib/libva-glx.so.1

libva/vdpau-video are still compiled without symbols since you've likely
ignored the note about empty STRIP in my previous message. Try

  $ make install WITH_DEBUG= STRIP= DEBUG_FLAGS='-O0 -ggdb3'

FYI, I have a different crash in vdpau-video driver.

  Core was generated by `xbmc.bin'.
  Program terminated with signal 11, Segmentation fault.
  #0  0x000000080d04d6ce in gl_vdpau_bind_surface (s=0x0) at utils_glx.c:1437
  1437        if (s->is_bound)

  (gdb) i thr
  * 9 Thread 80dc07400 (LWP 100736/initial thread)  0x000000080d04d6ce in gl_vdpau_bind_surface (s=0x0) at utils_glx.c:1437
    8 Thread 80410b800 (LWP 101992/xbmc.bin)  0x000000080aae961c in select () at select.S:3
    7 Thread 80410b000 (LWP 101993/xbmc.bin)  0x000000080aa8acac in poll () at poll.S:3
    6 Thread 80410ac00 (LWP 101994/xbmc.bin)  0x000000080aae961c in select () at select.S:3
    5 Thread 80410a800 (LWP 101995/xbmc.bin)  0x000000080aacef7c in nanosleep () at nanosleep.S:3
    4 Thread 80410e400 (LWP 101996/xbmc.bin)  0x000000080ba924ec in _umtx_op_err () at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
    3 Thread 804108000 (LWP 101997/xbmc.bin)  0x000000080aacef7c in nanosleep () at nanosleep.S:3
    2 Thread 80410a400 (LWP 101998/xbmc.bin)  0x000000080aae961c in select () at select.S:3
    1 Thread 804108c00 (LWP 101386/xbmc.bin)  0x000000080aae961c in select () at select.S:3

  (gdb) bt
  #0  0x000000080d04d6ce in gl_vdpau_bind_surface (s=0x0) at utils_glx.c:1437
  #1  0x000000080d04aca7 in begin_render_glx_surface (driver_data=0x814c15d00, obj_glx_surface=0x814c16c48) at vdpau_video_glx.c:619
  #2  0x000000080d04af4d in copy_glx_surface (driver_data=0x814c15d00, obj_glx_surface=0x814c16c48, obj_surface=0x814c16250, flags=32) at vdpau_video_glx.c:740
  #3  0x000000080d04b086 in vdpau_CopySurfaceGLX (ctx=0x814fe91c0, gl_surface=0x814c16c48, surface=50331649, flags=32) at vdpau_video_glx.c:785
  #4  0x000000080ce335a0 in vaCopySurfaceGLX_impl_driver (ctx=0x814fe91c0, gl_surface=0x814c16c48, surface=50331649, flags=32) at va_glx_impl.c:842
  #5  0x000000080ce31c6b in vaCopySurfaceGLX (dpy=0x814c61080, gl_surface=0x814c16c48, surface=50331649, flags=32) at va_glx.c:163
  #6  0x0000000000ad2e8f in CLinuxRendererGL::UploadVAAPITexture (this=0x814e80600, index=0) at LinuxRendererGL.cpp:2433
  #7  0x0000000000ad24fe in CLinuxRendererGL::Render (this=0x814e80600, flags=<value optimized out>, renderBuffer=0) at LinuxRendererGL.cpp:1283
  #8  0x0000000000acdcfe in CLinuxRendererGL::RenderUpdate (this=0x814e80600, clear=false, flags=64, alpha=<value optimized out>) at LinuxRendererGL.cpp:835
  #9  0x0000000000acc327 in CXBMCRenderManager::RenderUpdate (this=0x1317d08, clear=false, flags=0, alpha=255) at RenderManager.cpp:254
  #10 0x0000000000b55cc7 in CGUIVideoControl::Render (this=0x81346ed80) at GUIVideoControl.cpp:61
  #11 0x0000000000afae27 in CGUIControl::DoRender (this=0x81346ed80, currentTime=<value optimized out>) at GUIControl.cpp:147
  #12 0x0000000000b0d59b in CGUIControlGroup::Render (this=0x813635b00) at GUIControlGroup.cpp:114
  #13 0x0000000000b5775d in CGUIWindow::Render (this=0x813635b00) at GUIWindow.cpp:329
  #14 0x0000000000b62b13 in CGUIWindowManager::Render (this=0x131cd80) at GUIWindowManager.cpp:509
  #15 0x000000000071a13b in CApplication::RenderNoPresent (this=0x1318158) at Application.cpp:1809
  #16 0x000000000070e979 in CApplication::Render (this=0x1318158) at Application.cpp:2046
  #17 0x00000000009a68c1 in CXBApplicationEx::Run (this=0x1318158) at XBApplicationEx.cpp:138
  #18 0x00000000009a706c in main (argc=2, argv=0x7ffffffedf38) at xbmc.cpp:174

However, my ports tree is littered with hacks, some are XBMC related[1]
and some are not. Apps break often. So, it's probably local but

  $ mplayer -vo vaapi:gl -va vaapi

works fine which implies that VA/GLX should, too. And XBMC doesn't
crash with VDPAU_VIDEO_GL_INTEROP=0 or LIBVA_DRIVER_NAME=dummy.

[1] python27 + build hacks for xbmc port - http://pastebin.com/RdgTW8QS

> it crash when i want to read a 1080p mkv file which run fine with VDPAU.
>
> vainfo run fine:
> libva: libva version 0.31.1
> Xlib:  extension "XFree86-DRI" missing on display ":0.0".
> libva: va_getDriverName() returns 0
> libva: Trying to open /usr/local/lib/va/nvidia_drv_video.so
> libva: va_openDriver() returns 0
> vainfo: VA API version: 0.31
> vainfo: Driver version: Splitted-Desktop Systems VDPAU backend for
> VA-API - 0.7.3.pre4
> vainfo: Supported profile and entrypoints
>       VAProfileMPEG2Simple            :	VAEntrypointVLD
>       VAProfileMPEG2Main              :	VAEntrypointVLD
>       VAProfileH264Main               :	VAEntrypointVLD
>       VAProfileH264High               :	VAEntrypointVLD
>       VAProfileVC1Simple              :	VAEntrypointVLD
>       VAProfileVC1Main                :	VAEntrypointVLD
>       VAProfileVC1Advanced            :	VAEntrypointVLD

Just a guess but since vdpau-video may crash on cards that don't support
VDPAU features it may as well crash on "certain" resolutions, too, e.g.

  VDPAU FEATURES NOTE 1 (from nvidia-driver about feature set "B")

  GPUs with this note may not support H.264 streams with the following widths:
  49, 54, 59, 64, 113, 118, 123, 128 macroblocks (769-784, 849-864, 929-944,
  1009-1024, 1793-1808, 1873-1888, 1953-1968, 2033-2048 pixels).

Not that I know what card you have nor the video resolution of the
tested file. So...

- can you play the file with VDPAU but without falling back[1], e.g. to
  software decoder?

- can you play the file with VAAPI in anything else, e.g. mplayer[2], vlc?

- does running with VDPAU_VIDEO_DEBUG=1 produce any debug messages
  besides nvidia-driver version?

[1] XBMC plays video fine with LIBVA_DRIVER_NAME=dummy unlike mplayer and vlc
[2] vaapi hack for mplayer port - http://pastebin.com/A1tcNZM0

> but vatest_10 and 11 failed:
> *** vatest_10: Create and destroy buffers
> libva: libva version 0.31.1
> Xlib:  extension "XFree86-DRI" missing on display ":0.0".
> libva: va_getDriverName() returns 0
> libva: Trying to open /usr/local/lib/va/nvidia_drv_video.so
> libva: va_openDriver() returns 0
> Assertion failed: (VA_STATUS_SUCCESS == va_status), function test,
> file test_10.c, line 112.
> *** vatest_11: Map and unmap buffers
> libva: libva version 0.31.1
> Xlib:  extension "XFree86-DRI" missing on display ":0.0".
> libva: va_getDriverName() returns 0
> libva: Trying to open /usr/local/lib/va/nvidia_drv_video.so
> libva: va_openDriver() returns 0
> --- vaCreateBuffer created buffer 04000000 of type 0
> --- vaCreateBuffer created buffer 04000001 of type 1
> --- vaCreateBuffer created buffer 04000002 of type 2
> Assertion failed: (VA_STATUS_SUCCESS == va_status), function test,
> file test_11.c, line 129.

They do fail on my card (0x0622), too. And asserts are disabled unless
you compile WITH_DEBUG.