GStreamer VA-API Troubleshooting

GStreamer VA-API is not a trivial piece of software. Even though, in my opinion it is a bit over-engineered, the complexity relies on its layered architecture: the user must troubleshoot in which layer is the failure.

So, bear in mind this architecture:

GStreamer VA-API is not a trivial piece of software. Even though, in my opinion it is a bit over-engineered, the complexity relies on its layered architecture: the user must troubleshoot in which layer is the failure.

So, bear in mind this architecture:

libva architecture
libva architecture

And the point of failure could be anywhere.

Drivers

libva is a library designed to load another library called driver or back-end. This driver is responsible to talk with the kernel, windowing platform, memory handling library, or any other piece of software or hardware that actually will do the video processing.

There are many drivers in the wild. As it is an API aiming to stateless video processing, and the industry is moving towards that way to process video, it is expected more drivers would appear in the future.

Nonetheless, not all the drivers have the same level of maturity, and some of them are abandon-ware. For this reason we decided in GStreamer VA-API, some time ago, to add a white list of functional drivers, basically, those developed by Mesa3D and this one from Intel™. If you wish to disable that white-list, you can do it by setting an environment variable:

$ export GST_VAAPI_ALL_DRIVERS=1

Remember, if you set it, you are on your own, since we do not trust on the maturity of that driver yet.

Internal libva↔driver version

Thus, there is an internal API between libva and the driver and it is versioned, meaning that the internal API version of the installed libva library must match with the internal API exposed by the driver. One of the causes that libva could not initialize a driver could be because the internal API version does not match.

Drivers path and driver name

By default there is a path where libva looks for drivers to load. That path is defined at compilation time. Following Debian’s file-system hierarchy standard (FHS) it should be set by distributions in /usr/lib/x86_64-linux-gnu/dri/. But the user can control this path with an environment variable:

$ export LIBVA_DRIVERS_PATH=${HOME}/src/intel-vaapi-driver/src/.libs

The driver path, as a directory, might contain several drivers. libva will try to guess the correct one by querying the instantiated VA display (which could be either KMS/DRM, Wayland, Android or X11). If the user instantiates a VA display different of his running environment, the guess will be erroneous, the library loading will fail.

Although, there is a way for the user to set the driver’s name too. Again, by setting an environment variable:

$ export LIBVA_DRIVER_NAME=iHD

With this setting, libva will try to load iHD_drv_video.so (a new and experimental open source driver from Intel™, targeted for MediaSDK —do not use it yet with GStreamer VAAPI—).

vainfo

vainfo is the diagnostic tool for VA-API. In a couple words, it will iterate on a list of VA displays, in try-and-error strategy, and try to initialize VA. In case of success, vainfo will report the driver signature, and it will query the driver for the available profiles and entry-points.

For example, my skylake board for development will report

$ vainfo
error: can't connect to X server!
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /home/vjaquez/gst/master/intel-vaapi-driver/src/.libs/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.1.pre1)
vainfo: Driver version: Intel i965 driver for Intel(R) Skylake - 2.1.1.pre1 (2.1.0-41-g99c3748)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileH264ConstrainedBaseline: VAEntrypointFEI
      VAProfileH264ConstrainedBaseline: VAEntrypointStats
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264Main               : VAEntrypointFEI
      VAProfileH264Main               : VAEntrypointStats
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointFEI
      VAProfileH264High               : VAEntrypointStats
      VAProfileH264MultiviewHigh      : VAEntrypointVLD
      VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
      VAProfileH264StereoHigh         : VAEntrypointVLD
      VAProfileH264StereoHigh         : VAEntrypointEncSlice
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileVP8Version0_3          : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice

And my AMD board with stable packages replies:

$ vainfo
libva info: VA-API version 0.40.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_0_40
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.40 (libva )
vainfo: Driver version: mesa gallium vaapi
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileNone                   : VAEntrypointVideoProc

Does this mean that VA-API processes video? No. It means that there is an usable VA display which could open a driver correctly and libva can extract symbols from it.

I would like to mention another tool, not official, but I like it a lot, since it extracts almost of the VA information available in the driver: vadumpcaps.c, written by Mark Thompson.

GStreamer VA-API registration

When GStreamer is launched, normally it will register all the available plugins and plugin features (elements, device providers, etc.). All that data is cache and keep until the cache file is deleted or the cache invalidated by some event.

At registration time, GStreamer VA-API will instantiate a DRM-based VA display, which works with no need of a real display (in other words, headless), and will query the driver for the profiles and entry-points tuples, in order to register only the available elements (encoders, decoders. sink, post-processor). If the DRM VA display fails, a list of VA displays will be tried.

In the case that libva could not load any driver, or the driver is not in the white-list, GStreamer VA-API will not register any element. Otherwise gst-inspect-1.0 will show the registered elements:

$ gst-inspect-1.0 vaapi
Plugin Details:
  Name                     vaapi
  Description              VA-API based elements
  Filename                 /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so
  Version                  1.12.4
  License                  LGPL
  Source module            gstreamer-vaapi
  Source release date      2017-12-07
  Binary package           gstreamer-vaapi
  Origin URL               http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

  vaapijpegdec: VA-API JPEG decoder
  vaapimpeg2dec: VA-API MPEG2 decoder
  vaapih264dec: VA-API H264 decoder
  vaapivc1dec: VA-API VC1 decoder
  vaapivp8dec: VA-API VP8 decoder
  vaapih265dec: VA-API H265 decoder
  vaapipostproc: VA-API video postprocessing
  vaapidecodebin: VA-API Decode Bin
  vaapisink: VA-API sink
  vaapimpeg2enc: VA-API MPEG-2 encoder
  vaapih265enc: VA-API H265 encoder
  vaapijpegenc: VA-API JPEG encoder
  vaapih264enc: VA-API H264 encoder

  13 features:
  +-- 13 elements

Beside the normal behavior, GStreamer VA-API will also invalidate GStreamer’s cache at every boot, or when any of the mentioned environment variables change.

Conclusion

A simple task list to review when GStreamer VA-API is not working at all is this:

#. Check your LIBVA_* environment variables
#. Verify that vainfo returns sensible information
#. Invalidate GStreamer’s cache (or just delete the file)
#. Check the output of gst-inspect-1.0 vaapi

And, if you decide to file a bug in bugzilla, please do not forget to attach the output of vainfo and the logs if the developer asks for them.

7 thoughts on “GStreamer VA-API Troubleshooting”

  1. Hello I am developing on Ubuntu 18.04 and try to make the GStreamer VA-API working on my integrated graphic card UHD630.
    So far I have compiled anythings necessary.
    vainfo can give valid output.

    libva info: VA-API version 1.4.1
    libva info: va_getDriverName() returns 0
    libva info: User requested driver ‘iHD’
    libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
    libva info: Found init function __vaDriverInit_1_4
    libva info: va_openDriver() returns 0
    vainfo: VA-API version: 1.4 (libva 2.1.0)
    vainfo: Driver version: Intel iHD driver – 19.1.0.git_42f6f23_2019-04-23
    vainfo: Supported profile and entrypoints
    VAProfileNone : VAEntrypointVideoProc
    VAProfileNone : VAEntrypointStats
    VAProfileMPEG2Simple : VAEntrypointVLD
    VAProfileMPEG2Simple : VAEntrypointEncSlice
    VAProfileMPEG2Main : VAEntrypointVLD
    VAProfileMPEG2Main : VAEntrypointEncSlice
    VAProfileH264Main : VAEntrypointVLD
    VAProfileH264Main : VAEntrypointEncSlice
    VAProfileH264Main : VAEntrypointFEI
    VAProfileH264Main : VAEntrypointEncSliceLP
    VAProfileH264High : VAEntrypointVLD
    VAProfileH264High : VAEntrypointEncSlice
    VAProfileH264High : VAEntrypointFEI
    VAProfileH264High : VAEntrypointEncSliceLP
    VAProfileVC1Simple : VAEntrypointVLD
    VAProfileVC1Main : VAEntrypointVLD
    VAProfileVC1Advanced : VAEntrypointVLD
    VAProfileJPEGBaseline : VAEntrypointVLD
    VAProfileJPEGBaseline : VAEntrypointEncPicture
    VAProfileH264ConstrainedBaseline: VAEntrypointVLD
    VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
    VAProfileH264ConstrainedBaseline: VAEntrypointFEI
    VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
    VAProfileVP8Version0_3 : VAEntrypointVLD
    VAProfileVP8Version0_3 : VAEntrypointEncSlice
    VAProfileHEVCMain : VAEntrypointVLD
    VAProfileHEVCMain : VAEntrypointEncSlice
    VAProfileHEVCMain : VAEntrypointFEI
    VAProfileHEVCMain10 : VAEntrypointVLD
    VAProfileHEVCMain10 : VAEntrypointEncSlice
    VAProfileVP9Profile0 : VAEntrypointVLD
    VAProfileVP9Profile2 : VAEntrypointVLD

    But when I go back to gst, there is no features available.

    gst-inspect-1.0 vaapi
    Plugin Details:
    Name vaapi
    Description VA-API based elements
    Filename /usr/local/lib/gstreamer-1.0/libgstvaapi.so
    Version 1.16.2
    License LGPL
    Source module gstreamer-vaapi
    Source release date 2019-12-03
    Binary package gstreamer-vaapi
    Origin URL http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

    0 features:

    I have no ideas what else should I do.
    Can you give me some suggestions?

  2. I’m testing the newest xubuntu 20.04 and have the same problem. I guess there’s mismatch between driver API version and gstreamer VAAPI expected version:
    vainfo
    libva info: VA-API version 1.7.0
    libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
    libva info: Found init function __vaDriverInit_1_7
    libva error: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
    libva info: va_openDriver() returns 1
    libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
    libva info: Found init function __vaDriverInit_1_6
    libva info: va_openDriver() returns 0
    vainfo: VA-API version: 1.7 (libva 2.6.0)
    vainfo: Driver version: Intel i965 driver for Intel(R) Sandybridge Mobile – 2.4.0
    vainfo: Supported profile and entrypoints
    VAProfileMPEG2Simple : VAEntrypointVLD
    VAProfileMPEG2Main : VAEntrypointVLD
    VAProfileH264ConstrainedBaseline: VAEntrypointVLD
    VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
    VAProfileH264Main : VAEntrypointVLD
    VAProfileH264Main : VAEntrypointEncSlice
    VAProfileH264High : VAEntrypointVLD
    VAProfileH264High : VAEntrypointEncSlice
    VAProfileH264StereoHigh : VAEntrypointVLD
    VAProfileVC1Simple : VAEntrypointVLD
    VAProfileVC1Main : VAEntrypointVLD
    VAProfileVC1Advanced : VAEntrypointVLD
    VAProfileNone : VAEntrypointVideoProc
    gst-inspect-1.0 vaapi
    Plugin Details:
    Name vaapi
    Description VA-API based elements
    Filename /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstvaapi.so
    Version 1.16.2
    License LGPL
    Source module gstreamer-vaapi
    Source release date 2019-12-03
    Binary package gstreamer-vaapi
    Origin URL http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer
    0 features:

  3. I did much more research and found this: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/issues/194
    So I deleted the cached registry:
    rm ~/.cache/gstreamer-1.0 -rf
    then ran (debug level 6): GST_DEBUG=*vaapi*:6 gst-inspect-1.0 vaapi
    the result was crazy, it opened successfully /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so
    and didn’t try further.
    I wiped the cache again and executed:
    LIBVA_DRIVER_NAME=i965 GST_DEBUG=*vaapi*:6 gst-inspect-1.0 vaapi
    then it rambled: DRM_IOCTL_I915_GEM_APERTURE failed: No such file or directory
    Assuming 131072kB available aperture size.
    May lead to reduced performance or incorrect rendering.
    (i skip some of the messages) and finally I got:
    vaapih264enc: VA-API H264 encoder
    vaapisink: VA-API sink
    vaapidecodebin: VA-API Decode Bin
    vaapipostproc: VA-API video postprocessing
    vaapivc1dec: VA-API VC1 decoder
    vaapih264dec: VA-API H264 decoder
    vaapimpeg2dec: VA-API MPEG2 decoder
    7 features

  4. Hi, thanks for this great article! I tried gstreamer-1.18 on Ubuntu 20.10, and my test pipeline using vaapijpegdec is 6 times faster with the i965 driver than with the iHD driver. Is it expected that iHD has a worse performance? The hardware is an Atom E3940 with a HD Graphics 500.
    My test-pipeline:
    gst-launch-1.0 -v multifilesrc location=”%03.jpg” index=0 ! jpegparse ! vaapijpegdec ! filesink location=out
    Performance with export LIBVA_DRIVER_NAME=i965: “Execution ended after ~2.9 seconds”
    Performance with export LIBVA_DRIVER_NAME=iHD: “Execution ended after ~19.6 seconds”

Leave a Reply

Your email address will not be published. Required fields are marked *