Herostratus’ legacy

words from a lazy coder

GstVA library in GStreamer 1.22 and some new features in 1.24

I know, it’s old news, but still, I was pending to write about the GstVA library to clarify its purpose and scope.I didn’t want to have a library for GstVA, because to maintain a library is a though duty. I learnt it in the bad way with GStreamer-VAAPI. Therefore, I wanted GstVA as simple as possible, direct in its libva usage, and self-contained.

As far as I know, the main usage of GStreamer-VAAPI library was to have access to the VASurface from the GstBuffer, for example, with appsink. Since the beginning of GstVA, a mechanism to access the surface ID was provided, as `GStreamer OpenGL** offers access to the texture ID: through a flag when mapping a GstGL memory backed buffer. You can see how this mechanism is used in the one of the GstVA sample apps.

Nevertheless, later another use case appeared which demanded a library for GstVA: Other GStreamer elements needed to produce or consume VA surface backed buffers. Or to say it more concretely: they needed to use the GstVA allocator and buffer pool, and the mechanism to share the GstVA context along the pipeline too. The plugins with those VA related elements are msdk and qsv, when they operate on Linux. Both elements use Intel OneVPL, so they are basically competitors. The main difference is that the first is maintained by Intel, and has more features, specially for Linux, whilst the former in maintained by Seungha Yang, and it appears to be better tested in Windows.

These are the objects exposed by the GstVA library API:

GstVaDisplay #

GstVaDisplay represents a VADisplay, the interface between the application and the hardware accelerator. This class is abstract, and it’s supposed not to be instantiated. Instantiation has to go through it derived classes, such as

This class is shared among all the elements in the pipeline via GstContext, so all the plugged elements share the same connection the hardware accelerator. Unless the plugged element in the pipeline has a specific name, as in the case of multi GPU systems.

Let’s talk a bit about multi GPU systems. This is the gst-inspect-1.0 output of a system with an Intel and an AMD GPU, both with VA support:

$ gst-inspect-1.0 va
Plugin Details:
  Name                     va
  Description              VA-API codecs plugin
  Filename                 /home/igalia/vjaquez/gstreamer/build/subprojects/gst-plugins-bad/sys/va/libgstva.so
  Version                  1.23.1.1
  License                  LGPL
  Source module            gst-plugins-bad
  Documentation            https://gstreamer.freedesktop.org/documentation/va/
  Binary package           GStreamer Bad Plug-ins git
  Origin URL               Unknown package origin

  vaav1dec: VA-API AV1 Decoder in Intel(R) Gen Graphics
  vacompositor: VA-API Video Compositor in Intel(R) Gen Graphics
  vadeinterlace: VA-API Deinterlacer in Intel(R) Gen Graphics
  vah264dec: VA-API H.264 Decoder in Intel(R) Gen Graphics
  vah264lpenc: VA-API H.264 Low Power Encoder in Intel(R) Gen Graphics
  vah265dec: VA-API H.265 Decoder in Intel(R) Gen Graphics
  vah265lpenc: VA-API H.265 Low Power Encoder in Intel(R) Gen Graphics
  vajpegdec: VA-API JPEG Decoder in Intel(R) Gen Graphics
  vampeg2dec: VA-API Mpeg2 Decoder in Intel(R) Gen Graphics
  vapostproc: VA-API Video Postprocessor in Intel(R) Gen Graphics
  varenderD129av1dec: VA-API AV1 Decoder in AMD Radeon Graphics in renderD129
  varenderD129av1enc: VA-API AV1 Encoder in AMD Radeon Graphics in renderD129
  varenderD129compositor: VA-API Video Compositor in AMD Radeon Graphics in renderD129
  varenderD129deinterlace: VA-API Deinterlacer in AMD Radeon Graphics in renderD129
  varenderD129h264dec: VA-API H.264 Decoder in AMD Radeon Graphics in renderD129
  varenderD129h264enc: VA-API H.264 Encoder in AMD Radeon Graphics in renderD129
  varenderD129h265dec: VA-API H.265 Decoder in AMD Radeon Graphics in renderD129
  varenderD129h265enc: VA-API H.265 Encoder in AMD Radeon Graphics in renderD129
  varenderD129jpegdec: VA-API JPEG Decoder in AMD Radeon Graphics in renderD129
  varenderD129postproc: VA-API Video Postprocessor in AMD Radeon Graphics in renderD129
  varenderD129vp9dec: VA-API VP9 Decoder in AMD Radeon Graphics in renderD129
  vavp9dec: VA-API VP9 Decoder in Intel(R) Gen Graphics

  22 features:
  +-- 22 elements

As you can observe, each card registers the supported element. The first GPU, the Intel one, doesn’t insert renderD129 after the va prefix, while the AMD Radeon, does. As you could imagine, renderD129 is the device name in /dev/dri:

$ ls /dev/dri
by-path  card0  card1  renderD128  renderD129

The appended device name expresses the DRM device the elements use. And only after the second GPU the device name is appended. This allows to use the untagged elements either for the first GPU or to allow the usage of a wrapped display injected by the user application, such as VA X11 or Wayland connections.

Notice that sharing DMABuf-based buffers between different GPUs is theoretically possible, but not assured.

Keep in mind that, currently, nvidia-vaapi-driver is not supported by GstVA, and the driver will be ignored by the plugin register since 1.24.

VA allocators #

There are two types of memory allocators in GstVA:

Also, VA allocators offer a couple methods that can be useful for applications that use appsink, for example gst_va_buffer_get_surface and gst_va_buffer_peek_display.

One particularity of both allocators is that they keep an internal pool of the allocated VA surfaces, since the mapping of buffers/memories is not always 1:1, as in the case of DMABuf; to reuse surfaces even if the buffer pool ditches them, and to avoid surface leaks, keeping track of them all the time.

GstVaPool #

This class is only useful for other GStreamer elements capable of consume VA surfaces, so they could instantiate, configure and propose a VA buffer pool, but not for applications.

And that’s all for now. Thank you for bear with me up to here. But remember, the API of the library is unstable, and it can change any time. So, no promises are made :)