Vulkan video with NVK driver
This blog post might interest you if you want to try the bleeding edge NVK driver which allows to decode H264/5 video with the power of the Vulkan extensions VK_KHR_video_decode_h26x
.
This is a summary of the instructions provided in the MR. This work needs a recent kernel with new features, so it will describe the steps to add this feature and build this new kernel on an Ubuntu based system.
To accomplish that what you would need, is a recent NVIDIA hardware such as GeForce RTX 4060, a Linux kernel greater than 6.12.8 and the mesa branch supporting Vulkan Video using NVK: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31867
Lets start by the root, the kernel.
Kernel build #
To run the NVK driver, you need a custom patch to be applied on top of the Nouveau
driver. This patch applies on minimum 6.12 kernel so you need to build a new kernel except if your distribution is running bleeding edge kernel which i doubt so here is my method I used to build this kernel.
I mainly got my inspiration from https://askubuntu.com/questions/718381/how-to-compile-and-install-custom-mainline-kernel but I had to adapt it a bit to make it work.
All of this blog post has been performed on a regular ubuntu 24.04.
First you need to install the following ubuntu package:
$ sudo apt install -y git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache
You’ll have to download a recent kernel. I ended up using a git clone as it was easier than a tarball to build all necessary bits.
$ git clone --branch v6.12.8 --depth 1 git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-stable
$ cd linux-stable
Then you’ll have to apply the given patch to your work tree:
diff --git a/drivers/gpu/drm/nouveau/nouveau_abi16.c b/drivers/gpu/drm/nouveau/nouveau_abi16.c
index 2a0617e5fe2a..eaef5effbb3d 100644
--- a/drivers/gpu/drm/nouveau/nouveau_abi16.c
+++ b/drivers/gpu/drm/nouveau/nouveau_abi16.c
@@ -379,6 +379,9 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
case NOUVEAU_FIFO_ENGINE_CE:
engine = NV_DEVICE_HOST_RUNLIST_ENGINES_CE;
break;
+ case 0x300:
+ engine = NV_DEVICE_HOST_RUNLIST_ENGINES_NVDEC;
+ break;
default:
return nouveau_abi16_put(abi16, -ENOSYS);
}
@@ -456,10 +459,12 @@ nouveau_abi16_ioctl_channel_alloc(ABI16_IOCTL_ARGS)
goto done;
break;
case NV_DEVICE_INFO_V0_TURING:
- ret = nvif_object_ctor(&chan->chan->user, "abi16CeWar", 0, TURING_DMA_COPY_A,
- NULL, 0, &chan->ce);
- if (ret)
- goto done;
+ if (engine != NV_DEVICE_HOST_RUNLIST_ENGINES_NVDEC) {
+ ret = nvif_object_ctor(&chan->chan->user, "abi16CeWar", 0, TURING_DMA_COPY_A,
+ NULL, 0, &chan->ce);
+ if (ret)
+ goto done;
+ }
break;
default:
break;
Next step will be to configure the kernel. The best option I’ll recommend you is to copy the kernel config, your distribution is shipping with. On Ubuntu you can find it in /boot
with the name config-6.8.0-52-generic
for example.
$ cp /boot/config-6.8.0-52-generic .config
Then to get the default config, your kernel will use, including the specific options coming with Ubuntu, you’ll have to run:
$ make defconfig
This will setup the build and make it ready to compile with this version of the kernel, auto configuring the new features.
Two options CONFIG_SYSTEM_TRUSTED_KEYS
and CONFIG_SYSTEM_REVOCATION_KEYS
must be disabled to avoid compilation errors with missing certificates. For that you can set it up within menuconfig
or you can edit .config
and set these values to ""
Then you should be ready to go for a break ☕, short or long depending on your machine to cook the brand new kernel debian packaged, ready to use:
$ make clean
$ make -j `getconf _NPROCESSORS_ONLN` deb-pkg LOCALVERSION=-custom
The process should end up with a new package named linux-image-6.12.8-custom_6.12.8-3_amd64.deb
in the upper folder which can then be installed along your previous kernel.
$ sudo dpkg -i ../linux-image-6.12.8-custom_6.12.8-3_amd64.deb
The first one will replace your current default menulist item in grub upon installation. This means that if you install it, next time you reboot, you’ll boot into that kernel.
Mesa build for NVK #
Now that we have a working kernel which allows to discuss with nvidia
codec engine using nouveau
driver. We can start the build of mesa.
Prerequisites #
Mesa depends on various system packages in addition to python modules and the the rust toolchain. So first we’ll have to install the given package which are all present in Ubuntu 24.04:
$ sudo apt install build-essential ninja-build glslang-tools python3-pip llvm byacc flex pkg-config bindgen libclc-19-dev libllvmspirvlib-18-dev libdrm-dev libclang-cpp-dev libclang-18-dev
$ sudo apt install libelf-dev libwayland-bin libwayland-dev libwayland-egl-backend-dev libx11-xcb-dev libxcb-randr0-dev libxfixes-dev libxcb-glx0-dev libxcb-shm0-dev libxcb-dri3-dev
$ sudo apt install libxcb-present-dev libxshmfence-dev libxxf86vm-dev
Then you’ll have to install the rust toolchain and some cargo package:
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ cargo install bindgen-cli
$ cargo install cbindgen
When all the system package are installed, you can install the python modules needed by Mesa build system.
$ sudo pip3 install --break-system-packages meson mako pyyaml
Configure and build #
As Mesa is highly depending on the system, you might need some additional packages but worth a try to launch the build with the given instructions:
$ cd ~/DEV/
$ git clone https://gitlab.freedesktop.org/dwlsalmeida/mesa.git --branch=nvk-vulkan-video
$ meson setup builddir --libdir lib64 --prefix `pwd`/builddir/install -Dbuildtype=debug -Dvulkan-drivers=nouveau -Dgallium-drivers=nouveau -Dtools=nouveau
$ ninja -C builddir install
Use of NVK driver #
Now that the kernel and the Mesa driver have been built and are available for your machine, you should be able to decode your first h264 stream with the NVK driver.
Important: Nouveau driver might be disabled !! #
As you might have use the Nvidia driver first and installed with your regular kernel, you might hit a weired error when invoking vulkaninfo
such as:
ERROR: [Loader Message] Code 0 : setup_loader_term_phys_devs: Failed to detect any valid GPUs in the current config
ERROR at ./vulkaninfo/./vulkaninfo.h:247:vkEnumeratePhysicalDevices failed with ERROR_INITIALIZATION_FAILED
Indeed nouveau
driver can not live along the Nvidia driver, so you’ll have to uninstall the Nvidia driver first to be able to use nouveau
properly and the vulkan extensions.
One other solution is to boot on your new custom kernel and modify the file /etc/modprobe.d/nvidia-installer-disable-nouveau.conf
to get something like:
# generated by nvidia-installer
#blacklist nouveau
options nouveau modeset=1
In that case the modeset=1
option will enable the driver and allow to use it.
Then you’ll have to reboot with this new configuration
Run vulkaninfo #
As you may have noticed, during the configure stage that we chose to install the artifacts of the build in a folder named mesa/builddir/install
.
Here is a script run-nvk.sh
which you can use before calling any binary which will use this folder as a base to set the environment variable dedicated to the NVK Vulkan driver
#!/bin/sh
MESA=$HOME/DEV/mesa/builddir/install \
LD_LIBRARY_PATH=$MESA/lib64:$MESA/lib:$LD_LIBRARY_PATH \
VK_ICD_FILENAMES=$MESA/share/vulkan/icd.d/nouveau_icd.x86_64.json \
VK_DRIVER_FILES=$MESA/share/vulkan/icd.d/nouveau_icd.x86_64.json \
NVK_I_WANT_A_BROKEN_VULKAN_DRIVER=true \
exec "$@"
After all these stage you should be able to run vulkaninfo
such as:
$ ~/DEV/SCRIPTS/run-nvk.sh vulkaninfo | grep video_decode
VK_KHR_video_decode_h264 : extension revision 9
VK_KHR_video_decode_h265 : extension revision 8
VK_KHR_video_decode_queue : extension revision 8
Run GStreamer playback #
Now its time to run a real application exploiting the power of Vulkan to decode multimedia content. For that I’ll recommend you to use GStreamer which ship with Vulkan elements for decoding in 1.24.2 version bundled in Ubuntu 24.04.
First of all, you’ll have to install the ubuntu packages for GStreamer
sudo apt install gstreamer1.0-plugins-bad gstreamer1.0-tools
Check that the elements are available with:
$ gst-inspect vulkan
vulkancolorconvert: Vulkan Color Convert
vulkandeviceprovider: Vulkan Device Provider
vulkandownload: Vulkan Downloader
vulkanh264dec: Vulkan H.264 decoder
vulkanh265dec: Vulkan H.265 decoder
vulkanimageidentity: Vulkan Image Identity
vulkanoverlaycompositor: Vulkan Overlay Compositor
vulkanshaderspv: Vulkan Shader SPV
vulkansink: Vulkan video sink
vulkanupload: Vulkan Uploader
vulkanviewconvert: Vulkan View Convert
If you succeed to see this list of elements, you should be able to run a GStreamer pipeline with Vulkan Video extensioms. Here is a pipeline to decode a content:
$ ~/DEV/SCRIPTS/run-nvk.sh gst-launch-1.0 urisourcebin uri=https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/1080/Big_Buck_Bunny_1080_10s_1MB.mp4 ! parsebin ! vulkanh264dec ! vulkandownload ! autovideosink
Hope you enjoyed this blog post and you are now able to decode h26x content with the open Vulkan Video extensions.
Thanks a lot to the work of David Airlie and Daniel Almeida who allow to decode h26x content with NVK driver.
- Previous: Vulkanised 2025