Herostratus’ legacy

words from a lazy coder

Vulkan Video Ecosystem Status #

This page intends to provide up-to-date links and information for the latest driver and application support for Khronos Vulkan Video extension. It gathers efforts from different persons, organizations and companies. Special thanks to Lynne (FFmpeg) and Dave Airlie (Mesa / RedHat) for all their work, along with Igalia (GStreamer / Mesa)

Last update: 2024-02-29

Drivers #

Vulkan Video extension support in Vulkan Drivers
H.264 H.265 AV1
Encode On the roadmap

SDK and specification #

Applications #

References #

Presentations #

Vulkanised 2024: Implementing a Video Encoder From Mesa to GStreamer - Stéphane Cerveau & Hyunjun Ko

Vulkanised 2024: Vulkan Video Encode API: A Deep Dive - Tony Zlatinski

XDC 2023: Status of the Vulkan Video ecosystem. Charles Turner, Igalia

GStreamer Conference 2023: Vulkan Video in GStreamer. Stéphane Cerveau & Victor Jáquez, Igalia

Vulkanised 2023: A deep dive into Vulkan Video. Tony Zlatinski, NVIDIA

Vulkanised 2023: Implementing a Vulkan decoder with GStreamer. Victor Jáquez & Stéphane Cerveau, Igalia

XDC 2021: Video decoding in Vulkan: VK_KHR_video_queue/decode APIs. Víctor Jáquez, Igalia

Examples #

GStreamer #

GStreamer decode #

gst-launch-1.0 filesrc location=./video.mp4 ! parsebin ! vulkanh264dec ! vulkandownload ! videoconvert ! autovideosink

GStreamer encode #

gst-launch-1.0 videotestsrc ! video/x-raw, format=NV12 ! vulkanupload ! vulkanh264enc ! vulkanh264dec ! vulkandownload ! videoconvert ! autovideosink sync=false

FFmpeg #

FFmpeg decode #

ffmpeg -hwaccel vulkan -init_hw_device vulkan=vk:0,debug=0 -hwaccel_output_format vulkan -i input -vf 'hwdownload,format=nv12' -pix_fmt yuv420p output.y4m

🚧 FFmpeg encode #

ffmpeg -y -hide_banner -hwaccel vulkan -hwaccel_output_format vulkan -i input.y4m -vf 'format=nv12,hwupload' -c:v h264_vulkan -frames 100 output_h264_vulkan.mp4