Everybody knows it: install Intel MediaSDK in GNU/Linux is a PITA. With CentOS or Yocto is less cumbersome, if you trust blindly on scripts ran as root.
I don’t like CentOS, I feel it like if I were living in the past. I like Debian (testing, of course) and I also wanted to understand a little more about MediaSDK. And this is what I did to have Intel MediaSDK working in Debian/testing.
First, I did a pristine installation of Debian testing with a netinst image in my NUC 6i5SYK, with a normal desktop user setup (Gnome3).
The madness comes later.
Intel’s identifies two types of MediaSDK installation: Gold and Generic. Gold is for CentOS, and Generic for the rest of distributions. Obviously, Generic means you’re on your own. For the purpose of this exercise I used as reference Generic Linux* Intel® Media Server Studio Installation.
Let’s begin by grabbing the Intel® Media Server Studio – Community Edition. You will need to register yourself and accept the user agreement, because this is proprietary software.
At the end, you should have a tarball named MediaServerStudioEssentials2017R3.tar.gz
Extract the files for Generic instalation
$ cd ~
$ tar xvf MediaServerStudioEssentials2017R3.tar.gz
$ cd MediaServerStudioEssentials2017R3
$ tar xvf SDK2017Production16.5.2.tar.gz
$ cd SDK2017Production16.5.2/Generic
$ mkdir tmp
$ tar -xvC tmp -f intel-linux-media_generic_16.5.2-64009_64bit.tar.gz
Kernel
Bad news: in order to get MediaSDK working you need to patch the mainlined kernel.
Worse news: the available patches are only for the version 4.4 the kernel.
Still, systemd works on 4.4, as far as I know, so it would not be a big problem.
Grab building dependencies
$ sudo apt install build-essential devscripts libncurses5-dev
$ sudo apt build-dep linux
Grab kernel source
I like to use the sources from the git repository, since it would be possible to do some rebasing and blaming in the future.
$ cd ~
$ git clone https://github.com/torvalds/linux.git
...
$ git pull -v --tags
$ git checkout -b 4.4 v4.4
Extract MediaSDK patches
$ cd ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/patches/kmd/4.4
$ tar xvf intel-kernel-patches.tar.bz2
$ cd intel-kernel-patches
$ PATCHDIR=$(pwd)
Patch the kernel
cd ~/linux
$ git am $PATCHDIR/*.patch
The patches should apply with some warnings but no fatal errors (don’t worry, be happy).
Still, there’s a problem with this old kernel: our recent compiler doesn’t build it as it is. Another patch is required:
$ wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8-rc2/0002-UBUNTU-SAUCE-no-up-disable-pie-when-gcc-has-it-enabl.patch
$ git am 0002-UBUNTU-SAUCE-no-up-disable-pie-when-gcc-has-it-enabl.patch
TODO: Shall I need to modify the EXTRAVERSION string in kernel’s Makefile?
Build and install the kernel
Notice that we are using our current kernel configuration. That is error prone. I guess that is why I had to select NVM manually.
$ cp /boot/config-4.12.0-1-amd64 ./.config
$ make olddefconfig
$ make nconfig # -- select NVM
$ scripts/config --disable DEBUG_INFO
$ make deb-pkg
...
$ sudo dpkg -i linux-image-4.4.0+_4.4.0+-2_amd64.deb linux-headers-4.4.0+_4.4.0+-2_amd64.deb linux-firmware-image-4.4.0+_4.4.0+-2_amd64.deb
Configure GRUB2 to boot Linux 4.4. by default
This part was absolutely tricky for me. It took me a long time to figure out how to specify the kernel ID in the grubenv.
$ sudo vi /etc/default/grub
And change the line GRUB_DEFAULT=saved. By default it is set to 0. And update GRUB.
$ sudo update-grub
Now look for the ID of the installed kernel image in /etc/grub/grub.cfg and use it:
$ sudo grub-set-default "gnulinux-4.4.0+-advanced-2c246bc6-65bb-48ea-9517-4081b016facc>gnulinux-4.4.0+-advanced-2c246bc6-65bb-48ea-9517-4081b016facc"
Please note it is twice and separated by a >. Don’t ask me why.
Copy MediaSDK firmware (and libraries too)
I like to use rsync rather normal cp because there are the options like --dry-run and --itemize-changes to verify what I am doing.
$ cd ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp
$ sudo rsync -av --itemize-changes ./lib /
$ sudo rsync -av --itemize-changes ./opt/intel/common /opt/intel
$ sudo rsync -av --itemize-changes ./opt/intel/mediasdk/{include,lib64,plugins} /opt/intel/mediasdk
All these directories contain blobs that do the MediaSDK magic. They are dlopened by hard coded paths by mfx_dispatch, which will be explain later.
In /lib lives the firmware (kernel blob).
In /opt/intel/common… I have no idea what are those shared objects.
In /opt/intel/mediasdk/include live header files for programming an compilation.
In /opt/intel/mediasdk/lib64 live the driver for the modified libva (iHD) and other libraries.
In /opt/intel/mediasdk/plugins live, well, plugins…
In conclusion, all these bytes are darkness and mystery.
Reboot
$ sudo systemctl reboot
The system should boot, automatically, in GNU/Linux 4.4.
Please, log with Xorg, not in Wayland, since it is not supported, as far as I know.
GStreamer
For compiling GStreamer I will use gst-uninstalled. Someone may say that I should use gst-build because is newer and faster, but I feel more comfortable doing the following kind of hacks with the old&good autotools.
Basically this is a reproduction of Quick-start guide to gst-uninstalled for GStreamer 1.x.
$ sudo apt build-dep gst-plugins-{base,good,bad}1.0
$ wget https://cgit.freedesktop.org/gstreamer/gstreamer/plain/scripts/create-uninstalled-setup.sh -q -O - | sh
I will modify the gst-uninstalled script, and keep it outside of the repository. For that I will use the systemd file-hierarchy spec for user’s executables.
$ cd ~/gst
$ mkdir -p ~/.local/bin
$ mv master/gstreamer/scripts/gst-uninstalled ~/.local/bin
$ ln -sf ~/.local/bin/gst-uninstalled ./gst-master
Do not forget to edit your ~/.profile to add ~/.local/bin in the environment variable PATH.
Patch ~/.local/bin/gst-uninstalled
The modifications are to handle the three dependencies libraries that are required by MediaSDK: libdrm, libva and mfx_dispatch.
diff --git a/scripts/gst-uninstalled b/scripts/gst-uninstalled
index 81f83b6c4..d79f19abd 100755
--- a/scripts/gst-uninstalled
+++ b/scripts/gst-uninstalled
@@ -122,7 +122,7 @@ GI_TYPELIB_PATH=$GST/gstreamer/gst:$GI_TYPELIB_PATH
export LD_LIBRARY_PATH
export DYLD_LIBRARY_PATH
export GI_TYPELIB_PATH
-
+
export PKG_CONFIG_PATH="\
$GST_PREFIX/lib/pkgconfig\
:$GST/gstreamer/pkgconfig\
@@ -140,6 +140,9 @@ $GST_PREFIX/lib/pkgconfig\
:$GST/orc\
:$GST/farsight2\
:$GST/libnice/nice\
+:$GST/drm\
+:$GST/libva/pkgconfig\
+:$GST/mfx_dispatch\
${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
export GST_PLUGIN_PATH="\
@@ -227,6 +230,16 @@ export GST_VALIDATE_APPS_DIR=$GST_VALIDATE_APPS_DIR:$GST/gst-editing-services/te
export GST_VALIDATE_PLUGIN_PATH=$GST_VALIDATE_PLUGIN_PATH:$GST/gst-devtools/validate/plugins/
export GIO_EXTRA_MODULES=$GST/prefix/lib/gio/modules:$GIO_EXTRA_MODULES
+# MediaSDK
+export LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib64
+export LIBVA_DRIVER_NAME=iHD
+export LD_LIBRARY_PATH="\
+/opt/intel/common/mdf/lib64\
+:$GST/drm/.libs\
+:$GST/drm/intel/.libs\
+:$GST/libva/va/.libs\
+:$LD_LIBRARY_PATH"
+
Now, initialize the gst-uninstalled environment:
$ cd ~/gst
$ ./gst-master
libdrm
Grab libdrm from its repository and switch to the branch with the supported version by MediaSDK.
$ cd ~/gst/master
$ git clone git://anongit.freedesktop.org/mesa/drm
$ cd drm
$ git checkout -b intel libdrm-2.4.67
Extract the distributed tarball in the cloned repository.
$ tar -xv --strip-components=1 -C . -f ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/libdrm/2.4.67-64009/libdrm-2.4.67.tar.bz2
Then we could check the big delta between upstream and the changes done by Intel for MediaSDK.
Let’s put it in a commit for later rebases.
$ git add -u
$ git add .
$ git commit -m "mediasdk changes"
Get build dependencies and compile.
$ sudo apt build-dep libdrm
$ ./configure
$ make -j8
Since the pkgconfig files (*.pc) of libdrm are generated to work installed, it is needed to modify them in order to work uninstalled.
$ prefix=${HOME}/gst/master/drm
$ sed -i -e "s#^libdir=.*#libdir=${prefix}/.libs#" ${prefix}/*.pc
$ sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/*.pc
In order to C preprocessor could find the uninstalled libdrm header files we need to make them available in the expected path according to the pkgconfig file and right now they are not there. To fix that it is possible to create proper symbolic links.
$ cd ~/gst/master/drm
$ ln -s include/drm/ libdrm
libva
This modified a version of libva. These modifications messed a bit with the opensource version of libva, because Intel decided not to prefix the library, or some other strategy. In gstreamer-vaapi we had to blacklist VA-API version 0.99, because it is the version number, arbitrary set, of this modified version of libva for MediaSDK.
Again, grab the original libva from repo and change the branch aiming to the divert point. It was difficult to find the divert commit id since even the libva version number was changed. Doing some archeology I guessed the branch point was in version 1.0.15, but I’m not sure.
$ cd ~/gst/master
$ git clone https://github.com/01org/libva.git
$ cd libva
$ git checkout -b intel libva-1.0.15
$ tar -xv --strip-components=1 -C . -f ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/libva/1.67.0.pre1-64009/libva-1.67.0.pre1.tar.bz2
$ git add -u
$ git add .
$ git commit -m "mediasdk"
Before compile, verify that Makefile is going to link against the uninstalled libdrm. You can do that by grepping for LIBDRM in Makefile.
Get compilation dependencies and build.
$ sudo apt build-dep libva
$ ./configure
$ make -j8
Moidify the pkgconfig files for uninstalled
$ prefix=${HOME}/gst/master/libva
$ sed -i -e "s#^libdir=.*#libdir=${prefix}/va/.libs#" ${prefix}/pkgconfig/*.pc
$ sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/pkgconfig/*.pc
Fix header path with symbolic links
$ cd ~/gst/master/libva/va
$ ln -sf drm/va_drm.h
mfx_dispatch
This static library which must be linked with MediaSDK applications. In our case, to the GStreamer plugin.
According to its documentation (included in the tarball):
the dispatcher is a layer that lies between application and the SDK implementations. Upon initialization, the dispatcher locates the appropiate platform-specific SDK implementation. If there is none, it will select the software SDK implementation. The dispatcher will redirect subsequent function calls to the same functions in the selected SDK implementation.
In the tarball there is the source of the mfx_dispatcher, but it only compiles with cmake. I have not worked with cmake on uninstalled setups, but we are lucky, there is a repository with autotools support:
$ cd ~/gst/master
$ git clone https://github.com/lu-zero/mfx_dispatch.git
And compile. After running ./configure it is better to confirm, grepping the generated Makefie, that the uninstalled versions of libdrm and libva are going to be used.
$ autoreconf --install
$ ./configure
$ make -j8
Finally, just as the other libraries, it is required to fix the pkgconfig files:d
$ prefix=${HOME}/gst/master/mfx_dispatch
$ sed -i -e "s#^libdir=.*#libdir=${prefix}/.libs#" ${prefix}/*.pc
$ sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/*.pc
Test it!
At last we are in a position where it is possible to test if everything works as expected. For it we are going to run the pre-compiled version of vainfo bundled in the tarball.
We will copy it to our uninstalled setup, thus we would running without specifing the path.
$ sync -av /home/vjaquez/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/usr/bin/vainfo ./prefix/bin/
$ vainfo
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.99 (libva 1.67.0.pre1)
vainfo: Driver version: 16.5.2.64009-ubit
vainfo: Supported profile and entrypoints
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: <unknown entrypoint>
VAProfileH264ConstrainedBaseline: <unknown entrypoint>
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : <unknown entrypoint>
VAProfileH264Main : <unknown entrypoint>
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : <unknown entrypoint>
VAProfileH264High : <unknown entrypoint>
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileVP8Version0_3 : VAEntrypointEncSlice
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileVP8Version0_3 : <unknown entrypoint>
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileVP9Profile0 : <unknown entrypoint>
<unknown profile> : VAEntrypointVideoProc
VAProfileNone : VAEntrypointVideoProc
VAProfileNone : <unknown entrypoint>
It works!
Compile GStreamer
I normally make a copy of ~/gst/master/gstreamer/script/git-update.sh in ~/.local/bin in order to modify it, like adding support for ccache, disabling gtkdoc and gobject-instrospections, increase the parallel tasks, etc. But that is out of the scope of this document.
$ cd ~/gst/master/
$ ./gstreamer/scripts/git-update.sh
Everything should be built without issues and, at the end, we could test if the gst-msdk elements are available:
$ gst-inspect-1.0 msdk
Plugin Details:
Name msdk
Description Intel Media SDK encoders
Filename /home/vjaquez/gst/master/gst-plugins-bad/sys/msdk/.libs/libgstmsdk.so
Version 1.13.0.1
License BSD
Source module gst-plugins-bad
Source release date 2017-11-23 16:39 (UTC)
Binary package GStreamer Bad Plug-ins git
Origin URL Unknown package origin
msdkh264dec: Intel MSDK H264 decoder
msdkh264enc: Intel MSDK H264 encoder
msdkh265dec: Intel MSDK H265 decoder
msdkh265enc: Intel MSDK H265 encoder
msdkmjpegdec: Intel MSDK MJPEG decoder
msdkmjpegenc: Intel MSDK MJPEG encoder
msdkmpeg2enc: Intel MSDK MPEG2 encoder
msdkvp8dec: Intel MSDK VP8 decoder
msdkvp8enc: Intel MSDK VP8 encoder
9 features:
+-- 9 elements
Great!
Now, let’s run a simple pipeline. Please note that gst-msdk elements have rank zero, then they will not be autoplugged, it is necessary to craft the pipeline manually:
$ gst-launch-1.0 filesrc location= ~/test.264 ! h264parse ! msdkh264dec ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:02.502411331
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
\o/