Intel MediaSDK on Debian (testing)

24 November 2017
debian
gstreamer
mediasdk

Everybody knows it: install Intel MediaSDK in GNU/Linux is a PITA. With CentOS or Yocto is less cumbersome, if you trust blindly on scripts ran as root.I don’t like CentOS, I feel it like if I were living in the past. I like Debian (testing, of course) and I also wanted to understand a little more about MediaSDK. And this is what I did to have Intel MediaSDK working in Debian/testing.

First, I did a pristine installation of Debian testing with a netinst image in my NUC 6i5SYK, with a normal desktop user setup (Gnome3).

The madness comes later.

Intel’s identifies two types of MediaSDK installation: Gold and Generic. Gold is for CentOS, and Generic for the rest of distributions. Obviously, Generic means you’re on your own. For the purpose of this exercise I used as reference Generic Linux* Intel® Media Server Studio Installation.

Let’s begin by grabbing the Intel® Media Server Studio – Community Edition. You will need to register yourself and accept the user agreement, because this is proprietary software.

At the end, you should have a tarball named MediaServerStudioEssentials2017R3.tar.gz

Extract the files for Generic installation #

cd ~
tar xvf MediaServerStudioEssentials2017R3.tar.gz
cd MediaServerStudioEssentials2017R3
tar xvf SDK2017Production16.5.2.tar.gz
cd SDK2017Production16.5.2/Generic
mkdir tmp
tar -xvC tmp -f intel-linux-media_generic_16.5.2-64009_64bit.tar.gz

Kernel #

Bad news: in order to get MediaSDK working you need to patch the mainlined kernel.

Worse news: the available patches are only for the version 4.4 the kernel.

Still, systemd works on 4.4, as far as I know, so it would not be a big problem.

Grab building dependencies #

sudo apt install build-essential devscripts libncurses5-dev
sudo apt build-dep linux

Grab kernel source #

I like to use the sources from the git repository, since it would be possible to do some rebasing and blaming in the future.

cd ~
git clone https://github.com/torvalds/linux.git
git pull -v --tags
git checkout -b 4.4 v4.4

Extract MediaSDK patches #

cd ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/patches/kmd/4.4
tar xvf intel-kernel-patches.tar.bz2
cd intel-kernel-patches
PATCHDIR=$(pwd)

Patch the kernel #

cd ~/linux
git am $PATCHDIR/*.patch

The patches should apply with some warnings but no fatal errors (don’t worry, be happy).

Still, there’s a problem with this old kernel: our recent compiler doesn’t build it as it is. Another patch is required:

wget http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8-rc2/0002-UBUNTU-SAUCE-no-up-disable-pie-when-gcc-has-it-enabl.patch
git am 0002-UBUNTU-SAUCE-no-up-disable-pie-when-gcc-has-it-enabl.patch

TODO: Shall I need to modify the EXTRAVERSION string in kernel’s Makefile?

Build and install the kernel #

Notice that we are using our current kernel configuration. That is error prone. I guess that is why I had to select NVM manually.

cp /boot/config-4.12.0-1-amd64 ./.config
make olddefconfig
make nconfig # -- select NVM
scripts/config --disable DEBUG_INFO
make deb-pkg
sudo dpkg -i linux-image-4.4.0+_4.4.0+-2_amd64.deb linux-headers-4.4.0+_4.4.0+-2_amd64.deb linux-firmware-image-4.4.0+_4.4.0+-2_amd64.deb

Configure GRUB2 to boot Linux 4.4. by default #

This part was absolutely tricky for me. It took me a long time to figure out how to specify the kernel ID in the grubenv.

sudo vi /etc/default/grub

And change the line GRUB_DEFAULT=saved. By default it is set to 0. And update GRUB.

sudo update-grub

Now look for the ID of the installed kernel image in /etc/grub/grub.cfg and use it:

sudo grub-set-default "gnulinux-4.4.0+-advanced-2c246bc6-65bb-48ea-9517-4081b016facc>gnulinux-4.4.0+-advanced-2c246bc6-65bb-48ea-9517-4081b016facc"

Please note it is twice and separated by a >. Don’t ask me why.

Copy MediaSDK firmware (and libraries too) #

I like to use rsync rather normal cp because there are the options like --dry-run and --itemize-changes to verify what I am doing.

cd ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp
sudo rsync -av --itemize-changes ./lib /
sudo rsync -av --itemize-changes ./opt/intel/common /opt/intel
sudo rsync -av --itemize-changes ./opt/intel/mediasdk/{include,lib64,plugins} /opt/intel/mediasdk

All these directories contain blobs that do the MediaSDK magic. They are dlopened by hard coded paths by mfx_dispatch, which will be explain later.

In /lib lives the firmware (kernel blob).

In /opt/intel/common… I have no idea what are those shared objects.

In /opt/intel/mediasdk/include live header files for programming an compilation.

In /opt/intel/mediasdk/lib64 live the driver for the modified libva (iHD) and other libraries.

In /opt/intel/mediasdk/plugins live, well, plugins…

In conclusion, all these bytes are darkness and mystery.

Reboot #

sudo systemctl reboot

The system should boot, automatically, in GNU/Linux 4.4.

Please, log with Xorg, not in Wayland, since it is not supported, as far as I know.

GStreamer #

For compiling GStreamer I will use gst-uninstalled. Someone may say that I should use gst-build because is newer and faster, but I feel more comfortable doing the following kind of hacks with the old&good autotools.

Basically this is a reproduction of Quick-start guide to gst-uninstalled for GStreamer 1.x.

sudo apt build-dep gst-plugins-{base,good,bad}1.0
wget https://cgit.freedesktop.org/gstreamer/gstreamer/plain/scripts/create-uninstalled-setup.sh -q -O - | sh

I will modify the gst-uninstalled script, and keep it outside of the repository. For that I will use the systemd file-hierarchy spec for user’s executables.

cd ~/gst
mkdir -p ~/.local/bin
mv master/gstreamer/scripts/gst-uninstalled ~/.local/bin
ln -sf ~/.local/bin/gst-uninstalled ./gst-master

Do not forget to edit your ~/.profile to add ~/.local/bin in the environment variable PATH.

Patch ~/.local/bin/gst-uninstalled #

The modifications are to handle the three dependencies libraries that are required by MediaSDK: libdrm, libva and mfx_dispatch.

diff --git a/scripts/gst-uninstalled b/scripts/gst-uninstalled
index 81f83b6c4..d79f19abd 100755
--- a/scripts/gst-uninstalled
+++ b/scripts/gst-uninstalled
@@ -122,7 +122,7 @@ GI_TYPELIB_PATH=$GST/gstreamer/gst:$GI_TYPELIB_PATH
 export LD_LIBRARY_PATH
 export DYLD_LIBRARY_PATH
 export GI_TYPELIB_PATH
-
+
 export PKG_CONFIG_PATH="\
 $GST_PREFIX/lib/pkgconfig\
 :$GST/gstreamer/pkgconfig\
@@ -140,6 +140,9 @@ $GST_PREFIX/lib/pkgconfig\
 :$GST/orc\
 :$GST/farsight2\
 :$GST/libnice/nice\
+:$GST/drm\
+:$GST/libva/pkgconfig\
+:$GST/mfx_dispatch\
 ${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"

 export GST_PLUGIN_PATH="\
@@ -227,6 +230,16 @@ export GST_VALIDATE_APPS_DIR=$GST_VALIDATE_APPS_DIR:$GST/gst-editing-services/te
 export GST_VALIDATE_PLUGIN_PATH=$GST_VALIDATE_PLUGIN_PATH:$GST/gst-devtools/validate/plugins/
 export GIO_EXTRA_MODULES=$GST/prefix/lib/gio/modules:$GIO_EXTRA_MODULES

+# MediaSDK
+export LIBVA_DRIVERS_PATH=/opt/intel/mediasdk/lib64
+export LIBVA_DRIVER_NAME=iHD
+export LD_LIBRARY_PATH="\
+/opt/intel/common/mdf/lib64\
+:$GST/drm/.libs\
+:$GST/drm/intel/.libs\
+:$GST/libva/va/.libs\
+:$LD_LIBRARY_PATH"
+

Now, initialize the gst-uninstalled environment:

cd ~/gst
./gst-master

libdrm

Grab libdrm from its repository and switch to the branch with the supported version by MediaSDK.

cd ~/gst/master
git clone git://anongit.freedesktop.org/mesa/drm
cd drm
git checkout -b intel libdrm-2.4.67

Extract the distributed tarball in the cloned repository.

tar -xv --strip-components=1 -C . -f ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/libdrm/2.4.67-64009/libdrm-2.4.67.tar.bz2

Then we could check the big delta between upstream and the changes done by Intel for MediaSDK.

Let’s put it in a commit for later rebases.

git add -u
git add .
git commit -m "mediasdk changes"

Get build dependencies and compile.

sudo apt build-dep libdrm
./configure
make -j8

Since the pkgconfig files (*.pc) of libdrm are generated to work installed, it is needed to modify them in order to work uninstalled.

prefix=${HOME}/gst/master/drm
sed -i -e "s#^libdir=.*#libdir=${prefix}/.libs#" ${prefix}/*.pc
sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/*.pc

In order to C preprocessor could find the uninstalled libdrm header files we need to make them available in the expected path according to the pkgconfig file and right now they are not there. To fix that it is possible to create proper symbolic links.

cd ~/gst/master/drm
ln -s include/drm/ libdrm

libva #

This modified a version of libva. These modifications messed a bit with the opensource version of libva, because Intel decided not to prefix the library, or some other strategy. In gstreamer-vaapi we had to blacklist VA-API version 0.99, because it is the version number, arbitrary set, of this modified version of libva for MediaSDK.

Again, grab the original libva from repo and change the branch aiming to the divert point. It was difficult to find the divert commit id since even the libva version number was changed. Doing some archaeology I guessed the branch point was in version 1.0.15, but I’m not sure.

cd ~/gst/master
git clone https://github.com/01org/libva.git
cd libva
git checkout -b intel libva-1.0.15
tar -xv --strip-components=1 -C . -f ~/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/opt/intel/mediasdk/opensource/libva/1.67.0.pre1-64009/libva-1.67.0.pre1.tar.bz2
git add -u
git add .
git commit -m "mediasdk"

Before compile, verify that Makefile is going to link against the uninstalled libdrm. You can do that by grepping for LIBDRM in Makefile.

Get compilation dependencies and build.

sudo apt build-dep libva
./configure
make -j8

Modify the pkgconfig files for uninstalled

prefix=${HOME}/gst/master/libva
sed -i -e "s#^libdir=.*#libdir=${prefix}/va/.libs#" ${prefix}/pkgconfig/*.pc
sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/pkgconfig/*.pc

Fix header path with symbolic links

cd ~/gst/master/libva/va
ln -sf drm/va_drm.h

mfx_dispatch #

This static library which must be linked with MediaSDK applications. In our case, to the GStreamer plugin.

According to its documentation (included in the tarball):

the dispatcher is a layer that lies between application and the SDK implementations. Upon initialization, the dispatcher locates the appropriate platform-specific SDK implementation. If there is none, it will select the software SDK implementation. The dispatcher will redirect subsequent function calls to the same functions in the selected SDK implementation.

In the tarball there is the source of the mfx_dispatcher, but it only compiles with cmake. I have not worked with cmake on uninstalled setups, but we are lucky, there is a repository with autotools support:

cd ~/gst/master
git clone https://github.com/lu-zero/mfx_dispatch.git

And compile. After running ./configure it is better to confirm, grepping the generated Makefie, that the uninstalled versions of libdrm and libva are going to be used.

autoreconf  --install
./configure
make -j8

Finally, just as the other libraries, it is required to fix the pkgconfig files:d

prefix=${HOME}/gst/master/mfx_dispatch
sed -i -e "s#^libdir=.*#libdir=${prefix}/.libs#" ${prefix}/*.pc
sed -i -e "s#^includedir=.*#includedir=${prefix}#" ${prefix}/*.pc

Test it #

At last we are in a position where it is possible to test if everything works as expected. For it we are going to run the pre-compiled version of vainfo bundled in the tarball.

We will copy it to our uninstalled setup, thus we would running without specifying the path.

$ sync -av /home/vjaquez/MediaServerStudioEssentials2017R3/SDK2017Production16.5.2/Generic/tmp/usr/bin/vainfo ./prefix/bin/
$ vainfo
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.99 (libva 1.67.0.pre1)
vainfo: Driver version: 16.5.2.64009-ubit
vainfo: Supported profile and entrypoints
     VAProfileH264ConstrainedBaseline: VAEntrypointVLD
     VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
     VAProfileH264ConstrainedBaseline: <unknown entrypoint>
     VAProfileH264ConstrainedBaseline: <unknown entrypoint>
     VAProfileH264Main               : VAEntrypointVLD
     VAProfileH264Main               : VAEntrypointEncSlice
     VAProfileH264Main               : <unknown entrypoint>
     VAProfileH264Main               : <unknown entrypoint>
     VAProfileH264High               : VAEntrypointVLD
     VAProfileH264High               : VAEntrypointEncSlice
     VAProfileH264High               : <unknown entrypoint>
     VAProfileH264High               : <unknown entrypoint>
     VAProfileMPEG2Simple            : VAEntrypointEncSlice
     VAProfileMPEG2Simple            : VAEntrypointVLD
     VAProfileMPEG2Main              : VAEntrypointEncSlice
     VAProfileMPEG2Main              : VAEntrypointVLD
     VAProfileVC1Advanced            : VAEntrypointVLD
     VAProfileVC1Main                : VAEntrypointVLD
     VAProfileVC1Simple              : VAEntrypointVLD
     VAProfileJPEGBaseline           : VAEntrypointVLD
     VAProfileJPEGBaseline           : VAEntrypointEncPicture
     VAProfileVP8Version0_3          : VAEntrypointEncSlice
     VAProfileVP8Version0_3          : VAEntrypointVLD
     VAProfileVP8Version0_3          : <unknown entrypoint>
     VAProfileHEVCMain               : VAEntrypointVLD
     VAProfileHEVCMain               : VAEntrypointEncSlice
     VAProfileVP9Profile0            : <unknown entrypoint>
     <unknown profile>               : VAEntrypointVideoProc
     VAProfileNone                   : VAEntrypointVideoProc
     VAProfileNone                   : <unknown entrypoint>

It works!

Compile GStreamer #

I normally make a copy of ~/gst/master/gstreamer/script/git-update.sh in ~/.local/bin in order to modify it, like adding support for ccache, disabling gtkdoc and gobject-introspection, increase the parallel tasks, etc. But that is out of the scope of this document.

cd ~/gst/master/
./gstreamer/scripts/git-update.sh

Everything should be built without issues and, at the end, we could test if the gst-msdk elements are available:

$ gst-inspect-1.0 msdk
Plugin Details:
 Name                     msdk
 Description              Intel Media SDK encoders
 Filename                 /home/vjaquez/gst/master/gst-plugins-bad/sys/msdk/.libs/libgstmsdk.so
 Version                  1.13.0.1
 License                  BSD
 Source module            gst-plugins-bad
 Source release date      2017-11-23 16:39 (UTC)
 Binary package           GStreamer Bad Plug-ins git
 Origin URL               Unknown package origin

 msdkh264dec: Intel MSDK H264 decoder
 msdkh264enc: Intel MSDK H264 encoder
 msdkh265dec: Intel MSDK H265 decoder
 msdkh265enc: Intel MSDK H265 encoder
 msdkmjpegdec: Intel MSDK MJPEG decoder
 msdkmjpegenc: Intel MSDK MJPEG encoder
 msdkmpeg2enc: Intel MSDK MPEG2 encoder
 msdkvp8dec: Intel MSDK VP8 decoder
 msdkvp8enc: Intel MSDK VP8 encoder

 9 features:
 +-- 9 elements

Great!

Now, let’s run a simple pipeline. Please note that gst-msdk elements have rank zero, then they will not be autoplugged, it is necessary to craft the pipeline manually:

$ gst-launch-1.0 filesrc location= ~/test.264 ! h264parse ! msdkh264dec ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:02.502411331
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

\o/

Previous: GStreamer Conference 2017
Next: Enabling HuC for SKL/KBL in Debian/testing

Herostratus’ legacy

words from a lazy coder