As part of my work in the graphics team at Igalia, I’ve been helping with the effort to enable GL_ARB_gl_spirv for Intel’s i965 driver in Mesa. Most of the functionality of the extension is already working so in order to get more test coverage and discover unknown missing features we have been working to automatically convert some Piglit GLSL tests to use SPIR-V. Most of the GLSL tests are using a tool internal to Piglit called shader_runner. This tool largely simplifies the work needed to create a test by allowing it to be specified as a simple text file that just contains the required shaders and a list of commands to execute using them. The commands are very highlevel such as draw rect to submit a rectangle or probe rgba to check for a specific colour of a pixel in the framebuffer.

A number of times during this work I’ve encountered problems that look like they are general problems with Mesa’s SPIR-V compiler and aren’t specific to the GL_ARB_gl_spirv work. I wanted a way to easily confirm this but writing a minimal test case from scratch in Vulkan seemed like quite a lot of hassle. Therefore I decided it would be nice to have a tool like shader_runner that works with Vulkan. I made a start on implementing this and called it VkRunner.

Example

Here is an example shader test file for VkRunner:

[vertex shader]
#version 430

layout(location = 0) in vec3 pos;

void
main()
{
        gl_Position = vec4(pos.xy * 2.0 - 1.0, 0.0, 1.0);
}

[fragment shader]
#version 430

layout(location = 0) out vec4 color_out;

layout(std140, push_constant) uniform block {
        vec4 color_in;
};

void
main()
{
        color_out = color_in;
}

[vertex data]
0/r32g32b32_sfloat

0.25 0.0775 0.0
0.145 0.3875 0.0
0.25 0.3175 0.0

0.25 0.0775 0.0
0.355 0.3875 0.0
0.25 0.3175 0.0

0.0775 0.195 0.0
0.4225 0.195 0.0
0.25 0.3175 0.0

[test]
# Clear the framebuffer to green
clear color 0.0 0.6 0.0 1.0
clear

# White rectangle in the topleft corner
uniform vec4 0 1.0 1.0 1.0 1.0
draw rect 0 0 0.5 0.5

# Green star in the topleft
uniform vec4 0 0.0 0.6 0.0 1.0
draw arrays TRIANGLE_LIST 0 9

# Verify a rectangle colour
relative probe rect rgb (0.5, 0.0, 0.5, 1.0) (0.0, 0.6, 0.0)

If this is run through VkRunner it will convert the shaders to SPIR-V on the fly by piping them through glslangValidator, create a pipeline and a framebuffer and then run the test commands. The framebuffer is just an offscreen buffer so VkRunner doesn’t use any window system extensions. However you can still see the result by passing the -i image.ppm option which cause it to write a PPM image of the final rendering.

The format is meant to be as close to shader_runner as possible but there are a few important differences. Vulkan can’t have uniforms in the default buffer and there is no way to access them by name. Instead you can use a push constant buffer and refer to the individual uniform using its byte offset in the buffer. VkRunner doesn’t yet support UBOs. The same goes for vertex attributes which are now specified using an explicit location rather than by name. In the vertex data section you can use names from VkFormat to specify the format with maximum flexibility, but it can also accept Piglit-style names for compatibilty.

Bonus features

VkRunner supports some extra features that shader_runner does not. Firstly you can specify a format for the framebuffer in the optional [require] section. For example you can make it use a floating-point framebuffer to be able to accurately probe results from functions.

[require]
framebuffer R32G32B32A32_SFLOAT

[vertex shader passthrough]

[fragment shader]
#version 430

layout(location = 0) out vec4 color;

void
main()
{
        color = vec4(atan(0.0, -1.0),
                     42.0,
                     length(vec2(1.0, 1.0)),
                     fma(2.0, 3.0, 1.0));
}

[test]
clear
draw rect -1 -1 2 2
probe all rgba 3.141592653589793 42.0 1.4142135623730951 7.0

If you want to use SPIR-V instead of GLSL in the source, you can specify the shader in a [fragment shader spirv] section. This is useful to test corner cases of the driver with tweaked shaders that glslang wouldn’t generate. You can get a base for the SPIR-V source by passing the -d option to VkRunner to get the disassembly of the generated SPIR-V. There is an example of this in the repo.

Status

Although it can already be used for a range of tests, VkRunner still has a lot of missing features compared to shader_runner. It does not support textures, UBOs and SSBOs. Take a look at the README for the complete documentation.

As I have been creating tests for problems that I encounter with Mesa I’ve been adding them to a branch called tests in the Github repo. I get the impression that Vulkan is somewhat under-tested compared to GL so it might be interesting to use these tests as a base to make a more complete Vulkan test suite. It could also potentially be merged into Piglit.

UPDATE: Patches to merge VkRunner into Piglit have been posted to the mailing list.