pyrox.dev
1# GPU acceleration {#sec-gpu-accel}
2
3NixOS provides various APIs that benefit from GPU hardware acceleration,
4such as VA-API and VDPAU for video playback; OpenGL and Vulkan for 3D
5graphics; and OpenCL for general-purpose computing. This chapter
6describes how to set up GPU hardware acceleration (as far as this is not
7done automatically) and how to verify that hardware acceleration is
8indeed used.
9
10Most of the aforementioned APIs are agnostic with regards to which
11display server is used. Consequently, these instructions should apply
12both to the X Window System and Wayland compositors.
13
14## OpenCL {#sec-gpu-accel-opencl}
15
16[OpenCL](https://en.wikipedia.org/wiki/OpenCL) is a general compute API.
17It is used by various applications such as Blender and Darktable to
18accelerate certain operations.
19
20OpenCL applications load drivers through the *Installable Client Driver*
21(ICD) mechanism. In this mechanism, an ICD file specifies the path to
22the OpenCL driver for a particular GPU family. In NixOS, there are two
23ways to make ICD files visible to the ICD loader. The first is through
24the `OCL_ICD_VENDORS` environment variable. This variable can contain a
25directory which is scanned by the ICL loader for ICD files. For example:
26
27```ShellSession
28$ export \
29 OCL_ICD_VENDORS=`nix-build '<nixpkgs>' --no-out-link -A rocmPackages.clr.icd`/etc/OpenCL/vendors/
30```
31
32The second mechanism is to add the OpenCL driver package to
33[](#opt-hardware.opengl.extraPackages).
34This links the ICD file under `/run/opengl-driver`, where it will be visible
35to the ICD loader.
36
37The proper installation of OpenCL drivers can be verified through the
38`clinfo` command of the clinfo package. This command will report the
39number of hardware devices that is found and give detailed information
40for each device:
41
42```ShellSession
43$ clinfo | head -n3
44Number of platforms 1
45Platform Name AMD Accelerated Parallel Processing
46Platform Vendor Advanced Micro Devices, Inc.
47```
48
49### AMD {#sec-gpu-accel-opencl-amd}
50
51Modern AMD [Graphics Core
52Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
53supported through the rocmPackages.clr.icd package. Adding this package to
54[](#opt-hardware.opengl.extraPackages)
55enables OpenCL support:
56
57```nix
58{
59 hardware.opengl.extraPackages = [
60 rocmPackages.clr.icd
61 ];
62}
63```
64
65### Intel {#sec-gpu-accel-opencl-intel}
66
67[Intel Gen8 and later
68GPUs](https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8)
69are supported by the Intel NEO OpenCL runtime that is provided by the
70intel-compute-runtime package. The proprietary Intel OpenCL runtime, in
71the intel-ocl package, is an alternative for Gen7 GPUs.
72
73The intel-compute-runtime or intel-ocl package can be added to
74[](#opt-hardware.opengl.extraPackages)
75to enable OpenCL support. For example, for Gen8 and later GPUs, the following
76configuration can be used:
77
78```nix
79{
80 hardware.opengl.extraPackages = [
81 intel-compute-runtime
82 ];
83}
84```
85
86## Vulkan {#sec-gpu-accel-vulkan}
87
88[Vulkan](https://en.wikipedia.org/wiki/Vulkan_(API)) is a graphics and
89compute API for GPUs. It is used directly by games or indirectly though
90compatibility layers like
91[DXVK](https://github.com/doitsujin/dxvk/wiki).
92
93By default, if [](#opt-hardware.opengl.driSupport)
94is enabled, mesa is installed and provides Vulkan for supported hardware.
95
96Similar to OpenCL, Vulkan drivers are loaded through the *Installable
97Client Driver* (ICD) mechanism. ICD files for Vulkan are JSON files that
98specify the path to the driver library and the supported Vulkan version.
99All successfully loaded drivers are exposed to the application as
100different GPUs. In NixOS, there are two ways to make ICD files visible
101to Vulkan applications: an environment variable and a module option.
102
103The first option is through the `VK_ICD_FILENAMES` environment variable.
104This variable can contain multiple JSON files, separated by `:`. For
105example:
106
107```ShellSession
108$ export \
109 VK_ICD_FILENAMES=`nix-build '<nixpkgs>' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json
110```
111
112The second mechanism is to add the Vulkan driver package to
113[](#opt-hardware.opengl.extraPackages).
114This links the ICD file under `/run/opengl-driver`, where it will be
115visible to the ICD loader.
116
117The proper installation of Vulkan drivers can be verified through the
118`vulkaninfo` command of the vulkan-tools package. This command will
119report the hardware devices and drivers found, in this example output
120amdvlk and radv:
121
122```ShellSession
123$ vulkaninfo | grep GPU
124 GPU id : 0 (Unknown AMD GPU)
125 GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
126 ...
127GPU0:
128 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
129 deviceName = Unknown AMD GPU
130GPU1:
131 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
132```
133
134A simple graphical application that uses Vulkan is `vkcube` from the
135vulkan-tools package.
136
137### AMD {#sec-gpu-accel-vulkan-amd}
138
139Modern AMD [Graphics Core
140Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
141supported through either radv, which is part of mesa, or the amdvlk
142package. Adding the amdvlk package to
143[](#opt-hardware.opengl.extraPackages)
144makes amdvlk the default driver and hides radv and lavapipe from the device list.
145A specific driver can be forced as follows:
146
147```nix
148{
149 hardware.opengl.extraPackages = [
150 pkgs.amdvlk
151 ];
152
153 # To enable Vulkan support for 32-bit applications, also add:
154 hardware.opengl.extraPackages32 = [
155 pkgs.driversi686Linux.amdvlk
156 ];
157
158 # Force radv
159 environment.variables.AMD_VULKAN_ICD = "RADV";
160 # Or
161 environment.variables.VK_ICD_FILENAMES =
162 "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
163}
164```
165
166## VA-API {#sec-gpu-accel-va-api}
167
168[VA-API (Video Acceleration API)](https://www.intel.com/content/www/us/en/developer/articles/technical/linuxmedia-vaapi.html)
169is an open-source library and API specification, which provides access to
170graphics hardware acceleration capabilities for video processing.
171
172VA-API drivers are loaded by `libva`. The version in nixpkgs is built to search
173the opengl driver path, so drivers can be installed in
174[](#opt-hardware.opengl.extraPackages).
175
176VA-API can be tested using:
177
178```ShellSession
179$ nix-shell -p libva-utils --run vainfo
180```
181
182### Intel {#sec-gpu-accel-va-api-intel}
183
184Modern Intel GPUs use the iHD driver, which can be installed with:
185
186```nix
187{
188 hardware.opengl.extraPackages = [
189 intel-media-driver
190 ];
191}
192```
193
194Older Intel GPUs use the i965 driver, which can be installed with:
195
196```nix
197{
198 hardware.opengl.extraPackages = [
199 intel-vaapi-driver
200 ];
201}
202```
203
204## Common issues {#sec-gpu-accel-common-issues}
205
206### User permissions {#sec-gpu-accel-common-issues-permissions}
207
208Except where noted explicitly, it should not be necessary to adjust user
209permissions to use these acceleration APIs. In the default
210configuration, GPU devices have world-read/write permissions
211(`/dev/dri/renderD*`) or are tagged as `uaccess` (`/dev/dri/card*`). The
212access control lists of devices with the `uaccess` tag will be updated
213automatically when a user logs in through `systemd-logind`. For example,
214if the user *alice* is logged in, the access control list should look as
215follows:
216
217```ShellSession
218$ getfacl /dev/dri/card0
219# file: dev/dri/card0
220# owner: root
221# group: video
222user::rw-
223user:alice:rw-
224group::rw-
225mask::rw-
226other::---
227```
228
229If you disabled (this functionality of) `systemd-logind`, you may need
230to add the user to the `video` group and log in again.
231
232### Mixing different versions of nixpkgs {#sec-gpu-accel-common-issues-mixing-nixpkgs}
233
234The *Installable Client Driver* (ICD) mechanism used by OpenCL and
235Vulkan loads runtimes into its address space using `dlopen`. Mixing an
236ICD loader mechanism and runtimes from different version of nixpkgs may
237not work. For example, if the ICD loader uses an older version of glibc
238than the runtime, the runtime may not be loadable due to missing
239symbols. Unfortunately, the loader will generally be quiet about such
240issues.
241
242If you suspect that you are running into library version mismatches
243between an ICL loader and a runtime, you could run an application with
244the `LD_DEBUG` variable set to get more diagnostic information. For
245example, OpenCL can be tested with `LD_DEBUG=files clinfo`, which should
246report missing symbols.