pyrox.dev
1# GPU acceleration {#sec-gpu-accel}
2
3NixOS provides various APIs that benefit from GPU hardware acceleration,
4such as VA-API and VDPAU for video playback; OpenGL and Vulkan for 3D
5graphics; and OpenCL for general-purpose computing. This chapter
6describes how to set up GPU hardware acceleration (as far as this is not
7done automatically) and how to verify that hardware acceleration is
8indeed used.
9
10Most of the aforementioned APIs are agnostic with regards to which
11display server is used. Consequently, these instructions should apply
12both to the X Window System and Wayland compositors.
13
14## OpenCL {#sec-gpu-accel-opencl}
15
16[OpenCL](https://en.wikipedia.org/wiki/OpenCL) is a general compute API.
17It is used by various applications such as Blender and Darktable to
18accelerate certain operations.
19
20OpenCL applications load drivers through the *Installable Client Driver*
21(ICD) mechanism. In this mechanism, an ICD file specifies the path to
22the OpenCL driver for a particular GPU family. In NixOS, there are two
23ways to make ICD files visible to the ICD loader. The first is through
24the `OCL_ICD_VENDORS` environment variable. This variable can contain a
25directory which is scanned by the ICL loader for ICD files. For example:
26
27```ShellSession
28$ export \
29 OCL_ICD_VENDORS=`nix-build '<nixpkgs>' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/
30```
31
32The second mechanism is to add the OpenCL driver package to
33[](#opt-hardware.opengl.extraPackages).
34This links the ICD file under `/run/opengl-driver`, where it will be visible
35to the ICD loader.
36
37The proper installation of OpenCL drivers can be verified through the
38`clinfo` command of the clinfo package. This command will report the
39number of hardware devices that is found and give detailed information
40for each device:
41
42```ShellSession
43$ clinfo | head -n3
44Number of platforms 1
45Platform Name AMD Accelerated Parallel Processing
46Platform Vendor Advanced Micro Devices, Inc.
47```
48
49### AMD {#sec-gpu-accel-opencl-amd}
50
51Modern AMD [Graphics Core
52Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
53supported through the rocm-opencl-icd package. Adding this package to
54[](#opt-hardware.opengl.extraPackages)
55enables OpenCL support:
56
57```nix
58hardware.opengl.extraPackages = [
59 rocm-opencl-icd
60];
61```
62
63### Intel {#sec-gpu-accel-opencl-intel}
64
65[Intel Gen8 and later
66GPUs](https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8)
67are supported by the Intel NEO OpenCL runtime that is provided by the
68intel-compute-runtime package. For Gen7 GPUs, the deprecated Beignet
69runtime can be used, which is provided by the beignet package. The
70proprietary Intel OpenCL runtime, in the intel-ocl package, is an
71alternative for Gen7 GPUs.
72
73The intel-compute-runtime, beignet, or intel-ocl package can be added to
74[](#opt-hardware.opengl.extraPackages)
75to enable OpenCL support. For example, for Gen8 and later GPUs, the following
76configuration can be used:
77
78```nix
79hardware.opengl.extraPackages = [
80 intel-compute-runtime
81];
82```
83
84## Vulkan {#sec-gpu-accel-vulkan}
85
86[Vulkan](https://en.wikipedia.org/wiki/Vulkan_(API)) is a graphics and
87compute API for GPUs. It is used directly by games or indirectly though
88compatibility layers like
89[DXVK](https://github.com/doitsujin/dxvk/wiki).
90
91By default, if [](#opt-hardware.opengl.driSupport)
92is enabled, mesa is installed and provides Vulkan for supported hardware.
93
94Similar to OpenCL, Vulkan drivers are loaded through the *Installable
95Client Driver* (ICD) mechanism. ICD files for Vulkan are JSON files that
96specify the path to the driver library and the supported Vulkan version.
97All successfully loaded drivers are exposed to the application as
98different GPUs. In NixOS, there are two ways to make ICD files visible
99to Vulkan applications: an environment variable and a module option.
100
101The first option is through the `VK_ICD_FILENAMES` environment variable.
102This variable can contain multiple JSON files, separated by `:`. For
103example:
104
105```ShellSession
106$ export \
107 VK_ICD_FILENAMES=`nix-build '<nixpkgs>' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json
108```
109
110The second mechanism is to add the Vulkan driver package to
111[](#opt-hardware.opengl.extraPackages).
112This links the ICD file under `/run/opengl-driver`, where it will be
113visible to the ICD loader.
114
115The proper installation of Vulkan drivers can be verified through the
116`vulkaninfo` command of the vulkan-tools package. This command will
117report the hardware devices and drivers found, in this example output
118amdvlk and radv:
119
120```ShellSession
121$ vulkaninfo | grep GPU
122 GPU id : 0 (Unknown AMD GPU)
123 GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
124 ...
125GPU0:
126 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
127 deviceName = Unknown AMD GPU
128GPU1:
129 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
130```
131
132A simple graphical application that uses Vulkan is `vkcube` from the
133vulkan-tools package.
134
135### AMD {#sec-gpu-accel-vulkan-amd}
136
137Modern AMD [Graphics Core
138Next](https://en.wikipedia.org/wiki/Graphics_Core_Next) (GCN) GPUs are
139supported through either radv, which is part of mesa, or the amdvlk
140package. Adding the amdvlk package to
141[](#opt-hardware.opengl.extraPackages)
142makes amdvlk the default driver and hides radv and lavapipe from the device list.
143A specific driver can be forced as follows:
144
145```nix
146hardware.opengl.extraPackages = [
147 pkgs.amdvlk
148];
149
150# To enable Vulkan support for 32-bit applications, also add:
151hardware.opengl.extraPackages32 = [
152 pkgs.driversi686Linux.amdvlk
153];
154
155# Force radv
156environment.variables.AMD_VULKAN_ICD = "RADV";
157# Or
158environment.variables.VK_ICD_FILENAMES =
159 "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
160```
161
162## Common issues {#sec-gpu-accel-common-issues}
163
164### User permissions {#sec-gpu-accel-common-issues-permissions}
165
166Except where noted explicitly, it should not be necessary to adjust user
167permissions to use these acceleration APIs. In the default
168configuration, GPU devices have world-read/write permissions
169(`/dev/dri/renderD*`) or are tagged as `uaccess` (`/dev/dri/card*`). The
170access control lists of devices with the `uaccess` tag will be updated
171automatically when a user logs in through `systemd-logind`. For example,
172if the user *jane* is logged in, the access control list should look as
173follows:
174
175```ShellSession
176$ getfacl /dev/dri/card0
177# file: dev/dri/card0
178# owner: root
179# group: video
180user::rw-
181user:jane:rw-
182group::rw-
183mask::rw-
184other::---
185```
186
187If you disabled (this functionality of) `systemd-logind`, you may need
188to add the user to the `video` group and log in again.
189
190### Mixing different versions of nixpkgs {#sec-gpu-accel-common-issues-mixing-nixpkgs}
191
192The *Installable Client Driver* (ICD) mechanism used by OpenCL and
193Vulkan loads runtimes into its address space using `dlopen`. Mixing an
194ICD loader mechanism and runtimes from different version of nixpkgs may
195not work. For example, if the ICD loader uses an older version of glibc
196than the runtime, the runtime may not be loadable due to missing
197symbols. Unfortunately, the loader will generally be quiet about such
198issues.
199
200If you suspect that you are running into library version mismatches
201between an ICL loader and a runtime, you could run an application with
202the `LD_DEBUG` variable set to get more diagnostic information. For
203example, OpenCL can be tested with `LD_DEBUG=files clinfo`, which should
204report missing symbols.