pyrox.dev
1<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-gpu-accel">
2 <title>GPU acceleration</title>
3 <para>
4 NixOS provides various APIs that benefit from GPU hardware
5 acceleration, such as VA-API and VDPAU for video playback; OpenGL
6 and Vulkan for 3D graphics; and OpenCL for general-purpose
7 computing. This chapter describes how to set up GPU hardware
8 acceleration (as far as this is not done automatically) and how to
9 verify that hardware acceleration is indeed used.
10 </para>
11 <para>
12 Most of the aforementioned APIs are agnostic with regards to which
13 display server is used. Consequently, these instructions should
14 apply both to the X Window System and Wayland compositors.
15 </para>
16 <section xml:id="sec-gpu-accel-opencl">
17 <title>OpenCL</title>
18 <para>
19 <link xlink:href="https://en.wikipedia.org/wiki/OpenCL">OpenCL</link>
20 is a general compute API. It is used by various applications such
21 as Blender and Darktable to accelerate certain operations.
22 </para>
23 <para>
24 OpenCL applications load drivers through the <emphasis>Installable
25 Client Driver</emphasis> (ICD) mechanism. In this mechanism, an
26 ICD file specifies the path to the OpenCL driver for a particular
27 GPU family. In NixOS, there are two ways to make ICD files visible
28 to the ICD loader. The first is through the
29 <literal>OCL_ICD_VENDORS</literal> environment variable. This
30 variable can contain a directory which is scanned by the ICL
31 loader for ICD files. For example:
32 </para>
33 <programlisting>
34$ export \
35 OCL_ICD_VENDORS=`nix-build '<nixpkgs>' --no-out-link -A rocm-opencl-icd`/etc/OpenCL/vendors/
36</programlisting>
37 <para>
38 The second mechanism is to add the OpenCL driver package to
39 <xref linkend="opt-hardware.opengl.extraPackages" />. This links
40 the ICD file under <literal>/run/opengl-driver</literal>, where it
41 will be visible to the ICD loader.
42 </para>
43 <para>
44 The proper installation of OpenCL drivers can be verified through
45 the <literal>clinfo</literal> command of the clinfo package. This
46 command will report the number of hardware devices that is found
47 and give detailed information for each device:
48 </para>
49 <programlisting>
50$ clinfo | head -n3
51Number of platforms 1
52Platform Name AMD Accelerated Parallel Processing
53Platform Vendor Advanced Micro Devices, Inc.
54</programlisting>
55 <section xml:id="sec-gpu-accel-opencl-amd">
56 <title>AMD</title>
57 <para>
58 Modern AMD
59 <link xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
60 Core Next</link> (GCN) GPUs are supported through the
61 rocm-opencl-icd package. Adding this package to
62 <xref linkend="opt-hardware.opengl.extraPackages" /> enables
63 OpenCL support:
64 </para>
65 <programlisting language="bash">
66hardware.opengl.extraPackages = [
67 rocm-opencl-icd
68];
69</programlisting>
70 </section>
71 <section xml:id="sec-gpu-accel-opencl-intel">
72 <title>Intel</title>
73 <para>
74 <link xlink:href="https://en.wikipedia.org/wiki/List_of_Intel_graphics_processing_units#Gen8">Intel
75 Gen8 and later GPUs</link> are supported by the Intel NEO OpenCL
76 runtime that is provided by the intel-compute-runtime package.
77 For Gen7 GPUs, the deprecated Beignet runtime can be used, which
78 is provided by the beignet package. The proprietary Intel OpenCL
79 runtime, in the intel-ocl package, is an alternative for Gen7
80 GPUs.
81 </para>
82 <para>
83 The intel-compute-runtime, beignet, or intel-ocl package can be
84 added to <xref linkend="opt-hardware.opengl.extraPackages" /> to
85 enable OpenCL support. For example, for Gen8 and later GPUs, the
86 following configuration can be used:
87 </para>
88 <programlisting language="bash">
89hardware.opengl.extraPackages = [
90 intel-compute-runtime
91];
92</programlisting>
93 </section>
94 </section>
95 <section xml:id="sec-gpu-accel-vulkan">
96 <title>Vulkan</title>
97 <para>
98 <link xlink:href="https://en.wikipedia.org/wiki/Vulkan_(API)">Vulkan</link>
99 is a graphics and compute API for GPUs. It is used directly by
100 games or indirectly though compatibility layers like
101 <link xlink:href="https://github.com/doitsujin/dxvk/wiki">DXVK</link>.
102 </para>
103 <para>
104 By default, if <xref linkend="opt-hardware.opengl.driSupport" />
105 is enabled, mesa is installed and provides Vulkan for supported
106 hardware.
107 </para>
108 <para>
109 Similar to OpenCL, Vulkan drivers are loaded through the
110 <emphasis>Installable Client Driver</emphasis> (ICD) mechanism.
111 ICD files for Vulkan are JSON files that specify the path to the
112 driver library and the supported Vulkan version. All successfully
113 loaded drivers are exposed to the application as different GPUs.
114 In NixOS, there are two ways to make ICD files visible to Vulkan
115 applications: an environment variable and a module option.
116 </para>
117 <para>
118 The first option is through the
119 <literal>VK_ICD_FILENAMES</literal> environment variable. This
120 variable can contain multiple JSON files, separated by
121 <literal>:</literal>. For example:
122 </para>
123 <programlisting>
124$ export \
125 VK_ICD_FILENAMES=`nix-build '<nixpkgs>' --no-out-link -A amdvlk`/share/vulkan/icd.d/amd_icd64.json
126</programlisting>
127 <para>
128 The second mechanism is to add the Vulkan driver package to
129 <xref linkend="opt-hardware.opengl.extraPackages" />. This links
130 the ICD file under <literal>/run/opengl-driver</literal>, where it
131 will be visible to the ICD loader.
132 </para>
133 <para>
134 The proper installation of Vulkan drivers can be verified through
135 the <literal>vulkaninfo</literal> command of the vulkan-tools
136 package. This command will report the hardware devices and drivers
137 found, in this example output amdvlk and radv:
138 </para>
139 <programlisting>
140$ vulkaninfo | grep GPU
141 GPU id : 0 (Unknown AMD GPU)
142 GPU id : 1 (AMD RADV NAVI10 (LLVM 9.0.1))
143 ...
144GPU0:
145 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
146 deviceName = Unknown AMD GPU
147GPU1:
148 deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
149</programlisting>
150 <para>
151 A simple graphical application that uses Vulkan is
152 <literal>vkcube</literal> from the vulkan-tools package.
153 </para>
154 <section xml:id="sec-gpu-accel-vulkan-amd">
155 <title>AMD</title>
156 <para>
157 Modern AMD
158 <link xlink:href="https://en.wikipedia.org/wiki/Graphics_Core_Next">Graphics
159 Core Next</link> (GCN) GPUs are supported through either radv,
160 which is part of mesa, or the amdvlk package. Adding the amdvlk
161 package to <xref linkend="opt-hardware.opengl.extraPackages" />
162 makes amdvlk the default driver and hides radv and lavapipe from
163 the device list. A specific driver can be forced as follows:
164 </para>
165 <programlisting language="bash">
166hardware.opengl.extraPackages = [
167 pkgs.amdvlk
168];
169
170# To enable Vulkan support for 32-bit applications, also add:
171hardware.opengl.extraPackages32 = [
172 pkgs.driversi686Linux.amdvlk
173];
174
175# Force radv
176environment.variables.AMD_VULKAN_ICD = "RADV";
177# Or
178environment.variables.VK_ICD_FILENAMES =
179 "/run/opengl-driver/share/vulkan/icd.d/radeon_icd.x86_64.json";
180</programlisting>
181 </section>
182 </section>
183 <section xml:id="sec-gpu-accel-common-issues">
184 <title>Common issues</title>
185 <section xml:id="sec-gpu-accel-common-issues-permissions">
186 <title>User permissions</title>
187 <para>
188 Except where noted explicitly, it should not be necessary to
189 adjust user permissions to use these acceleration APIs. In the
190 default configuration, GPU devices have world-read/write
191 permissions (<literal>/dev/dri/renderD*</literal>) or are tagged
192 as <literal>uaccess</literal>
193 (<literal>/dev/dri/card*</literal>). The access control lists of
194 devices with the <literal>uaccess</literal> tag will be updated
195 automatically when a user logs in through
196 <literal>systemd-logind</literal>. For example, if the user
197 <emphasis>alice</emphasis> is logged in, the access control list
198 should look as follows:
199 </para>
200 <programlisting>
201$ getfacl /dev/dri/card0
202# file: dev/dri/card0
203# owner: root
204# group: video
205user::rw-
206user:alice:rw-
207group::rw-
208mask::rw-
209other::---
210</programlisting>
211 <para>
212 If you disabled (this functionality of)
213 <literal>systemd-logind</literal>, you may need to add the user
214 to the <literal>video</literal> group and log in again.
215 </para>
216 </section>
217 <section xml:id="sec-gpu-accel-common-issues-mixing-nixpkgs">
218 <title>Mixing different versions of nixpkgs</title>
219 <para>
220 The <emphasis>Installable Client Driver</emphasis> (ICD)
221 mechanism used by OpenCL and Vulkan loads runtimes into its
222 address space using <literal>dlopen</literal>. Mixing an ICD
223 loader mechanism and runtimes from different version of nixpkgs
224 may not work. For example, if the ICD loader uses an older
225 version of glibc than the runtime, the runtime may not be
226 loadable due to missing symbols. Unfortunately, the loader will
227 generally be quiet about such issues.
228 </para>
229 <para>
230 If you suspect that you are running into library version
231 mismatches between an ICL loader and a runtime, you could run an
232 application with the <literal>LD_DEBUG</literal> variable set to
233 get more diagnostic information. For example, OpenCL can be
234 tested with <literal>LD_DEBUG=files clinfo</literal>, which
235 should report missing symbols.
236 </para>
237 </section>
238 </section>
239</chapter>