commit b0f64f56be60f27fe8e6db66e501b8be7501798c · pyrox.dev/nixpkgs

+117 -45

doc/languages-frameworks/cuda.section.md

···

       1
       1
        
       # CUDA {#cuda}

     

       2
       2
        
       

     

       3
       3
       -
       CUDA-only packages are stored in the `cudaPackages` packages set. This set

     

       4
       4
       -
       includes the `cudatoolkit`, portions of the toolkit in separate derivations,

     

       5
       5
       -
       `cudnn`, `cutensor` and `nccl`.

     

       3
       3
       +
       Compute Unified Device Architecture (CUDA) is a parallel computing platform and application programming interface (API) model created by NVIDIA. It's commonly used to accelerate computationally intensive problems and has been widely adopted for High Performance Computing (HPC) and Machine Learning (ML) applications.

     

       4
       4
       +
       

     

       5
       5
       +
       Packages provided by NVIDIA which require CUDA are typically stored in CUDA package sets.

     

       6
       6
       +
       

     

       7
       7
       +
       Nixpkgs provides a number of CUDA package sets, each based on a different CUDA release. Top-level attributes providing access to CUDA package sets follow these naming conventions:

     

       8
       8
       +
       

     

       9
       9
       +
       - `cudaPackages_x_y`: A major-minor-versioned package set for a specific CUDA release, where `x` and `y` are the major and minor versions of the CUDA release.

     

       10
       10
       +
       - `cudaPackages_x`: A major-versioned alias to the major-minor-versioned CUDA package set with the latest widely supported major CUDA release.

     

       11
       11
       +
       - `cudaPackages`: An alias to the major-versioned alias for the latest widely supported CUDA release. This package set referenced by this alias is also referred to as the "default" CUDA package set. As an example, which is an alias to the most recent variant of `cudaPackages_x`. This is also referred to as the "default" CUDA package set.

     

       12
       12
       +
       

     

       13
       13
       +
       Here are two examples to illustrate the naming conventions:

     

       14
       14
       +
       

     

       15
       15
       +
       - If `cudaPackages_12_8` is the latest release in the 12.x series, but core libraries like OpenCV or ONNX Runtime fail to build with it, `cudaPackages_12` may alias `cudaPackages_12_6` instead of `cudaPackages_12_8`.

     

       16
       16
       +
       - If `cudaPackages_13_1` is the latest release, but core libraries like PyTorch or Torch Vision fail to build with it, `cudaPackages` may alias `cudaPackages_12` instead of `cudaPackages_13`.

     

       17
       17
       +
       

     

       18
       18
       +
       All CUDA package sets include common CUDA packages like `libcublas`, `cudnn`, `tensorrt`, and `nccl`.

     

       19
       19
       +
       

     

       20
       20
       +
       ## Configuring Nixpkgs for CUDA {#cuda-configuring-nixpkgs-for-cuda}

     

       6
       21
        
       

     

       7
       7
       -
       A package set is available for each CUDA version, so for example

     

       8
       8
       -
       `cudaPackages_11_6`. Within each set is a matching version of the above listed

     

       9
       9
       -
       packages. Additionally, other versions of the packages that are packaged and

     

       10
       10
       -
       compatible are available as well. For example, there can be a

     

       11
       11
       -
       `cudaPackages.cudnn_8_3` package.

     

       22
       22
       +
       CUDA support is not enabled by default in Nixpkgs. To enable CUDA support, make sure Nixpkgs is imported with a configuration similar to the following:

     

       12
       23
        
       

     

       13
       13
       -
       To use one or more CUDA packages in an expression, give the expression a `cudaPackages` parameter, and in case CUDA is optional

     

       24
       24
       +
       ```nix

     

       25
       25
       +
       {

     

       26
       26
       +
         allowUnfreePredicate =

     

       27
       27
       +
           let

     

       28
       28
       +
             ensureList = x: if builtins.isList x then x else [ x ];

     

       29
       29
       +
           in

     

       30
       30
       +
           package:

     

       31
       31
       +
           builtins.all (

     

       32
       32
       +
             license:

     

       33
       33
       +
             license.free

     

       34
       34
       +
             || builtins.elem license.shortName [

     

       35
       35
       +
               "CUDA EULA"

     

       36
       36
       +
               "cuDNN EULA"

     

       37
       37
       +
               "cuSPARSELt EULA"

     

       38
       38
       +
               "cuTENSOR EULA"

     

       39
       39
       +
               "NVidia OptiX EULA"

     

       40
       40
       +
             ]

     

       41
       41
       +
           ) (ensureList package.meta.license);

     

       42
       42
       +
         cudaCapabilities = [ <target-architectures> ];

     

       43
       43
       +
         cudaForwardCompat = true;

     

       44
       44
       +
         cudaSupport = true;

     

       45
       45
       +
       }

     

       46
       46
       +
       ```

     

       47
       47
       +
       

     

       48
       48
       +
       The majority of CUDA packages are unfree, so either `allowUnfreePredicate` or `allowUnfree` should be set.

     

       49
       49
       +
       

     

       50
       50
       +
       The `cudaSupport` configuration option is used by packages to conditionally enable CUDA-specific functionality. This configuration option is commonly used by packages which can be built with or without CUDA support.

     

       51
       51
       +
       

     

       52
       52
       +
       The `cudaCapabilities` configuration option specifies a list of CUDA capabilities. Packages may use this option to control device code generation to take advantage of architecture-specific functionality, speed up compile times by producing less device code, or slim package closures. As an example, one can build for Ada Lovelace GPUs with `cudaCapabilities = [ "8.9" ];`. If `cudaCapabilities` is not provided, the default value is calculated per-package set, derived from a list of GPUs supported by that version of CUDA. Please consult [supported GPUs](https://en.wikipedia.org/wiki/CUDA#GPUs_supported) for specific cards. Library maintainers should consult [NVCC Docs](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/) and its release notes.

     

       53
       53
       +
       

     

       54
       54
       +
       ::: {.caution}

     

       55
       55
       +
       Certain CUDA capabilities are not targeted by default, including capabilities belonging to the Jetson family of devices (like `8.7`, which corresponds to the Jetson Orin) or non-baseline feature-sets (like `9.0a`, which corresponds to the Hopper exclusive feature set). If you need to target these capabilities, you must explicitly set `cudaCapabilities` to include them.

     

       56
       56
       +
       :::

     

       57
       57
       +
       

     

       58
       58
       +
       The `cudaForwardCompat` boolean configuration option determines whether PTX support for future hardware is enabled.

     

       59
       59
       +
       

     

       60
       60
       +
       ## Configuring CUDA package sets {#cuda-configuring-cuda-package-sets}

     

       61
       61
       +
       

     

       62
       62
       +
       CUDA package sets are created by `callPackage`-ing `pkgs/top-level/cuda-packages.nix` with an explicit argument for `cudaMajorMinorVersion`, a string of the form `"<major>.<minor>"` (e.g., `"12.2"`), which informs the CUDA package set tooling which version of CUDA to use. The majority of the CUDA package set tooling is available through the top-level attribute set `_cuda`, a fixed-point which exists apart from any instance of the CUDA package set.

     

       63
       63
       +
       

     

       64
       64
       +
       ::: {.caution}

     

       65
       65
       +
       The `cudaMajorMinorVersion` and `_cuda` attributes are not part of the CUDA package set fixed-point, but are instead provided by `callPackage` from the top-level in the construction of the package set. As such, they must be modified via the package set's `override` attribute.

     

       66
       66
       +
       :::

     

       67
       67
       +
       

     

       68
       68
       +
       ::: {.caution}

     

       69
       69
       +
       As indicated by the underscore prefix, `_cuda` is an implementation detail and no guarantees are provided with respect to its stability or API. The `_cuda` attribute set is exposed only to ease creation or modification of CUDA package sets by expert, out-of-tree users.

     

       70
       70
       +
       :::

     

       71
       71
       +
       

     

       72
       72
       +
       ::: {.note}

     

       73
       73
       +
       The `_cuda` attribute set fixed-point should be modified through its `extend` attribute.

     

       74
       74
       +
       :::

     

       75
       75
       +
       

     

       76
       76
       +
       The `_cuda.fixups` attribute set is a mapping from package name to a `callPackage`-able expression which will be provided to `overrideAttrs` on the result of our generic builder.

     

       77
       77
       +
       

     

       78
       78
       +
       ::: {.caution}

     

       79
       79
       +
       Fixups are chosen from `_cuda.fixups` by `pname`. As a result, packages with multiple versions (e.g., `cudnn`, `cudnn_8_9`, etc.) all share a single fixup function (i.e., `_cuda.fixups.cudnn`, which is `pkgs/development/cuda-modules/fixups/cudnn.nix`).

     

       80
       80
       +
       :::

     

       81
       81
       +
       

     

       82
       82
       +
       As an example, you can change the fixup function used for cuDNN for only the default CUDA package set with this overlay:

     

       83
       83
       +
       

     

       84
       84
       +
       ```nix

     

       85
       85
       +
       final: prev: {

     

       86
       86
       +
         cudaPackages = prev.cudaPackages.override (prevAttrs: {

     

       87
       87
       +
           _cuda = prevAttrs._cuda.extend (

     

       88
       88
       +
             _: prevAttrs': {

     

       89
       89
       +
               fixups = prevAttrs'.fixups // {

     

       90
       90
       +
                 cudnn = <your-fixup-function>;

     

       91
       91
       +
               };

     

       92
       92
       +
             }

     

       93
       93
       +
           );

     

       94
       94
       +
         });

     

       95
       95
       +
       }

     

       96
       96
       +
       ```

     

       97
       97
       +
       

     

       98
       98
       +
       ## Using cudaPackages {#cuda-using-cudapackages}

     

       99
       99
       +
       

     

       100
       100
       +
       ::: {.caution}

     

       101
       101
       +
       A non-trivial amount of CUDA package discoverability and usability relies on the various setup hooks used by a CUDA package set. As a result, users will likely encounter issues trying to perform builds within a `devShell` without manually invoking phases.

     

       102
       102
       +
       :::

     

       103
       103
       +
       

     

       104
       104
       +
       Nixpkgs makes CUDA package sets available under a number of attributes. While versioned package sets are available (e.g., `cudaPackages_12_2`), it is recommended to use the unversioned `cudaPackages` attribute, which is an alias to the latest version, as versioned attributes are periodically removed.

     

       105
       105
       +
       

     

       106
       106
       +
       To use one or more CUDA packages in an expression, give the expression a `cudaPackages` parameter, and in case CUDA support is optional, add a `config` and `cudaSupport` parameter:

     

       107
       107
       +
       

     

       14
       108
        
       ```nix

     

       15
       109
        
       {

     

       16
       110
        
         config,

     

       17
       111
        
         cudaSupport ? config.cudaSupport,

     

       18
       18
       -
         cudaPackages ? { },

     

       19
       19
       -
         ...

     

       112
       112
       +
         cudaPackages,

     

       20
       113
        
       }:

     

       21
       21
       -
       { }

     

       114
       114
       +
       <package-expression>

     

       22
       115
        
       ```

     

       23
       116
        
       

     

       24
       24
       -
       When using `callPackage`, you can choose to pass in a different variant, e.g.

     

       25
       25
       -
       when a different version of the toolkit suffices

     

       117
       117
       +
       In your package's derivation arguments, it is _strongly_ recommended the following are set:

     

       118
       118
       +
       

     

       26
       119
        
       ```nix

     

       27
       120
        
       {

     

       28
       28
       -
         mypkg = callPackage { cudaPackages = cudaPackages_11_5; };

     

       121
       121
       +
         __structuredAttrs = true;

     

       122
       122
       +
         strictDeps = true;

     

       29
       123
        
       }

     

       30
       124
        
       ```

     

       31
       125
        
       

     

       32
       32
       -
       If another version of say `cudnn` or `cutensor` is needed, you can override the

     

       33
       33
       -
       package set to make it the default. This guarantees you get a consistent package

     

       34
       34
       -
       set.

     

       126
       126
       +
       These settings ensure that the CUDA setup hooks function as intended.

     

       127
       127
       +
       

     

       128
       128
       +
       When using `callPackage`, you can choose to pass in a different variant, e.g. when a package requires a specific version of CUDA:

     

       129
       129
       +
       

     

       35
       130
        
       ```nix

     

       36
       131
        
       {

     

       37
       37
       -
         mypkg =

     

       38
       38
       -
           let

     

       39
       39
       -
             cudaPackages = cudaPackages_11_5.overrideScope (

     

       40
       40
       -
               final: prev: {

     

       41
       41
       -
                 cudnn = prev.cudnn_8_3;

     

       42
       42
       -
               }

     

       43
       43
       -
             );

     

       44
       44
       -
           in

     

       45
       45
       -
           callPackage { inherit cudaPackages; };

     

       132
       132
       +
         mypkg = callPackage { cudaPackages = cudaPackages_12_2; };

     

       46
       133
        
       }

     

       47
       134
        
       ```

     

       48
       135
        
       

     

       49
       49
       -
       The CUDA NVCC compiler requires flags to determine which hardware you

     

       50
       50
       -
       want to target for in terms of SASS (real hardware) or PTX (JIT kernels).

     

       51
       51
       -
       

     

       52
       52
       -
       Nixpkgs tries to target support real architecture defaults based on the

     

       53
       53
       -
       CUDA toolkit version with PTX support for future hardware.  Experienced

     

       54
       54
       -
       users may optimize this configuration for a variety of reasons such as

     

       55
       55
       -
       reducing binary size and compile time, supporting legacy hardware, or

     

       56
       56
       -
       optimizing for specific hardware.

     

       57
       57
       -
       

     

       58
       58
       -
       You may provide capabilities to add support or reduce binary size through

     

       59
       59
       -
       `config` using `cudaCapabilities = [ "6.0" "7.0" ];` and

     

       60
       60
       -
       `cudaForwardCompat = true;` if you want PTX support for future hardware.

     

       61
       61
       -
       

     

       62
       62
       -
       Please consult [GPUs supported](https://en.wikipedia.org/wiki/CUDA#GPUs_supported)

     

       63
       63
       -
       for your specific card(s).

     

       64
       64
       -
       

     

       65
       65
       -
       Library maintainers should consult [NVCC Docs](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/)

     

       66
       66
       -
       and release notes for their software package.

     

       136
       136
       +
       ::: {.caution}

     

       137
       137
       +
       Overriding the CUDA package set used by a package may cause inconsistencies, since the override does not affect dependencies of the package. As a result, it is easy to end up with a package which uses a different CUDA package set than its dependencies. If at all possible, it is recommended to change the default CUDA package set globally, to ensure a consistent environment.

     

       138
       138
       +
       :::

     

       67
       139
        
       

     

       68
       140
        
       ## Running Docker or Podman containers with CUDA support {#cuda-docker-podman}

     

       69
       141

+59 -50

doc/redirects.json

···

       20
       20
        
         "cmake-ctest-variables": [

     

       21
       21
        
           "index.html#cmake-ctest-variables"

     

       22
       22
        
         ],

     

       23
       23
       +
         "cuda": [

     

       24
       24
       +
           "index.html#cuda"

     

       25
       25
       +
         ],

     

       26
       26
       +
         "cuda-configuring-cuda-package-sets": [

     

       27
       27
       +
           "index.html#cuda-configuring-cuda-package-sets"

     

       28
       28
       +
         ],

     

       29
       29
       +
         "cuda-configuring-nixpkgs-for-cuda": [

     

       30
       30
       +
           "index.html#cuda-configuring-nixpkgs-for-cuda"

     

       31
       31
       +
         ],

     

       32
       32
       +
         "cuda-contributing": [

     

       33
       33
       +
           "index.html#cuda-contributing"

     

       34
       34
       +
         ],

     

       35
       35
       +
         "cuda-docker-podman": [

     

       36
       36
       +
           "index.html#cuda-docker-podman"

     

       37
       37
       +
         ],

     

       38
       38
       +
         "cuda-package-set-maintenance": [

     

       39
       39
       +
           "index.html#cuda-package-set-maintenance",

     

       40
       40
       +
           "index.html#adding-a-new-cuda-release"

     

       41
       41
       +
         ],

     

       42
       42
       +
         "cuda-passthru-testers": [

     

       43
       43
       +
           "index.html#cuda-passthru-testers"

     

       44
       44
       +
         ],

     

       45
       45
       +
         "cuda-passthru-tests": [

     

       46
       46
       +
           "index.html#cuda-passthru-tests"

     

       47
       47
       +
         ],

     

       48
       48
       +
         "cuda-specifying-what-devices-to-expose-to-the-container": [

     

       49
       49
       +
           "index.html#cuda-specifying-what-devices-to-expose-to-the-container",

     

       50
       50
       +
           "index.html#specifying-what-devices-to-expose-to-the-container"

     

       51
       51
       +
         ],

     

       52
       52
       +
         "cuda-updating-redistributables": [

     

       53
       53
       +
           "index.html#cuda-updating-redistributables",

     

       54
       54
       +
           "index.html#updating-cuda-redistributables"

     

       55
       55
       +
         ],

     

       56
       56
       +
         "cuda-updating-cutensor": [

     

       57
       57
       +
           "index.html#cuda-updating-cutensor",

     

       58
       58
       +
           "index.html#updating-cutensor"

     

       59
       59
       +
         ],

     

       60
       60
       +
         "cuda-updating-supported-compilers-and-gpus": [

     

       61
       61
       +
           "index.html#cuda-updating-supported-compilers-and-gpus",

     

       62
       62
       +
           "index.html#updating-supported-compilers-and-gpus"

     

       63
       63
       +
         ],

     

       64
       64
       +
         "cuda-updating-the-cuda-package-set": [

     

       65
       65
       +
           "index.html#cuda-updating-the-cuda-package-set",

     

       66
       66
       +
           "index.html#updating-the-cuda-package-set"

     

       67
       67
       +
         ],

     

       68
       68
       +
         "cuda-updating-the-cuda-toolkit": [

     

       69
       69
       +
           "index.html#cuda-updating-the-cuda-toolkit",

     

       70
       70
       +
           "index.html#updating-the-cuda-toolkit"

     

       71
       71
       +
         ],

     

       72
       72
       +
         "cuda-using-cudapackages": [

     

       73
       73
       +
           "index.html#cuda-using-cudapackages"

     

       74
       74
       +
         ],

     

       75
       75
       +
         "cuda-using-docker-compose": [

     

       76
       76
       +
           "index.html#cuda-using-docker-compose",

     

       77
       77
       +
           "index.html#using-docker-compose"

     

       78
       78
       +
         ],

     

       79
       79
       +
         "cuda-writing-tests": [

     

       80
       80
       +
           "index.html#cuda-writing-tests"

     

       81
       81
       +
         ],

     

       23
       82
        
         "ex-build-helpers-extendMkDerivation": [

     

       24
       83
        
           "index.html#ex-build-helpers-extendMkDerivation"

     

       25
       84
        
         ],

     
···

       2752
       2811
        
         ],

     

       2753
       2812
        
         "building-a-crystal-package": [

     

       2754
       2813
        
           "index.html#building-a-crystal-package"

     

       2755
       2755
       -
         ],

     

       2756
       2756
       -
         "cuda": [

     

       2757
       2757
       -
           "index.html#cuda"

     

       2758
       2758
       -
         ],

     

       2759
       2759
       -
         "cuda-contributing": [

     

       2760
       2760
       -
           "index.html#cuda-contributing"

     

       2761
       2761
       -
         ],

     

       2762
       2762
       -
         "cuda-docker-podman": [

     

       2763
       2763
       -
           "index.html#cuda-docker-podman"

     

       2764
       2764
       -
         ],

     

       2765
       2765
       -
         "cuda-package-set-maintenance": [

     

       2766
       2766
       -
           "index.html#cuda-package-set-maintenance",

     

       2767
       2767
       -
           "index.html#adding-a-new-cuda-release"

     

       2768
       2768
       -
         ],

     

       2769
       2769
       -
         "cuda-passthru-testers": [

     

       2770
       2770
       -
           "index.html#cuda-passthru-testers"

     

       2771
       2771
       -
         ],

     

       2772
       2772
       -
         "cuda-passthru-tests": [

     

       2773
       2773
       -
           "index.html#cuda-passthru-tests"

     

       2774
       2774
       -
         ],

     

       2775
       2775
       -
         "cuda-specifying-what-devices-to-expose-to-the-container": [

     

       2776
       2776
       -
           "index.html#cuda-specifying-what-devices-to-expose-to-the-container",

     

       2777
       2777
       -
           "index.html#specifying-what-devices-to-expose-to-the-container"

     

       2778
       2778
       -
         ],

     

       2779
       2779
       -
         "cuda-updating-redistributables": [

     

       2780
       2780
       -
           "index.html#cuda-updating-redistributables",

     

       2781
       2781
       -
           "index.html#updating-cuda-redistributables"

     

       2782
       2782
       -
         ],

     

       2783
       2783
       -
         "cuda-updating-cutensor": [

     

       2784
       2784
       -
           "index.html#cuda-updating-cutensor",

     

       2785
       2785
       -
           "index.html#updating-cutensor"

     

       2786
       2786
       -
         ],

     

       2787
       2787
       -
         "cuda-updating-supported-compilers-and-gpus": [

     

       2788
       2788
       -
           "index.html#cuda-updating-supported-compilers-and-gpus",

     

       2789
       2789
       -
           "index.html#updating-supported-compilers-and-gpus"

     

       2790
       2790
       -
         ],

     

       2791
       2791
       -
         "cuda-updating-the-cuda-package-set": [

     

       2792
       2792
       -
           "index.html#cuda-updating-the-cuda-package-set",

     

       2793
       2793
       -
           "index.html#updating-the-cuda-package-set"

     

       2794
       2794
       -
         ],

     

       2795
       2795
       -
         "cuda-updating-the-cuda-toolkit": [

     

       2796
       2796
       -
           "index.html#cuda-updating-the-cuda-toolkit",

     

       2797
       2797
       -
           "index.html#updating-the-cuda-toolkit"

     

       2798
       2798
       -
         ],

     

       2799
       2799
       -
         "cuda-using-docker-compose": [

     

       2800
       2800
       -
           "index.html#cuda-using-docker-compose",

     

       2801
       2801
       -
           "index.html#using-docker-compose"

     

       2802
       2802
       -
         ],

     

       2803
       2803
       -
         "cuda-writing-tests": [

     

       2804
       2804
       -
           "index.html#cuda-writing-tests"

     

       2805
       2814
        
         ],

     

       2806
       2815
        
         "cuelang": [

     

       2807
       2816
        
           "index.html#cuelang"