Compile CUDA with SCALE#
This guide covers the steps required to compile an existing CUDA project for an AMD GPU using SCALE.
SCALE makes this as easy as possible by convincingly impersonating the NVIDIA CUDA Toolkit (from the point of view of your build system).
To use SCALE, we must simply cause your build system to use the "CUDA installation" offered by SCALE.
Install SCALE, if you haven't already.
Identifying GPU Target#
If you don't already know which AMD GPU target you need to compile for, you can
use the scaleinfo
command provided by SCALE to find out:
scaleinfo | grep gfx
Example output:
Device 0 (00:23:00.0): AMD Radeon Pro W6800 - gfx1030 (AMD) <amdgcn-amd-amdhsa--gfx1030>
In this example, the GPU target ID is gfx1030
.
If your GPU is not listed in the output of this command, it is not currently supported by SCALE.
If the scaleinfo
command is not found, ensure
that <SCALE install path>/bin
is in PATH
.
The easy way: scaleenv
#
SCALE offers a "venv
-flavoured" environment management script to allow
"magically" building CUDA projects.
The concept is simple:
- Activate the
scaleenv
for the AMD GPU target you want to build for. - Run the commands you normally use to build the project for an NVIDIA GPU.
- AMD binaries are sneakily produced instead of NVIDIA ones.
To activate a scaleenv:
source /opt/scale/bin/scaleenv gfx1030
You can exit a scaleenv
by typing deactivate
or closing your terminal.
While the environment is active: simply run the usual cmake
/make
/etc.
commands needed to build the project, and it will build for whatever AMD
target you handed to scaleenv
.
How it really works#
To allow compilation without build system changes, SCALE provides a series of
directories that are recognised by build systems as being CUDA Toolkit
installations. One such directory is provided for each supported AMD GPU
target. These directories can be found at <SCALE install
path>/targets/gfxXXXX
, where gfxXXXX
is the name of an AMD GPU target,
such as gfx1030
.
To achieve the desired effect, we need the build system to use the "CUDA toolkit" corresponding to the desired AMD GPU target.
For example: to build for gfx1030
you would tell your build system that
CUDA is installed at <SCALE install path>/targets/gfx1030
.
All scaleenv
is actually doing is setting various environment variables up
to make this happen. It's just a shell script: open it to see the variables
it is manipulating.
Finding the libraries at runtime#
For maximum compatibility with projects that depend on NVIDIA's "compute capability" numbering scheme, SCALE provides one "cuda mimic directory" per supported GPU target that maps the new target to "sm_86" in NVIDIA's numbering scheme.
This means that each of the target
subdirectories contains
identically-named libraries, so SCALE cannot meaningfully add them to the
system's library search path when it is installed. The built executable/library
therefore needs to be told how to find the libraries via another mechanism,
such as:
- rpath. With CMake, the simplest
thing that "usually just works" is to add
-DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON
to your cmake incantation. - Set
LD_LIBRARY_PATH
to include${SCALE_DIR}/lib
at runtime.scaleenv
does this, so if you keep that enabled when running your programs things will just work.
Support for multiple GPU architectures in a single binary ("Fat binaries") is in development.
Next steps#
- Learn about CUDA dialects and SCALE language extensions
- Report a bug