.. ## .. ## Copyright (c) Lawrence Livermore National Security, LLC and other .. ## RAJA Project Developers. See top-level LICENSE and COPYRIGHT .. ## files for dates and other details. No copyright assignment is required .. ## to contribute to RAJA. .. ## .. ## SPDX-License-Identifier: (BSD-3-Clause) .. ## .. _build_config-label: ************************** RAJA Build Configurations ************************** RAJA is tested with a wide range of compilers in a variety of build configurations. Automated continuous integration (CI) testing employed by the project is described in :ref:`ci-label`. Building and testing manually is also important for day-to-day development. We maintain two ways to build and test configurations in a reproducible manner: * **Build scripts.** The RAJA source repository contains a collection of simple shell scripts that we use to generate build configurations for a variety of platforms, such as Livermore Computing (LC) systems, MacOS, and Linux environments. * **Generated host-config files.** The RAJA repository includes a mechanism to generate *host-config* files (i.e., CMake cache files) that are identical to those generated by `Spack `_ for use in our GitLab CI testing. The configurations specify compiler versions and options, build targets (Release, Debug, etc.), RAJA features to enable (OpenMP, CUDA, HIP, etc.), and paths to required tool chains, such as CUDA, ROCm, etc. We describe them briefly in the following sections. .. _build_scripts-label: =================== RAJA Build Scripts =================== The build scripts mentioned above reside in subdirectories of the `RAJA/scripts `_ directory. These include: * ``RAJA/scripts/lc-builds`` for configurations on LC machines * ``RAJA/scripts/macos-builds`` for MacOS configurations * ``RAJA/scripts/ubuntu-builds`` for other Linux configurations To create a RAJA build space using one of the scripts, execute the script from the top-level RAJA directory, passing required arguments such as compiler version, target compute architecture for GPU builds, etc. For example, .. code-block:: bash $ ./scripts/lc-builds/toss3_clang.sh 10.0.1 will create a build space for the clang 10.0.1 compiler on a TOSS3 LC machine. .. note:: Each RAJA build script for an LC machine does the following: * Create a build space directory named for the specified build configuration in the top-level RAJA directory * Load a CMake module version appropriate for the configuration * Invoke CMake with appropriate arguments to create the specified build configuration. These arguments typically contain full paths to the compiler and any toolchains needed, and a *host-config* file that contains compiler flags. Host-config files, also known and *CMake cache files* that are passed to CMake are maintained in subdirectories of the `RAJA/host-configs `_ directory. After running a build script, one then goes into that directory and runs 'make' to build RAJA and its tests, example codes, etc. For example, .. code-block:: bash $ ./scripts/lc-builds/toss4_amdclang.sh 6.4.1 gfx942 $ cd build_lc_toss4-amdclang-6.4.1-gfx942 $ make -j $ make test .. _spack_host_config-label: ================================== Spack-Generated Host-Config Files ================================== As described in :ref:`gitlab_ci_depend-label`, the RAJA repository contains two submodules `Uberenv `_ and `RADIUSS Spack Configs `_ that work together with Spack to generate host-config files. Uberenv and RADIUSS Spack Configs are projects in the GitHub LLNL organization and contain utilities shared and maintained by various LLNL open-source projects. Uberenv contains a Python script that is used to drive Spack to generate a *host-config* file. The generated file can be passed to CMake using the '-C' option to create a build configuration. *Spack specs* defining compiler and toolchain configurations, and Spack packages for RAJA and other projects are maintained in the RADIUSS Spack Configs project. Additional information about this process is available in the `RADIUSS Uberenv Guide `_. .. important:: The process of generating a host-config file and using it to test a build of RAJA is very useful since the same procedure is done in our GitLab CI testing. See :ref:`ci-label` for more information about our GitLab testing process. Generating a RAJA host-config file ------------------------------------ This section describes the host-config file generation process for RAJA. Platform configurations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Compiler configurations for Livermore machines are contained in sub-directories of the ``RAJA/scripts/radiuss-spack-configs`` submodule directory: .. code-block:: bash $ ls -c1 ./scripts/radiuss-spack-configs toss_4_x86_64_ib_cray toss_4_x86_64_ib toss_3_x86_64_ib packages modules.yaml darwin blueos_3_ppc64le_ib config.yaml blueos_3_ppc64le_ib_p9 ... Each subdirectory named for a system type contains a ``compilers.yaml`` file and ``packages.yaml`` file with compiler and toolchain information associated with the system. Generating a host-config file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To generate a host-config file for a desired configuration, run the ``uberenv.py`` python script from the top-level RAJA directory. .. important:: **DO NOT** run the ``uberenv.py`` script, which invokes Spack, in your LC home directory. **Running Spack in your home directory may cause you to exceed your LC disk space quota.** For example, .. code-block:: bash $ python3 ./scripts/uberenv/uberenv.py --spec="%gcc@8.1.0" will generate a basic configuration for building RAJA with the GNU 8.1.0 compiler, while .. code-block:: bash $ python3 ./scripts/uberenv/uberenv.py --spec="+openmp +tests +desul %clang@10.0.1 cxxflags==--gcc-toolchain=/usr/tce/packages/gcc/gcc-8.3.1 cflags==--gcc-toolchain=/usr/tce/packages/gcc/gcc-8.3.1" will generate a configuration for building with the clang 10.0.1 compiler using the GNU 8.3.1 toolchain (for C++ standard library headers, etc.) and with RAJA OpenMP back-end, tests, and desul atomics enabled. Running a command like one of these will generate a host-config file in the top-level RAJA directory. The name of the file will contain the name of the machine and OS to which it applies, as well as the compiler and version. For example, running the first example command above on the LC machine quartz will generate a host-config file named something like this: .. code-block:: bash hc-quartz-toss_3_x86_64_ib-gcc@8.1.0-fjcjwd6ec3uen5rh6msdqujydsj74ubf.cmake Building RAJA with a generated host-config file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To build RAJA with a generated host-config file, create a build directory and run CMake in it by passing a host-config file to CMake using the '-C' option. Then, run 'make' to build RAJA. To ensure the build was successful, you may want to run the RAJA tests. For example, .. code-block:: bash $ pwd path-to-raja/RAJA $ mkdir && cd $ cmake -C /.cmake .. $ make -j $ ctest --output-on-failure -T test You may also run the RAJA tests with the command .. code-block:: bash $ make test as an alternative to the 'ctest' command used above. It is also possible to use the configuration with the RAJA GitLab CI script outside of the GitLab environment: .. code-block:: bash $ HOST_CONFIG=/.cmake ./scripts/gitlab/build_and_test.sh .. important:: Before adding a new configuration to RAJA GitLab CI, it is prudent to manually perform the process of generating the corresponding host-config file and testing it by running it though our GitLab testing script as described here. MacOS ^^^^^ In RAJA, the Spack configuration for MacOS contains the default compiler corresponding to the OS version in the ``compilers.yaml`` file in the ``RAJA/scripts/radiuss-spack-configs/darwin/`` directory, and a commented section to illustrate how to add `CMake` as an external package in the ``packages.yaml`` in the same directory. You may also install CMake with `Homebrew `_, for example, and follow the process outlined above after it is installed. .. _docker_local-label: ================================== Reproducing Docker Builds Locally ================================== RAJA uses Docker container images that it shares with other LLNL GitHub projects for GitHub Actions CI testing (see :ref:`github_actions_ci-label` for more information). We use GitHub Actions for Linux, Windows, and MacOS builds for build environments and compiler versions that are not available on LLNL LC machines. You can reproduce GitHub Actions builds locally for testing using the following steps if you have access to Docker. #. Run the command to build a local Docker image:: $ DOCKER_BUILDKIT=1 docker build --target --no-cache Here, ```` is one of the names following ``AS`` in the `RAJA Dockerfile `_. #. To get dropped into a terminal in the Docker image, run the following:: $ docker run -it ghcr.io/llnl/radiuss: /bin/bash Here, ```` is the OS image and compiler image you want (see the aforementioned Dockerfile description). Then, you can build, run tests, edit files, etc. in the Docker image. Note that the Docker command has a ``-v`` argument that you can use to mount a local directory in the image. For example:: docker -v pwd:/opt/RAJA will mount your current local directory as ``/opt/RAJA`` in the image.