Authors: Jonathan Coles (CSCS), Benjamin Donald Cumming (CSCS), Theofilos-Ioannis Manitaras (CSCS), Jean-Guillaume Piccinali (CSCS), Simon Pintarelli (CSCS), Harmen Stoppels (Stoppels Consulting)
Abstract: We describe a method for defining, building and deploying alternative programming environments alongside the CPE on HPE Cray EX Alps infrastructure at CSCS.
This addresses an important strategic need at CSCS to deliver tailored environments within our versatile cluster (vCluster) configuration.
We provide compact, testable, optimized software environments that can be updated independently of the CPE release cycle.
The environments are defined with a descriptive YAML recipe, which is processed by a novel configuration tool that builds the software stackusing Spack and generates a SquashFS image. Cray-MPICH is provided through a custom Spack package without the need for a CPE installation. We describe the command line tools and Slurm plugin that facilitate loading environments per user and per job. Through a series of benchmarks we demonstrate application and micro-benchmark performance that matches CPE.
Long Description: We describe a method for defining, building and deploying alternative programming environments alongside the CPE on HPE Cray EX Alps infrastructure at CSCS.
This addresses an important strategic need at CSCS to deliver tailored environments within our versatile cluster (vCluster) configuration.
We provide compact, testable, optimized software environments that can be updated independently of the CPE release cycle.
The environments are defined with a descriptive YAML recipe, which is processed by a novel configuration tool that builds the software stackusing Spack and generates a SquashFS image. Cray-MPICH is provided through a custom Spack package without the need for a CPE installation. We describe the command line tools and Slurm plugin that facilitate loading environments per user and per job. Through a series of benchmarks we demonstrate application and micro-benchmark performance that matches CPE.
Paper: PDF