Package Managers
Within Ramble’s variants configuration section, users can control which package manager is used for a set of experiments.
The package manager used controls many aspects of the experiment, such as how software is installed. Each experiment within a workspace can use a different package manager, to explore the impact of changing this dimension of an experimental design.
Configuring Package Managers
Package managers are controlled through a config option in the variants configuration section. The following shows an example of controlling this
variants:
package_manager: <package_manager_name>
The default package manager is user-managed which tells Ramble the user will manage installing their own software, and providing Ramble with paths to the installations. The value of the package manager variant used can be a reference to a variable, and will be expanded following Ramble’s variable definitions logic.
Supported Package Managers
Some of the currently supported package managers in Ramble include:
environment-modules
eessi
pip
spack
spack-lightweight
spack-pip
user-managed
Environment Modules Package Manager
Setting the package manager config option to environment-modules enables
the use of Environment Modules
within the resulting experiments. Ramble will inject module load commands
into each experiment, to prepare the execution environment.
The software configuration section is used to determine what modules to load within the resulting environment.
The use of this package manager requires an installation of
environment-modules outside of Ramble. This is handled by system
administrators on several clusters, but for more information see
environment-modules’s documentation.
EESSI Package Manager
Setting the package manager config option to eessi enables the use of
the European Environment for Scientific Software Installations (EESSI)
for each experiment. Ramble will then inject commands to initialize the use of
EESSI, and load the correct module files for the execution environment.
The software configuration section is used to determine what modules to load within the resulting environment.
The use of this package manager requires an installation of EESSI outside of Ramble. For more information, see EESSI’s documentation.
Pip Package Manager
Setting the package manager config option to pip enables the use of pip and
python virtual environments for each experiment. During the setup pipeline,
Ramble will construct a python virtual environment in the workspace’s
software directory. This environment will contain pip installed python
packages and will be automatically loaded within experiments using this package
manager.
The software configuration section is used to determine what packages to install within the resulting virtual environment.
The use of this package manager requires pip to be installed outside of
Ramble. This happens automatically in several Python installations. For more
information see
pip’s documentation.
Spack Package Manager
Setting the package manager config option to spack enables the use of
Spack for each experiment. During the setup pipeline,
Ramble will construct Spack environments and install the requested software.
The experiments using Spack will automatically load the environment to prepare
the experiment for execution.
The software configuration section is used to determine what packages to install within the resulting environment.
When using the Spack package manager, workspaces can also use the
push-to-cache and mirror pipelines to cache compiled binaies, and
mirror software source.
The use of this package manager requires an external installation of Spack. For instructions on installing Spack, see Spack’s documentation.
Spack Lightweight Package Manager
Setting the package manager config option to spack-lightweight enables the
use of a lightweight version of Spack for each
experiment. During the setup pipeline, Ramble will construct Spack
environments, however installation is deferred. This allows experiments to
install their own software, enabling parallel installation, rather than
requiring sequential installation at workspace setup time.
The software configuration section is used to determine what packages to install within the resulting environment.
When using the Spack package manager, workspaces can also use the
push-to-cache and mirror pipelines to cache compiled binaies, and
mirror software source.
The use of this package manager requires an external installation of Spack. For instructions on installing Spack, see Spack’s documentation.
Spack Pip Package Manager
Setting the package manager config option to spack-pip enables a hybrid
mode where Spack is used to install base software, and pip is used to
install additional Python packages into the environment.
User-Managed Package Manager
Setting the package manager config option to user-managed disables Ramble’s
automatic software installation. In this mode, the user is responsible for
ensuring that the required software is available in the environment before
running experiments.