10) Modifiers

In this tutorial, you will learn how to discover modifiers, learn about their functionality, and apply them to experiments for WRF, a free and open-source application for atmospheric research and operational forecasting applications.

This tutorial builds off of concepts introduced in previous tutorials. Please make sure you review those before starting with this tutorial’s content.

NOTE: In this tutorial, you will encounter expected errors when copying and pasting the commands. This is to help show situations you might run into when trying to use Ramble on your own, and illustrate how you might fix them.

Create a Workspace

To begin with, you need a workspace to configure the experiments. This can be created with the following command:

$ ramble workspace create modifiers_wrf

Activate the Workspace

Several of Ramble’s commands require an activated workspace to function properly. Activate the newly created workspace using the following command: (NOTE: you only need to run this if you do not currently have the workspace active).

$ ramble workspace activate modifiers_wrf

Configure Experiment Definitions

To being with, you need to configure the workspace. The workspace’s root location can be seen under the Location output of:

$ ramble workspace info

Alternatively, the files can be edited directly with:

$ ramble workspace edit

Within the ramble.yaml file, write the following contents, which is the final configuration from a previous tutorial.

NOTE: This workspace utilizes the spack package manager. As a result, it requires spack is installed and available in your path. Modifications to the package_manager variant will change this behavior.

# Copyright 2022-2026 The Ramble Authors

# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.

ramble:
  variants:
    package_manager: spack
  env_vars:
    set:
      OMP_NUM_THREADS: '{n_threads}'
  variables:
    processes_per_node: 16
    n_ranks: '{processes_per_node}*{n_nodes}'
    batch_submit: '{execute_experiment}'
    mpi_command: mpirun -n {n_ranks}
  applications:
    wrf@4.2:
      workloads:
        CONUS_12km:
          experiments:
            scaling_{n_nodes}:
              variables:
                n_nodes: [1, 2]
  software:
    packages:
      gcc14:
        pkg_spec: gcc@14.2.0
      intel-mpi:
        pkg_spec: intel-oneapi-mpi@2021.17.2
        compiler: gcc14
      wrfv4-{application::wrf::version}:
        pkg_spec: wrf@{application::wrf::version} build_type=dm+sm compile_type=em_real nesting=basic ~chem
          ~pnetcdf
        compiler: gcc14
    environments:
      wrf@4.2:
        packages:
        - intel-mpi
        - wrfv4-{application::wrf::version}

The above configuration will execute 2 experiments, comprising a basic scaling study on 2 different sets of nodes. This is primarily defined by the use of vector experiments, which are documented in the vector logic portion of the workspace configuration file documentation. Vector experiments were also introduced in the vector and matrix tutorial.

Discovering Modifiers

In Ramble, a modifier is an abstract object that can be applied to a large set of experiments. Modifiers are intended to encasulate reusable patterns that perform a specific extension of a workload focused experiment. Some examples of useful modifiers include:

Collecting system level information

Changing underlying system / library functionality

Injecting Performance Analysis Tools

To discover which modifiers are available, execute:

$ ramble list --type modifiers

Which might output the following:

==> 4 modifiers
conditional-psm3  gcp-metadata  intel-aps  lscpu

This shows there are four modifiers in this installation of Ramble. Two very general modifiers in this list are lscpu and intel-aps. Modifiers are allowed to behave in different ways. Their functionality should be documented at a high-level through the ramble info --type modifiers command. To get information about the lscpu modifier, execute:

$ ramble info --type modifiers lscpu

This modifier adds the execution of lscpu to each experiment in a workspace (to capture additional platform level details, such as the CPU model), and defines additional figures of merit related to the output to lscpu.

The output of this command lists all supported modes of operation for the modifier, along with some of the behaviors each mode presents. A mode is a grouping of behaviors that control what the modifier will do to the experiments. Some modifiers only have a single mode of operation (as lscpu only has standard), while others might contain more. Modes are provided to allow users to easily control the behavior of the modifier.

Applying the `lscpu` Modifier

In the minimal case, a modifier can be added to all experiments in a workspace with two lines. In the general case, each modifier that is applied to experiments must define the mode of operation it will use. In the case that a modifier only has a single mode, it will be automatically selected as the default. Additionally, modifiers are allowed to define their own default modes to simplify its usage within a workspace.

To apply the lscpu modifier to your experiments, edit the workspace configuration file with:

$ ramble workspace edit

And add the following lines, at the workspace scope:

modifiers:
- name: lscpu

The resulting configuration file should look like the following.

# Copyright 2022-2026 The Ramble Authors

# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.

ramble:
  variants:
    package_manager: spack
  env_vars:
    set:
      OMP_NUM_THREADS: '{n_threads}'
  variables:
    processes_per_node: 16
    n_ranks: '{processes_per_node}*{n_nodes}'
    batch_submit: '{execute_experiment}'
    mpi_command: mpirun -n {n_ranks}
  modifiers:
  - name: lscpu
  applications:
    wrf@4.2:
      workloads:
        CONUS_12km:
          experiments:
            scaling_{n_nodes}:
              variables:
                n_nodes: [1, 2]
  software:
    packages:
      gcc14:
        pkg_spec: gcc@14.2.0
      intel-mpi:
        pkg_spec: intel-oneapi-mpi@2021.17.2
        compiler: gcc14
      wrfv4-{application::wrf::version}:
        pkg_spec: wrf@{application::wrf::version} build_type=dm+sm compile_type=em_real
          nesting=basic ~pnetcdf
        compiler: gcc14
    environments:
      wrf@4.2:
        packages:
        - intel-mpi
        - wrfv4-{application::wrf::version}

The modifiers dictionary can be defined at any scope variables can be defined at. Defined modifiers are inherited by all experiments at lower scope levels.

Execute Experiments

Now that you have made the appropriate modifications, set up, execute, and analyze the new experiments using:

$ ramble workspace setup
$ ramble on
$ ramble workspace analyze

This creates a results file in the root of the workspace that contains extracted figures of merit. If the experiments were successful, this file will show the following results:

Average Timestep Time: Time (in seconds) on average each timestep takes
Cumulative Timestep Time: Time (in seconds) spent executing all timesteps
Minimum Timestep Time: Minimum time (in seconds) spent on any one timestep
Maximum Timestep Time: Maximum time (in seconds) spent on any one timestep
Number of timesteps: Count of total timesteps performed
Avg. Max Ratio Time: Ratio of Average Timestep Time and Maximum Timestep Time

In addition to these WRF defined figures of merit, you should also see lscpu defined figures of merit. These might include the following:

CPU family

Model

Core(s) per socket

Socket(s)

You may also examine the execute_experiment script in each of the experiment run directories to see what changed after applying the lscpu modifier.

Advanced Modifiers

Some modifiers have additional functionality, which can include requiring specific software packages to be present. An example of this is the intel-aps modifier, which applies Intel’s Application Performance Snapshot to a workspace’s experiments.

To get information about the intel-aps modifier, execute:

$ ramble info --type modifiers intel-aps

In the output from this command, you should see a mode named mpi. One additional difference relateive to lscpu is that the Software Specs: section at the bottom of the output now has information in it. This is because Intel APS needs to be installed to be able to run applications under it.

To apply the intel-aps modifier, edit the workspace configuration file using:

$ ramble workspace edit

Within this file, add:

- name: intel-aps

To the list of modifiers. This will cause both lscpu and intel-aps to be applied to all of the experiments. The resulting configuration file should look like the following:

# Copyright 2022-2026 The Ramble Authors

# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.

ramble:
  variants:
    package_manager: spack
  env_vars:
    set:
      OMP_NUM_THREADS: '{n_threads}'
  variables:
    processes_per_node: 16
    n_ranks: '{processes_per_node}*{n_nodes}'
    batch_submit: '{execute_experiment}'
    mpi_command: mpirun -n {n_ranks}
  modifiers:
  - name: lscpu
  - name: intel-aps
  applications:
    wrf@4.2:
      workloads:
        CONUS_12km:
          experiments:
            scaling_{n_nodes}:
              variables:
                n_nodes: [1, 2]
  software:
    packages:
      gcc14:
        pkg_spec: gcc@14.2.0
      intel-mpi:
        pkg_spec: intel-oneapi-mpi@2021.17.2
        compiler: gcc14
      wrfv4-{application::wrf::version}:
        pkg_spec: wrf@{application::wrf::version} build_type=dm+sm compile_type=em_real
          nesting=basic ~pnetcdf
        compiler: gcc14
    environments:
      wrf@4.2:
        packages:
        - intel-mpi
        - wrfv4-{application::wrf::version}

To test some aspects of the workspace configuration before spending time performing a real setup, use:

$ ramble workspace setup --dry-run

This will perform many of the validation checks, and generate the experiment directories as a normal setup would, but can execute much faster. Before this command proceeds too far, you should see the following error message:

==> Error: Software spec intel-oneapi-vtune is not defined in environment wrf@4.2, but is required by the intel-aps modifier definition

As mentioned earlier, this is because the intel-aps modifier requires additional software to function properly. In this case, it requires the intel-oneapi-vtune package. However, this package is not defined, nor is it in your environment definition.

To remedy this issue, again edit your workspace configuration file using:

$ ramble workspace edit

And write a Spack package definition for intel-oneapi-vtune. After the package is defined, add the package to the wrf@4.2 environment. The resulting configuration file should look like the following:

# Copyright 2022-2026 The Ramble Authors

# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
# <LICENSE-MIT or https://opensource.org/licenses/MIT>, at your
# option. This file may not be copied, modified, or distributed
# except according to those terms.

ramble:
  variants:
    package_manager: spack
  env_vars:
    set:
      OMP_NUM_THREADS: '{n_threads}'
  variables:
    processes_per_node: 16
    n_ranks: '{processes_per_node}*{n_nodes}'
    batch_submit: '{execute_experiment}'
    mpi_command: mpirun -n {n_ranks}
  modifiers:
  - name: lscpu
  - name: intel-aps
  applications:
    wrf@4.2:
      workloads:
        CONUS_12km:
          experiments:
            scaling_{n_nodes}:
              variables:
                n_nodes: [1, 2]
  software:
    packages:
      gcc14:
        pkg_spec: gcc@14.2.0
      intel-mpi:
        pkg_spec: intel-oneapi-mpi@2021.17.2
        compiler: gcc14
      aps:
        pkg_spec: intel-oneapi-vtune
      wrfv4-{application::wrf::version}:
        pkg_spec: wrf@{application::wrf::version} build_type=dm+sm compile_type=em_real
          nesting=basic ~pnetcdf
        compiler: gcc14
    environments:
      wrf@4.2:
        packages:
        - intel-mpi
        - wrfv4-{application::wrf::version}
        - aps

Execute Experiments

Now that you have made the appropriate modifications, set up, execute, and analyze the new experiments using:

$ ramble workspace setup
$ ramble on
$ ramble workspace analyze

This creates a results file in the root of the workspace that contains extracted figures of merit. If the experiments were successful, this file will show the following results:

Average Timestep Time: Time (in seconds) on average each timestep takes
Cumulative Timestep Time: Time (in seconds) spent executing all timesteps
Minimum Timestep Time: Minimum time (in seconds) spent on any one timestep
Maximum Timestep Time: Maximum time (in seconds) spent on any one timestep
Number of timesteps: Count of total timesteps performed
Avg. Max Ratio Time: Ratio of Average Timestep Time and Maximum Timestep Time

In addition to these WRF defined figures of merit, you should also see lscpu, and intel-aps defined figures of merit. These might include the following:

CPU family - From lscpu

Model - From lscpu

Core(s) per socket - From lscpu

Socket(s) - From lscpu

MPI Time - From intel-aps

Disk I/O Time - From intel-aps

Clean the Workspace

Once you are finished with the tutorial content, make sure you deactivate your workspace:

$ ramble workspace deactivate

Additionally, you can remove the workspace and all of its content with:

$ ramble workspace remove modifiers_wrf

10) Modifiers

Create a Workspace

Activate the Workspace

Configure Experiment Definitions

Discovering Modifiers

Applying the lscpu Modifier

Execute Experiments

Advanced Modifiers

Execute Experiments

Clean the Workspace

Applying the `lscpu` Modifier