SC26AI agents lab + dinner on the river · Nov 17
Parallel Works

Environments

An environment describes where and how work can be scheduled on a cluster: a Slurm partition or PBS queue, its wall-time limits, and the scheduling fields users fill in (such as account and QoS). Users pick an environment to get a compute node for a desktop, a VS Code session, or a worker, and ACTIVATE submits the scheduler job for them.

Environments are available on cloud clusters, managed clusters, and existing clusters that have a scheduler.

Loading diagram...

Viewing environments

Open a cluster and click the Environments tab. The table shows each environment's Name, Partition, Nodes (available out of total), Wall time (default, with the maximum in parentheses), Source, and Status.

Screenshot of a cluster's Environments tab listing a detected and a custom environment, with the Add button highlighted.

Anyone who can see the cluster can view its environments. Users with login access to the cluster can schedule onto its active environments. Adding, editing, and deleting environments requires write access to the cluster.

Automatically detected environments

Connected clusters report their partitions to ACTIVATE, and ACTIVATE creates one environment per partition, with Source auto-detected. These stay in sync with the scheduler: node counts and wall-time limits update as the cluster reports them, and an environment whose partition disappears becomes inactive.

For many clusters, the detected environments are all you need. Edit one to add defaults or require an account (see Editing an environment).

Adding an environment

Add an environment yourself when you want a curated option, for example a "GPU visualization" choice with sensible defaults, on top of a partition.

  1. On the Environments tab, click Add.

  2. Fill in New environment:

    FieldDescription
    NameRequired. Unique within the cluster.
    Partition or queueRequired. The Slurm partition or PBS queue jobs are submitted to.
    SchedulerSLURM or PBS.
    Display nameOptional. What users see in pickers. Defaults to the name.
    DescriptionOptional. Shown under the name in the table.
    Default wall timeOptional. Used when a user doesn't set one. Use HH:MM:SS, or D-HH:MM:SS for multi-day limits.
    Maximum wall timeOptional. Users can't request more. Same format.
    Account is required to submitSlurm only. Users must pick an account.
    QoS is required to submitSlurm only. Users must pick a QoS.
    Screenshot of the New environment dialog filled in for a debug partition.
  3. Click Create. The environment reopens in edit mode so you can set its defaults.

Editing an environment

Click Edit on the environment's row. You can change the display name, description, wall times, required account and QoS, and Status (Active or Inactive). The name, partition, and scheduler can't be changed.

Expand Defaults and directives to set default values for the scheduling fields. Users see defaults as placeholders and can override them; fields left blank use the default.

Screenshot of the Edit environment dialog with Defaults and directives expanded.

Editing a detected environment's limits, status, or defaults changes its Source to custom. ACTIVATE then stops updating it from the scheduler, and you can delete it. Changing only the display name or description keeps it detected.

Scheduling fields

These are the fields users can set when they schedule onto an environment. Desktop and VS Code sessions always request one node.

FieldSlurmPBSNotes
Number of Nodes✓✓Default 1.
Number of tasks✓Default 1.
Cores per Task✓✓Number of CPU cores to allocate per task.
Wall Time✓✓Required. Can't exceed the maximum wall time.
Account✓✓Slurm lists the accounts the cluster reports; shown only when accounting is enabled. PBS takes a project or allocation code.
QoS✓Listed from the cluster; shown only when accounting is enabled.
Dedicated node✓✓Reserve a whole node for this worker instead of sharing it with other jobs. On by default.
Scheduler directives✓✓Extra directives, one #SBATCH or #PBS line per row. Use this for GPUs, memory, constraints, and similar options.

For example, to request a GPU on Slurm, set Scheduler directives to:

#SBATCH --gpus-per-node=1

Deleting an environment

Click Delete on a custom environment's row and confirm. Sessions and workers can no longer be scheduled with it. You can't delete an environment while workers are still using it; delete those workers first. Detected environments can't be deleted; set them to Inactive instead.

From the command line

pw environments ls --cluster my-cluster
pw environments create --cluster my-cluster --name gpu-viz --partition gpu --scheduler slurm --default-time 04:00:00
pw environments update gpu-viz --cluster my-cluster --default walltime=02:00:00

See pw environments for all commands.

Troubleshooting

MessageCause and fix
Cluster must be provisioned before creating environmentsStart the cloud cluster first.
Environment … is not activeThe environment is Inactive, or its partition is no longer reported.
Partition for environment … is …The partition is down on the scheduler.
Cannot delete this environment while … worker(s) are still using itDelete those workers from Workers first.