Skip to main content

Getting Started: Cloud Clusters

Because there are many uses for ACTIVATE, this tutorial's aim is to familiarize users with the platform and typical steps a user might follow. You will learn how to provision elastic HPC clusters, manage buckets, transfer data, and run computations.

Provision a Bucket

You can use several different types of storage on ACTIVATE. For this guide, we'll be using bucket storage. Buckets are efficient, inexpensive, and and easy to use.

Navigate to Buckets.

Screenshot of the user clicking Buckets in the ACTIVATE sidebar.

Click +Add Storage.

Screenshot of the user clicking the +Add Storage button.

Choose which type of bucket you'll create.

Give your bucket a Resource Name. Optionally, add a Description and Tags.

Click Add Storage.

Screenshot of the user clicking the Add Storage button.

On the next page, the bucket will be pre-configured with your organization's default Cloud Infrastructure and Group. If you need to change these parameters, click Save Changes after doing so.

Screenshot of the user clicking Save Changes for a bucket.

Click the bucket's power button. The bucket will take a few minutes to start. You can proceed to the next step while you wait.

Provision a Cluster

Navigate to the Marketplace.

Screenshot of the user clicking Marketplace.

Select Compute to see pre-configured clusters.

Screenshot of the user clicking Compute in the Marketplace.

Select the cluster that best suits your needs.

Screenshot of the user clicking a cluster in the Marketplace.

Click Fork latest.

Screenshot of the user clicking the Fork button.

Enter a name for your cluster under the New Compute Name option and click Fork.

Screenshot of the user clicking the Fork button.

Use the Group dropdown menu to select the group that your organization uses to allocate costs. The remaining cluster settings have already been configured. Click Save Changes.

Screenshot of the user clicking the Save Changes button.

Attach a Bucket

Click Edit on your cluster's page.

Screenshot of the user clicking Edit on a cluster.

At the bottom of the page, click +Add Attached Filesystems.

Screenshot of the user clicking Add Attached Filesystems.

Select the bucket you want to use from the Storage dropdown menu. Enter /bucket in the Mount Point option. Click Save Changes.

Screenshot of the user clicking Save Changes on a cluster.

Start the Cluster

Start the cluster by clicking Start/Resume.

Screenshot of the user clicking a cluster's Start/Resume button.

You'll see the message Starting [Cluster Name]. While the cluster starts, the Status will show provisioning with a yellow bubble.

When your cluster is online, both the Status will show active with a green bubble.

Screenshot of an active cluster.

Log In to the Cluster

Clusters use one node, called a controller, to delegate tasks to compute nodes. You'll need to log in to the controller to interact with the cluster.

While on your cluster's page, click its IP Address.

Screenshot of a user clicking a cluster's IP address.

A terminal will pop up on the bottom of the screen, showing the following message:

Warning: Permanently added '34.60.48.100' (ED25519) to the list of known hosts.
[username@cluster-name-01 ~]$

Create a File

Next, you'll create a file inside your user container, then transfer it to your bucket. The instructions in the following sections follow this MPI tutorial.

In the terminal, click the plus icon to open another tab.

Screenshot of a user clicking the plus button in a terminal on ACTIVATE.

In the new terminal tab, create a file with this code:

cat << EOF >> hello-world.c
#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
// Initialize the MPI environment
MPI_Init(NULL, NULL);

// Get the number of processes
int world_size;
MPI_Comm_size(MPI_COMM_WORLD, &world_size);

// Get the rank of the process
int world_rank;
MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

// Get the name of the processor
char processor_name[MPI_MAX_PROCESSOR_NAME];
int name_len;
MPI_Get_processor_name(processor_name, &name_len);

// Print off a hello world message
printf("Hello world from processor %s, rank %d out of %d processors\n",
processor_name, world_rank, world_size);

// Finalize the MPI environment.
MPI_Finalize();
}
EOF

Press Enter.

Retrieve Bucket Credentials

All buckets use short-term credentials, making it easy to transfer data between a bucket and other resources. You'll need those credentials to manage access to the bucket.

Navigate to your bucket's page. Click Credentials.

Screenshot of a user clicking the Credentials button on an active bucket.

A dialog box will appear with CSP-specific commands for your bucket.

Screenshot of AWS bucket credentials.

Copy and paste the first command into the same terminal tab where you created the file above. Press Enter.

Upload File to Bucket

Next, you'll use the PW CLI to transfer the file you created above.

In the same terminal tab, enter this commmand to copy hello-world.c to your bucket:

pw buckets cp hello-world.c $BUCKET_URI/hello-world.c

Use the list command pw buckets ls to check that your file transferred:

username@pw-user-username:~/pw$ pw buckets ls pw://username/bucketname
pw://username/bucketname/hello-world.c
username@pw-user-username:~/pw$

Authenticate the PW CLI

Before you transfer the file from the bucket to your cluster, you'll need to authenticate the PW CLI on the cluster.

First, create a short-term token on ACTIVATE. You can follow our instructions here. Copy the token.

Log in to the cluster again (this step above).

Enter pw auth token and paste your token. Press Enter. You'll see the following message:

2025-03-31T16:23:19Z [INFO] Authenticated as username
[username@usernamecluster-3 ~]$

Download File to Cluster

Copy and paste your bucket's credentials again (this step above).

Copy and paste the exact command below. The value for BUCKET_URI will be auto-populated with your bucket's URI.

pw buckets cp $BUCKET_URI/hello-world.c hello-world.c

Compile and Run MPI Test

Enter this command:

export PATH=/usr/lib64/openmpi/bin:$PATH

Enter this command:

mpicc -o hello hello-world.c

Next, you'll create a test job using a script. Enter the following:

cat << EOF >> hello-world.sbatch
#!/bin/bash

#SBATCH -N 2
#SBATCH --ntasks-per-node=2

export PATH=/usr/lib64/openmpi/bin:$PATH

mpirun -np 2 -N 1 $HOME/hello
EOF

Press Enter.

Run the job with this command:

sbatch hello-world.sbatch

Monitor Job

Go to your cluster's page. In the Observability module, you'll see your cluster's CPU activity change as it runs the job you just submitted.

Screenshot of a cluster&#39;s Observability module.

Click Jobs.

Screenshot of the user clicking the Jobs tab on a cluster&#39;s page.

You'll see the state of your partition.

Screenshot of a cluster&#39;s running job.

Shut Down Resources

Navigate to the configuration page for your bucket. Click the power button. A module will appear. Click Deprovision.

Next, navigate to the configuration page for your cluster. Click Destroy.

Screenshot of a user clicking the Destroy button a cluster.

A module will appear. Click Destroy.