Skip to main content

Flexible Cloud Storage Resources

· 2 min read
Eric Garcia
Eric Garcia
Software Engineer
Olivia Shoup
Olivia Shoup
Technical Writer

Announcing our new feature to manage HPC storage resources in May 2023

As of today, you can access the newest page on the Parallel Works platform, Storage. Similar to our Compute and Workflows pages, you can navigate to Storage and create storage resources independently of your resources and workflows.

To reflect these changes, we've also added these pages to our user guide:

FAQ​

What kinds of storage can I add?​

Currently, you can create these storage resources:

  • AWS FSx for Lustre
  • Lustre on Azure
  • Lustre on Google
  • AWS S3 Bucket
  • Google Cloud Bucket

For more information about each type's configuration settings, please see Configuring Storage.

We'll also be adding more storage resources in the future.

What about Lustre storage?​

Previously, we had only one storage resource, Lustre, which you could configure inside a cluster's definition page. Now, Lustre and the other storage resources can all be configured on the Storage page.

The Lustre definition settings on the cluster configuration page have been replaced with a section called Attached Filesystems. You can use this section to attach Lustre or any other storage resources you've configured. Please see Attaching Storage for more information.

The three Lustre storage resources are available in a mode we're calling "ephemeral storage." Ephemeral storage shares the lifecycle of a cluster, and is provisioned when the power button of the cluster is pressed, and destroyed when it is pressed again.

We've also introduced "persistent storage," which is provisioned independently of clusters and can be mounted on a cluster while it's running.

For more information about ephemeral storage and persistent storage, please see Storage Types.

Can I use more than one storage at a time?​

Yes. With this latest update, you can configure more than one filesystem per cluster. Before these changes, clusters could only be configured with one storage definition at a time. Now, you can attach multiple filesystems at once, including Lustre and bucket storage from AWS or Google.