SC26AI agents lab + dinner on the river · Nov 17
Parallel Works
View as Markdown
Share:

About Networks

In order to provision cloud clusters, you must first deploy a network into your cloud account. A network is a networking stack deployed into your cloud service provider (CSP) that clusters run on top of. Networks are managed per cloud account, and you can deploy multiple networks within a single account.

Below, you can find specific details about the implementation of networks for each CSP. Each diagram has a tab for every networking mode, and choosing a mode on one diagram switches the others too.

How ACTIVATE Reaches Clusters on a Network

ACTIVATE does not need inbound access to your network to manage a cluster. The ACTIVATE agent on each cluster's controller opens an encrypted outbound connection to the platform on port 443, and terminals, sessions, file transfers, and jobs travel over it. The controller therefore needs an outbound route to the platform, for example through its public IP, a NAT gateway, or platform peering on AWS.

When a controller has a public IP, ACTIVATE connects to its SSH port (22) directly instead, so networks are created with a security rule that allows inbound SSH. On a Private network, all access goes through the agent connection.

AWS Networks

Deploying a network provisions the following resources on AWS. Which ones you get depends on the networking mode:

  • a Virtual Private Cloud (VPC) with a Route 53 private hosted zone and a security group
  • public subnets, private subnets, or both, in each availability zone of the region
  • an internet gateway, except on a Private network peered to the platform
  • a NAT gateway with an Elastic IP address, in Centralized NAT mode or on a Private network with Enable NAT gateway selected
  • a Transit Gateway attachment, on a Private network peered to the platform

The machine image used for the controller and compute nodes is inside the Parallel Works AWS account. Choose a mode below to see how a cluster connects in it.

Diagram of an ACTIVATE cluster on an AWS network in Centralized NAT mode: the controller sits in a public subnet and runs the ACTIVATE agent, which opens an encrypted tunnel to the platform through the internet gateway. Compute nodes sit in private subnets and reach the internet through one NAT gateway shared by the region. Because the controller has a public IP, the platform can also reach it directly over SSH on port 22.

Expense Information

A NAT gateway results in an additional hourly charge in your AWS account. For more information, please see this AWS pricing list.

Azure Networks

Deploying a network on Azure provisions a resource group, a virtual network, and subnets, plus a NAT gateway in the modes that use one. Each cluster gets its own network security group.

Compute nodes are always in a private subnet. In every mode except Private, the controller sits in a public subnet with its own public IP. On Azure, Controller NAT uses a load balancer created for the cluster rather than the controller, and Public IP gives a public IP to the controller only.

On a Private network, Azure default outbound access is turned off for its subnets, so a NAT gateway from Enable NAT gateway is the only way out. Without one, the network is isolated and its clusters can't reach the platform. Private networks created before this change keep default outbound access.

Diagram of an ACTIVATE cluster on an Azure network in Centralized NAT mode: the controller sits in a public subnet with its own public IP and runs the ACTIVATE agent, which opens an encrypted tunnel to the platform. Compute nodes in the private subnet reach the internet through an Azure NAT gateway with its own public IP. Because the controller has a public IP, the platform can also reach it directly over SSH on port 22.

Google Networks

Deploying a network on Google Cloud provisions a VPC network and firewall rules in the specified project, with one subnet per region rather than separate public and private subnets. Centralized NAT and Private networks with Enable NAT gateway selected also get a Cloud Router and Cloud NAT.

Diagram of an ACTIVATE cluster on a Google Cloud network in Centralized NAT mode: Google networks use one regional subnet. The controller has an external IP and runs the ACTIVATE agent, which opens an encrypted tunnel to the platform. Compute nodes have no external IPs and reach the internet through Cloud Router and Cloud NAT. Because the controller has an external IP, the platform can also reach it directly over SSH on port 22.

Oracle Networks

Deploying a network on Oracle Cloud provisions a Virtual Cloud Network (VCN) with security lists and route tables. Which other resources you get depends on the networking mode:

  • Centralized NAT creates an internet gateway and a NAT gateway, with a public and a private subnet in each availability domain.
  • Controller NAT and Public IP create an internet gateway and one subnet for the region.
  • Private creates one subnet for the region that doesn't allow public IPs, with no internet gateway or NAT gateway. Oracle doesn't offer Enable NAT gateway or Peer to platform, so clusters on a Private network can reach the platform only through an outbound route you add yourself, such as a VPN or proxy.

To set up an Oracle cloud account first, see Preparing Oracle.

OpenStack Networks

For OpenStack environments, network configuration is specific to your OpenStack deployment. Networks are created using your provider's available networking resources, regions, and flavors.