# Session Tunnel Providers

> Source: https://parallelworks.com/docs/ai/ai-chat-providers/session-tunnels

# Session Tunnel Providers

A session tunnel provider routes AI Chat traffic through a secure tunnel to a model running inside an ACTIVATE compute session. This allows you to use models hosted on your own cluster hardware without exposing them to the public internet.

## How Session Tunnels Work

When you start a session in ACTIVATE with an OpenAI-compatible model server running on it, you can create a tunnel that makes the model available to AI Chat. The tunnel establishes a secure connection between the ACTIVATE platform and the model endpoint inside your session.

Session tunnel providers appear in the AI Chat provider list with the type `openai-tunnel`. They are automatically discovered when a tunnel with the OpenAI flag enabled is active for your user account.

## Prerequisites

Before using a session tunnel provider:

- You have an active ACTIVATE session running on a cluster
- An OpenAI-compatible model server is running inside the session (for example, vLLM, Ollama, or TGI)
- A tunnel has been created from the session with the **OpenAI** option enabled

## Setting Up a Session Tunnel

1. Start a **session** on your cluster with enough resources to run your model (GPU, memory).
2. Inside the session, start an OpenAI-compatible inference server on a known port.
3. Create a **tunnel** for that session and enable the **OpenAI** toggle.
4. Once the tunnel status shows as **running**, the provider appears automatically in your AI Chat provider list.

You do not need to manually create a provider entry. ACTIVATE detects tunnels with the OpenAI flag and includes them in your available providers.

## Using a Session Tunnel Provider

1. Open **AI Chat**.
2. In the provider dropdown, look for your tunnel. It will show with the status of the tunnel (for example, "running" or "stopped").
3. Select the tunnel provider and send a message.

If the tunnel is in a "stopped" state, you must restart the session and tunnel before it can serve requests.

## Use Cases

- **Air-gapped environments** — Run models on infrastructure that has no direct internet access. The tunnel provides a secure path from ACTIVATE to the model.
- **Sensitive data** — Keep model inference on your own hardware to ensure data does not leave your environment.
- **Custom fine-tuned models** — Serve models you have fine-tuned on proprietary data without uploading them to a cloud provider.
- **Experimentation** — Quickly test different models by starting sessions with different configurations.

## Limitations

- **Session dependency** — The tunnel provider is only available while the session and tunnel are active. If the session stops, the provider becomes unavailable.
- **No sharing** — Session tunnel providers are tied to the user who created the tunnel. They cannot be shared with other users via the permissions system.
- **Single user** — Each tunnel is associated with one user account.

## Related Documentation

- [AI Chat Providers Overview](/docs/ai/ai-chat-providers) — Overview of all provider types
- [Custom OpenAI-Compatible](/docs/ai/ai-chat-providers/custom-openai-compatible) — Connecting a persistent external endpoint
- [Sessions](/docs/run/sessions) — Managing compute sessions