# Self-Hosted Session Providers

> Source: https://parallelworks.com/docs/ai/ai-providers/self-hosted-sessions

# Self-Hosted Session Providers

A **Self-hosted session** provider serves a model from an OpenAI-compatible server you run yourself, inside an ACTIVATE session. Requests reach the server through the session's tunnel, so the model runs on your own cluster, workspace, or machine and needs no public endpoint.

You don't add these from **Add provider**. A session becomes a provider when you mark it as serving an OpenAI-compatible API, and it then appears in **AI** > **Providers** with the type **Self-hosted session**, a live **Status**, and its models.

## Prerequisites

- A platform administrator has enabled the **Self-hosted session** entry in the platform's provider catalog. Until it is enabled, marking a session as OpenAI-compatible fails with "Self-hosted AI providers are not enabled on this platform". See [Platform Setup](/docs/ai/ai-providers/platform-setup).
- An OpenAI-compatible model server (for example vLLM, Ollama, TGI, or llama.cpp) serving `/v1/chat/completions` and `/v1/models`.

## Expose a Model From a Tunnel Session

1. Start your model server on a cluster or in your user workspace, listening on a known port.
2. Navigate to **Sessions**, click **Create**, and choose **Tunnel session**.
3. Choose the **Target** and enter the server's **Remote port**.
4. Under the advanced options, turn on **OpenAI** ("Mark this session as providing an OpenAI-compatible API").
5. If the server requires a key, enter it in **API key**.
6. Click **Create session**.

See [Sessions](/docs/run/sessions) for the other session fields.

## Expose a Model From Your Own Machine

To serve a model running on your laptop or another machine outside ACTIVATE, use an endpoint session with `--openai`:

```bash
pw endpoints http --openai --name my-llm 11434
```

See [Expose a local AI model](/docs/run/sessions/endpoints/#expose-a-local-ai-model) for `pw endpoints run` and servers, such as Ollama, that need `--rewrite-host`.

## Use the Model

Once the session is running, its models are listed with IDs of the form `session:owner:session-name/model`, for example `session:jsmith:my-llm/llama3`. Pick them in Chat or `pw code`, or call them from external tools.

Session models are not metered and need no allocation. They are available only while the session is running. If the providers list shows **service not responding**, the tunnel is connected but nothing is listening on the remote port; start the model server to resolve it.

## Share a Session Model

Other users in your organization can use a session's models when the session is shared with them:

1. In **AI** > **Providers**, open the session's row menu and choose **Share**. This opens the session's access page, the same one you reach from **Sessions**.
2. Under **Organization access**, share the session with your whole organization, or under **Group access**, choose the groups to share it with.

Users in the organization or in those groups see the session's models and can send requests to them. Sharing a session also lets those users open whatever the session serves, so only share with groups you trust. **Public access** does not make a session usable as a model by other users.

Your organization's policy can prevent sharing sessions with the entire organization.

## Use Cases

- **Private or air-gapped models**: keep inference on infrastructure with no public endpoint.
- **Sensitive data**: keep prompts and responses on your own hardware.
- **Fine-tuned models**: serve models trained on proprietary data without uploading them to a vendor.
- **Experimentation**: try different models by starting sessions with different servers.

## Related Documentation

- [AI Providers Overview](/docs/ai/ai-providers): All provider types
- [Sessions](/docs/run/sessions): Creating and sharing sessions
- [Endpoint Sessions](/docs/run/sessions/endpoints): Exposing apps on your own machine
- [Custom OpenAI-Compatible Providers](/docs/ai/ai-providers/custom-openai-compatible): Persistent internal endpoints in the catalog