# Custom OpenAI-Compatible Provider

> Source: https://parallelworks.com/docs/ai/ai-chat-providers/custom-openai-compatible

A custom provider connects ACTIVATE AI Chat to any endpoint that implements the OpenAI chat completions API. This gives you flexibility to use self-hosted models, third-party inference services, or any other compatible backend.

## Endpoint Requirements

Your endpoint must support the following:

- **POST `/v1/chat/completions`** — The standard OpenAI chat completions endpoint. ACTIVATE sends all chat messages to this route.
- **GET `/v1/models`** (optional) — If your endpoint exposes this route, ACTIVATE can list available models dynamically so users can switch models without reconfiguring the provider.

The endpoint must accept an API key via the `Authorization: Bearer <key>` header.

## Prerequisites

Before creating a custom provider, ensure that:

- Your inference endpoint is running and reachable from the ACTIVATE platform
- You have the endpoint URL and a valid API key

## Creating a Provider

1. Navigate to **AI Chat Providers** in the ACTIVATE sidebar.
2. Click **Add Provider**.
3. Select **Custom** as the cloud service provider.
4. Fill in the required fields:
   - **Name** — A descriptive name for the provider (for example, "vLLM Llama 3").
   - **Endpoint** — The full URL to your chat completions endpoint (for example, `https://my-server.example.com/v1/chat/completions`).
   - **API Key** — The authentication key for your endpoint.
5. Optionally add a **Description** and **Tags**.
6. Click **Create**.

Custom providers are available immediately after creation — no provisioning step is required.

## Verifying the Provider

1. Open **AI Chat**.
2. Select your new custom provider from the provider dropdown.
3. Send a test message to confirm the endpoint responds correctly.

ACTIVATE automatically discovers available models by querying your endpoint's `/v1/models` route. If the model list dropdown is populated, your endpoint supports this route and ACTIVATE has successfully queried it.

## Updating a Provider

You can update the endpoint and API key for a custom provider at any time:

1. Navigate to **AI Chat Providers**.
2. Select the provider you want to update.
3. Modify the fields as needed.
4. Save your changes.

This is useful for rotating API keys or pointing the provider at a different endpoint without creating a new provider.

## Notes for Common Inference Servers

**vLLM** — Start vLLM with the `--api-key` flag to enable API key authentication. The default port is 8000. Your endpoint URL will be `http://<host>:8000/v1/chat/completions`.

**Ollama** — Ollama exposes an OpenAI-compatible API on port 11434. Set the endpoint to `http://<host>:11434/v1/chat/completions`. The model name should match the Ollama model tag (for example, "llama3").

**Text Generation Inference (TGI)** — TGI provides an OpenAI-compatible Messages API. Set the endpoint to `http://<host>:8080/v1/chat/completions`.

## Deleting a Provider

1. Navigate to **AI Chat Providers**.
2. Find the provider you want to remove.
3. Click the delete action for that provider.
4. Confirm the deletion.

Deleting a custom provider only removes it from ACTIVATE. It does not affect the external inference endpoint.

## Related Documentation

- [AI Chat Providers Overview](/docs/ai/ai-chat-providers) — Overview of all provider types
- [Managing Permissions](/docs/ai/ai-chat-providers/managing-permissions) — Sharing providers with your team
- [Session Tunnels](/docs/ai/ai-chat-providers/session-tunnels) — Tunneling to a model running in a compute session