Custom OpenAI-Compatible Provider

A custom provider connects ACTIVATE AI Chat to any endpoint that implements the OpenAI chat completions API. This gives you flexibility to use self-hosted models, third-party inference services, or any other compatible backend.

Endpoint Requirements

Your endpoint must support the following:

• POST /v1/chat/completions — The standard OpenAI chat completions endpoint. ACTIVATE sends all chat messages to this route.
• GET /v1/models (optional) — If your endpoint exposes this route, ACTIVATE can list available models dynamically so users can switch models without reconfiguring the provider.

The endpoint must accept an API key via the Authorization: Bearer <key> header.

Prerequisites

Before creating a custom provider, ensure that:

• Your inference endpoint is running and reachable from the ACTIVATE platform
• You have the endpoint URL and a valid API key
• You know the model name to use (for example, "meta-llama/Llama-3-70b-chat-hf")

Creating a Provider

1. Navigate to AI Chat Providers in the ACTIVATE sidebar.
2. Click Add Provider.
3. Select Custom as the cloud service provider.
4. Fill in the required fields:
   • Name — A descriptive name for the provider (for example, "vLLM Llama 3").
   • Endpoint — The full URL to your chat completions endpoint (for example, https://my-server.example.com/v1/chat/completions).
   • API Key — The authentication key for your endpoint.
   • Model — The model identifier your endpoint expects (for example, "meta-llama/Llama-3-70b-chat-hf").
5. Optionally add a Description and Tags.
6. Click Create.

Custom providers are available immediately after creation — no provisioning step is required.

Verifying the Provider

1. Open AI Chat.
2. Select your new custom provider from the provider dropdown.
3. Send a test message to confirm the endpoint responds correctly.

If the model list dropdown is populated, your endpoint supports the /v1/models route and ACTIVATE has successfully queried it.

Updating a Provider

You can update the endpoint, API key, and model for a custom provider at any time:

1. Navigate to AI Chat Providers.
2. Select the provider you want to update.
3. Modify the fields as needed.
4. Save your changes.

This is useful for rotating API keys or pointing the provider at a different model without creating a new provider.

Notes for Common Inference Servers

vLLM — Start vLLM with the --api-key flag to enable API key authentication. The default port is 8000. Your endpoint URL will be http://<host>:8000/v1/chat/completions.

Ollama — Ollama exposes an OpenAI-compatible API on port 11434. Set the endpoint to http://<host>:11434/v1/chat/completions. The model name should match the Ollama model tag (for example, "llama3").

Text Generation Inference (TGI) — TGI provides an OpenAI-compatible Messages API. Set the endpoint to http://<host>:8080/v1/chat/completions.

Deleting a Provider

1. Navigate to AI Chat Providers.
2. Find the provider you want to remove.
3. Click the delete action for that provider.
4. Confirm the deletion.

Deleting a custom provider only removes it from ACTIVATE. It does not affect the external inference endpoint.

Related Documentation

• AI Chat Providers Overview — Overview of all provider types
• Managing Permissions — Sharing providers with your team
• Session Tunnels — Tunneling to a model running in a compute session