SC26Join us at Booth 5405 · Nov 16–19
Parallel Works

Custom OpenAI-Compatible Providers

A custom provider is a private or internal OpenAI-compatible endpoint, such as an inference server your organization runs, that a platform administrator adds to the provider catalog. The catalog entry pins the endpoint and its TLS policy. Users then connect to it from Add provider like any built-in entry, supplying only an API key.

Users can't type an endpoint of their own. To serve a model you run yourself without going through the catalog, expose it from a session instead; see Self-Hosted Sessions.

Endpoint Requirements

The endpoint must:

  • Serve HTTPS with TLS 1.2 or later.
  • Implement the OpenAI Chat Completions API under the configured API path. If it also implements the Responses API, choose the OpenAI Responses protocol so clients can use it.
  • Accept an API key in the Authorization: Bearer <key> header.
  • Be reachable from the ACTIVATE platform.

If the endpoint lists its models on the API path's /models route, users can enable models from that list.

Add a Custom Provider to the Catalog

  1. Open the Admin area and go to Products > AI > Provider catalog.
  2. Under Custom providers, click Add provider.
  3. Under Provider, fill in:
    • Stable ID: lowercase letters, numbers, and hyphens. It can't be changed after creation.
    • Display name: what users see in the Add provider menu.
    • Icon (optional): click Set icon to choose an icon shown wherever this provider appears.
  4. Under Endpoint, fill in:
    • Protocol: OpenAI Chat Completions or OpenAI Responses.
    • Tool calling: Native, Emulated from text, or Unavailable. A mode is required. See Tool Calling & Emulation.
    • HTTPS origin: for example https://inference.example.com.
    • API path: for example /v1.
  5. Under Transport security, set Minimum TLS (TLS 1.2 or TLS 1.3) and Certificate verification:
    • Publicly trusted certificate
    • Custom CA certificate: paste the CA in Approved custom CA (PEM).
    • Skip verification (insecure): traffic is still encrypted but the certificate isn't checked. Use it only when no CA you can supply will chain to the endpoint's certificate.
  6. Under Models, optionally fill in:
    • Allowed models: one model ID per line. "Users only see and enable models on this list. Leave empty to allow every model the provider reports."
    • Text-only models: models that reject image input, so clients skip sending them images.
  7. Click Create provider.

A new custom provider is enabled as soon as it is created. Its row menu under Custom providers has Edit provider, Disable provider (or Enable provider), and Delete provider. Disabling a custom provider suspends its connections without deleting them. Deleting one that connections still use offers Delete provider and connections, which permanently removes those connections, their credentials, and their access grants. Historical usage is preserved.

Connect to a Custom Provider

Once the entry is enabled, users connect to it the same way as any API-key entry:

  1. Navigate to AI > Providers.
  2. Click Add provider and choose the custom provider's display name.
  3. Enter a Provider name. The Endpoint URL is shown read-only.
  4. Paste the endpoint's API key.
  5. Click Create, then enable models in the provider's Models card.

Organization admins can turn on Organization-wide to create a metered organization provider instead. See Adding a Provider and Organization Providers.

In the providers list, connections to a custom provider are labeled OpenAI compatible. Owners can rotate the key from Edit, but the endpoint always comes from the catalog entry. When a platform administrator changes the entry's endpoint, every connection to it follows.

Notes for Common Inference Servers

These servers expose an OpenAI-compatible API. Put them behind HTTPS before adding them to the catalog.

  • vLLM: serves /v1/chat/completions and /v1/models. Start it with --api-key to require a key, and use /v1 as the API path.
  • Ollama: serves an OpenAI-compatible API under /v1. Model IDs are Ollama model tags, for example llama3.
  • Text Generation Inference (TGI): provides an OpenAI-compatible Messages API under /v1.