Tool Calling & Emulation

Agentic clients such as pw code work by calling tools: the model asks to read a file, run a command, or search the workspace, and the client executes the request and returns the result. Many model endpoints implement tool calling directly, but some do not.

Every model reachable through the ACTIVATE AI gateway is advertised with one of three tool-calling modes. Models whose endpoint has no tool-calling API can still be used for agentic work through emulated tool calling, which the gateway handles on their behalf.

The Three Modes

Where the Mode Comes From

Each catalog entry has a tool-calling mode, and a connection inherits it:

• Built-in entries carry their mode in their built-in definition, and platform administrators can't change it. The built-in entries are all native.
• Custom entries get their mode from the platform administrator, who must choose one (Native, Emulated from text, or Unavailable) when adding the entry. See Custom OpenAI-Compatible Providers.

Some connections can override the inherited mode. The Tool calling row (Emulated, Native, or Off) appears on a provider's Edit page when its catalog entry is emulated, or when the provider type serves some models without native tool calls next to models that have them. GenAI.mil is the built-in example: it defaults to native, and a connection can switch to emulated to use its agent models. The provider's details page shows the result as Native, Emulated from text, or Disabled.

In native mode, models that are known not to support native tool calls are hidden from that connection's model list. For GenAI.mil these are models matching *-assist and text-embedding-*. Switching the connection to emulated lists them.

Seeing the Mode

The mode is reported wherever models are listed:

# The "Tool calling" column shows yes, emulated, or no
pw ai models ls

The models endpoint returns it as tool_calling_mode on each entry:

GET /api/openai/v1/models

Inside an interactive pw code session, the status line below the composer shows emulated tools when the active model is running in emulated mode, and tools unavailable when the model's mode is none.

Why Emulation Exists

Some endpoints, including agent-wrapper services and a number of self-hosted inference servers, accept OpenAI-compatible chat requests but do not support the tools parameter. Without emulation, those models would be limited to chat and could not be used with pw code.

Emulation makes tool calling something the gateway provides rather than something the provider must support. Clients send the same OpenAI-shaped request with a tools array and receive the same tool_calls in the response, so no client-side configuration changes.

Architecture

All three modes follow one path from the client to the model. The gateway resolves the provider, reads its connection's tool-calling mode, and applies the matching translation layer. Only the emulation layer rewrites the request; native passes it through and none rejects it.

The client uses the same contract in every mode. It sends tools, receives tool_calls, and runs them locally in your workspace. The model does not execute anything itself in either mode.

How Emulation Works

When a request with tool definitions is routed to an emulated provider, the gateway translates in both directions. A single tool round trip looks like this.

1. The Tool Contract Moves Into the Prompt

The gateway strips tools, tool_choice, and parallel_tool_calls from the upstream request and appends an instruction block to the conversation describing:

• The full JSON Schema of every tool the client defined.
• The exact shape of a tool-call envelope the model must emit.
• The tool policy for this request, derived from tool_choice (optional, required, or one named function) and from whether parallel calls are allowed.

The block is appended to the user turn rather than sent as a system message. Some agent-wrapper endpoints apply their own system prompt and discard the one supplied by the caller, while still honoring user content, so instructions sent as a system message would not reach the model on those providers.

2. The Model Replies With an Envelope

To call a tool, the model responds with a single JSON object and nothing else:

{
  "protocol": "parallelworks.tool-call.v1",
  "toolset": "sha256:9f2c…",
  "nonce": "4b81f0c39ad2",
  "calls": [
    { "name": "Bash", "arguments": { "command": "ls -la", "description": "list files" } }
  ]
}

The gateway checks three framing values before it honors an envelope:

• protocol identifies the envelope format.
• toolset is a digest of the tool definitions sent with this request, so an envelope written against a different set of tools is not accepted.
• nonce is generated for each request and must be echoed back. Content that existed before the request cannot contain the current nonce, so an envelope taken from earlier conversation content or from tool output is not accepted as a live call.

3. The Gateway Validates and Converts

The gateway locates the envelope in the model's output, allowing for code fences and surrounding prose, then checks it:

• Protocol, toolset, and nonce must match this request.
• The named tool must be one the client defined.
• The arguments must validate against that tool's JSON Schema.
• The call count must match the request's parallel-call policy.

A valid envelope becomes ordinary OpenAI tool_calls with generated call IDs and a tool_calls finish reason, so the client receives a standard tool-calling response.

Output that is not an envelope attempt is returned unchanged as a plain text answer. Ordinary replies, including replies that contain JSON, are not affected.

4. Near Misses Are Repaired, Then Retried Once

Models trained on other tool formats produce a small set of predictable variations. The gateway repairs these locally rather than making another upstream call:

• Alternate field names for the call list (tool_calls, toolCalls) and for arguments (parameters, args, input).
• A call wrapped in a function / function_call object.
• A single call emitted bare, or a call list emitted as one object instead of an array.
• Arguments emitted as a JSON string instead of an object.
• Extra top-level fields, duplicate keys, and reordered or reserialized JSON.

Repairs are re-validated against the same rules, and a repaired envelope that still fails validation is not accepted. Nonce repair is declined when the envelope appears to have been copied out of tool output.

If validation still fails, the gateway makes one corrective retry. It replays the rejected output, states the reason it was rejected, and supplies a skeleton envelope with the correct protocol, toolset, and nonce already filled in, leaving the model only the call to write. If the retry also fails, the request returns an error rather than passing malformed text back as an answer.

5. Tool Results Go Back as Fenced, Untrusted Data

On the next turn, the client's tool messages are rendered as text the model can read. Each result is wrapped in [TOOL_OUTPUT <nonce>] markers carrying the same per-request nonce and labeled as untrusted data that must not be followed as instructions. Since the nonce is new for each request, earlier output cannot reproduce the markers to close the fence early or imitate gateway framing. Images in tool results are passed through as images.

Behavior to Expect

Streaming. A streaming request to an emulated model is still streamed from the provider, but the gateway needs the complete response before it can tell whether the output is an envelope or an answer. The response is buffered and then sent to the client as a stream, so tokens do not appear incrementally as they do with a native provider.

Prompt caching. The tool contract sits in the user turn and carries a per-request nonce, so the prompt prefix differs on every request. Emulated requests do not benefit from provider-side prompt-prefix caching.

Token cost. Each request repeats the JSON Schema of every tool in the prompt, which accounts for a noticeable share of input tokens when the toolset is large. A corrective retry adds the cost of a second call for that turn. Usage from both calls is metered.

The Responses API. Emulation is available over Chat Completions only. A request to /api/openai/v1/responses for an emulated provider is rejected with tool_call_emulation_requires_chat_completions, and the models list doesn't offer the Responses API for emulated connections. The same applies to connections with Fold history turned on, which are rejected with history_folding_requires_chat_completions.

Reliability. Emulation depends on the model following instructions and copying the framing values accurately. Models that follow instructions well use tools consistently; smaller models, and models behind an agent wrapper, call tools less consistently than they would through a native API.

Limits

Requests that exceed these bounds are rejected before they are sent upstream.

Tool names must be 1 to 64 characters of letters, numbers, underscores, or hyphens, and must be unique within a request. Only function tools are supported. Parameter schemas that expand to a very large number of branches through $ref and anyOf are rejected before compilation.

Troubleshooting

The status line says emulated tools unexpectedly. The active model's connection is in emulated mode. Switch to a model from a native provider with /model. If the provider's Edit page has a Tool calling row, its owner can switch it to Native; otherwise ask your platform administrator whether the catalog entry's mode is correct for its endpoint.

The model answers in prose instead of using a tool. This is the most common emulation failure. Stating the action directly ("read main.go and tell me what it does") usually helps, as does selecting a larger model from the same provider.

"This model does not support tool calling." The connection's mode is none. Run pw code --no-tools for a chat-only session, or choose a different model.

Calls fail repeatedly on one specific tool. Emulated arguments are validated against the tool's JSON Schema as written. A large or deeply nested schema is harder for a model to satisfy from a prompt than through a native API, so simplifying the schema usually resolves it.

Related Documentation

• Custom OpenAI-Compatible Providers: Register an endpoint and set its tool-calling mode
• Adding a Provider: Override the mode on a connection
• pw code: The AI coding agent that uses tool calling
• Models & Allocations: Discovering and switching models
• AI Keys: Programmatic access through the OpenAI-compatible API