Tool Calling & Emulation

Agentic clients such as pw code work by calling tools: the model asks to read a file, run a command, or search the workspace, and the client executes the request and returns the result. Many model endpoints implement tool calling directly, but some do not.

Every model reachable through the ACTIVATE AI gateway is advertised with one of three tool-calling modes. Models whose endpoint has no tool-calling API can still be used for agentic work through emulated tool calling, which the gateway handles on their behalf.

The Three Modes

The mode is a property of the provider integration, not of the individual model. Platform administrators set it when creating or editing a provider, and it is required when registering a custom OpenAI-compatible integration. Built-in integrations ship with the correct mode already set; GenAI.mil, for example, is registered as emulated.

Seeing the Mode

The mode is reported wherever models are listed:

# The "Tool calling" column shows yes, emulated, or no
pw ai models ls

The models endpoint returns it as tool_calling_mode on each entry:

GET /api/openai/v1/models

Inside an interactive pw code session, the status line below the composer shows emulated tools when the active model is running in emulated mode, and tools unavailable when the model's mode is none.

Why Emulation Exists

Some endpoints, including agent-wrapper services and a number of self-hosted inference servers, accept OpenAI-compatible chat requests but do not support the tools parameter. Without emulation, those models would be limited to chat and could not be used with pw code.

Emulation makes tool calling a property of the gateway rather than of the provider. Clients send the same OpenAI-shaped request with a tools array and receive the same tool_calls in the response, so no client-side configuration changes.

Architecture

All three modes follow one path from the client to the model. The gateway resolves the provider, reads its tool-calling mode, and applies the matching translation layer. Only the emulation layer rewrites the request; native passes it through and none rejects it.

The client uses the same contract in every mode. It sends tools, receives tool_calls, and runs them locally in your workspace. The model does not execute anything itself in either mode.

How Emulation Works

When a request with tool definitions is routed to an emulated provider, the gateway translates in both directions. A single tool round trip looks like this.

1. The Tool Contract Moves Into the Prompt

The gateway strips tools, tool_choice, and parallel_tool_calls from the upstream request and appends an instruction block to the conversation describing:

• The full JSON Schema of every tool the client defined.
• The exact shape of a tool-call envelope the model must emit.
• The tool policy for this request, derived from tool_choice (optional, required, or one named function) and from whether parallel calls are allowed.

The block is appended to the user turn rather than sent as a system message. Some agent-wrapper endpoints apply their own system prompt and discard the one supplied by the caller, while still honoring user content, so instructions sent as a system message would not reach the model on those providers.

2. The Model Replies With an Envelope

To call a tool, the model responds with a single JSON object and nothing else:

{
  "protocol": "parallelworks.tool-call.v1",
  "toolset": "sha256:9f2c…",
  "nonce": "4b81f0c39ad2",
  "calls": [
    { "name": "Bash", "arguments": { "command": "ls -la", "description": "list files" } }
  ]
}

The gateway checks three framing values before it honors an envelope:

• protocol identifies the envelope format.
• toolset is a digest of the tool definitions sent with this request, so an envelope written against a different set of tools is not accepted.
• nonce is generated for each request and must be echoed back. Content that existed before the request cannot contain the current nonce, so an envelope taken from earlier conversation content or from tool output is not accepted as a live call.

3. The Gateway Validates and Converts

The gateway locates the envelope in the model's output, allowing for code fences and surrounding prose, then checks it:

• Protocol, toolset, and nonce must match this request.
• The named tool must be one the client defined.
• The arguments must validate against that tool's JSON Schema.
• The call count must match the request's parallel-call policy.

A valid envelope becomes ordinary OpenAI tool_calls with generated call IDs and a tool_calls finish reason, so the client receives a standard tool-calling response.

Output that is not an envelope attempt is returned unchanged as a plain text answer. Ordinary replies, including replies that contain JSON, are not affected.

4. Near Misses Are Repaired, Then Retried Once

Models trained on other tool formats produce a small set of predictable variations. The gateway repairs these locally rather than making another upstream call:

• Alternate field names for the call list (tool_calls, toolCalls) and for arguments (parameters, args, input).
• A call wrapped in a function / function_call object.
• A single call emitted bare, or a call list emitted as one object instead of an array.
• Arguments emitted as a JSON string instead of an object.
• Extra top-level fields, duplicate keys, and reordered or reserialized JSON.

Repairs are re-validated against the same rules, and a repaired envelope that still fails validation is not accepted. Nonce repair is declined when the envelope appears to have been copied out of tool output.

If validation still fails, the gateway makes one corrective retry. It replays the rejected output, states the reason it was rejected, and supplies a skeleton envelope with the correct protocol, toolset, and nonce already filled in, leaving the model only the call to write. If the retry also fails, the request returns an error rather than passing malformed text back as an answer.

5. Tool Results Go Back as Fenced, Untrusted Data

On the next turn, the client's tool messages are rendered as text the model can read. Each result is wrapped in [TOOL_OUTPUT <nonce>] markers carrying the same per-request nonce and labeled as untrusted data that must not be followed as instructions. Since the nonce is new for each request, earlier output cannot reproduce the markers to close the fence early or imitate gateway framing. Images in tool results are passed through as images.

Behavior to Expect

Streaming. A streaming request to an emulated model is still streamed from the provider, but the gateway needs the complete response before it can tell whether the output is an envelope or an answer. The response is buffered and then sent to the client as a stream, so tokens do not appear incrementally as they do with a native provider.

Prompt caching. The tool contract sits in the user turn and carries a per-request nonce, so the prompt prefix differs on every request. Emulated requests do not benefit from provider-side prompt-prefix caching.

Token cost. Each request repeats the JSON Schema of every tool in the prompt, which accounts for a noticeable share of input tokens when the toolset is large. A corrective retry adds the cost of a second call for that turn. Usage from both calls is metered.

The Responses API. Emulation is available over Chat Completions only. A request to /api/openai/v1/responses for an emulated provider is rejected with tool_call_emulation_requires_chat_completions, and the models list advertises emulated integrations as Chat Completions.

Reliability. Emulation depends on the model following instructions and copying the framing values accurately. Models that follow instructions well use tools consistently; smaller models, and models behind an agent wrapper, call tools less consistently than they would through a native API.

Limits

Requests that exceed these bounds are rejected before they are sent upstream.

Tool names must be 1 to 64 characters of letters, numbers, underscores, or hyphens, and must be unique within a request. Only function tools are supported. Parameter schemas that expand to a very large number of branches through $ref and anyOf are rejected before compilation.

Troubleshooting

The status line says emulated tools unexpectedly. The active model's provider is registered as emulated. Switch to a model from a native provider with /model, or ask an administrator whether the provider's mode is correct for its endpoint.

The model answers in prose instead of using a tool. This is the most common emulation failure. Stating the action directly ("read main.go and tell me what it does") usually helps, as does selecting a larger model from the same provider.

"This model does not support tool calling." The provider is registered as none. Run pw code --no-tools for a chat-only session, or choose a different model.

Calls fail repeatedly on one specific tool. Emulated arguments are validated against the tool's JSON Schema as written. A large or deeply nested schema is harder for a model to satisfy from a prompt than through a native API, so simplifying the schema usually resolves it.

Related Documentation

• Custom OpenAI-Compatible — Register an endpoint and set its tool-calling mode
• pw code — The AI coding agent that uses tool calling
• Models & Allocations — Discovering and switching models
• AI Keys — Programmatic access through the gateway