Using Reasoning Models
Reasoning models use an extended thinking process before generating a response. Chat supports these models and surfaces their reasoning output so you can follow the model's chain of thought.
What Are Reasoning Models?
Standard chat models generate responses token by token in a single pass. Reasoning models add a thinking phase before the final response. During this phase, the model works through the problem step by step, considering different approaches and verifying its logic before producing an answer.
This thinking process is especially beneficial for:
- Complex math and logic problems
- Multi-step analysis and planning
- Code debugging and architectural decisions
- Tasks that require careful evaluation of tradeoffs
How Reasoning Models Differ
When you use a reasoning model in Chat, the experience differs from standard models in several ways:
- Longer response times: The thinking phase adds processing time before the response begins streaming. This is normal and expected.
- Reasoning content: The model's reasoning may be displayed above the final response, allowing you to follow how it arrived at its answer.
- Reasoning tokens: Reasoning models consume additional tokens for the thinking phase. These reasoning tokens count toward the model's context window and usage.
Reasoning Effort
Some reasoning models support a reasoning effort setting that controls how much time the model spends thinking before responding. Chat does not have a reasoning effort control: messages sent from Chat use the model's default effort.
To set the effort yourself, send requests to the OpenAI-compatible API with the reasoning_effort parameter. The accepted levels are none, minimal, low, medium, high, xhigh, and max. Lower levels give faster responses; higher levels let the model think through the problem more thoroughly.
Model Support
Each model supports its own set of effort levels. The models endpoint (/api/openai/v1/models) lists them for each model as reasoning_efforts. A request with a level the model is known not to support is rejected with an error that lists the supported levels.
When to Use Reasoning Models
Use reasoning models when:
- The task involves complex logic, math, or multi-step problem-solving.
- You need the model to carefully evaluate multiple options before answering.
- Accuracy matters more than response speed.
- You are debugging intricate code or analyzing system architectures.
Use standard models when:
- You need quick, conversational responses.
- The task is straightforward (summarization, simple Q&A, text generation).
- Low latency is important.
- You want to minimize token usage.
Working with Reasoning Output
While a reasoning model is thinking, Chat shows Thinking… above the reply, with the latest lines of its reasoning. When the thinking phase ends, the line changes to Thought for followed by how long the model thought, and the reasoning folds away. Click the line to expand the reasoning in place, and click it again to collapse it. When Chat has no duration to show, the line reads Thought process.
This reasoning content shows the steps the model took to arrive at its answer. Reviewing this output can help you:
- Verify the model's logic and catch errors in its reasoning.
- Understand why the model chose a particular approach.
- Refine your prompt based on how the model interpreted your request.
Related Documentation
- Getting Started with Chat: First-time setup walkthrough
- Best Practices: Tips for model selection and effective prompting
- AI Providers: Configure providers that offer reasoning models