Structured output + thinking + tool use: two bugs in multi-turn conversations

[ootkin]

Environment

• models: claude-sonnet-4-5, claude-sonnet-4-6
• thinking (both extended and adaptive): {"type": "enabled", "budget_tokens": N} (also reproduced with {"type": "adaptive"} with or without specifying the effort)
• output_config: {"format": {"type": "json_schema", "schema": {...}}} or using the pydantic schema inside output_format
• Tools are present in the request input (multi-turn conversation with tool results)

Bug 1 — thinking block generated instead of tool_use blocks

In tool-calling turns, the model occasionally returns only a thinking block plus an empty text block (or no text block at all), with stop_reason: "end_turn", instead of generating tool_use blocks.

The thinking content correctly identifies which tools should be called, but no tool_use blocks follow.

Actual behavior

{
  ...
  content: [
    {"type": "thinking", "thinking": "I need to call tool_1 and tool_2 in parallel..."},
    {"type": "text", "text": ""}   // empty, no tool_use blocks
  ],
  stop_reason: "end_turn"
}

Expected behavior

If the model determines that tools must be called, it should emit one or more tool_use blocks (not end the turn with only thinking/empty text).

Impact

In this scenario, I cannot call anthropic.messages.parse, because the SDK attempts to parse an invalid/unfinished model response.

Bug 2 — final text block contains invalid JSON

Often, In the final turn (after tool results), the structured output text block contains invalid content.

Observed cases

• Case A: the text block is a mix of text and json and it is not possible to parse it.
• Case B: a malformed/partial JSON, followed by 20+ blank lines, followed by the valid JSON — both in a single text block:

{
  ...
  "content": [
    {
      "type": "text",
      // invalid json +  actual valid output, but appended in same text block
      "text": "{\"my_json\": \"is_broken \n\n\n\n\n\n\n\n\n {\"my_json\": \"is_not_broken\"}"
    }
  ]
}

Expected behavior

The final text block should contain exactly one valid JSON object matching the provided json_schema, with no duplicated or partial content.

Notes

• If I remove output_config.format, everything seems to work, but I no longer get structured JSON output.
• If I remove thinking and use messages.parse, the response sometimes contains empty content instead

minimal code

from anthropic import AsyncAnthropic
from pydantic import BaseModel
import asyncio

class Output(BaseModel):
    param1: str
    param2: str
    param3: str

client = AsyncAnthropic()

tools = []

def extract_tools_use(response):
    """Extracts the tools use block from the response"""
    return []

def execute_tools(tools_to_use):
    """Execute the tools and create the tool_result blocks"""
    return {"type": "tool_result", "tool_use_id": "", "content": "..."}


async def main():
        query = "What is the capital of France?"
        messages = [{"role": "user", "content": query}]
        
        # react loop
        for i in range(10):
            client = AsyncAnthropic()
            response = await client.messages.parse(
                model="claude-sonnet-4-6",
                max_tokens=10000,
                system="...prompt...",
                tool_choice={"type": "auto"},
                messages=[{"role": "user", "content": query}],
                tools=tools,
                thinking={"type": "adaptive"},
                output_config={"effort": "medium"},
                output_format=Output
            )
            
            tools_use = extract_tools_use(response)
            
            if len(tools_use) == 0:
                # final response in structured output
                print(response.parsed_output)
            else:
                # Append thinking and tool calls to messages for next iteration
                messages.append(response)
                
                # execute tools and get tool response
                tool_response = execute_tools(tools_use)
                
                # add tool response to messages for next iteration
                messages.append({"role": "tool", "content": tool_response})
            
        
if __name__ == "__main__":
    asyncio.run(main())

Conclusion

At the moment, the interaction between thinking, tool calling, and json_schema structured output appears to be broken or highly unstable.

• With thinking + structured output + tools, the model fails a lot to emit tool_use blocks and end the turn prematurely, or produce invalid structured output.
• With structured output + tools (without thinking), I sometimes receive empty content in the response.
• If I remove structured output and keep tools enabled, the flow appears to work more reliably, but I no longer receive output that conforms to the required schema.

In practice, this makes it difficult to reliably use thinking + tools together with structured JSON output in production.

[xXMrNidaXx]

Great bug report — I've been hitting similar issues in production with multi-turn agentic flows. A few observations that might help:

Bug 1: Thinking block without tool_use

This appears to happen more frequently when:

1. Context is borderline — the model is uncertain whether to call a tool or respond directly
2. Tool definitions have ambiguous triggering conditions — overlapping tool capabilities cause hesitation
3. Structured output schema is complex — the grammar constraint seems to interfere with tool emission

Workaround I've been using:

# After each turn, check for the premature end_turn pattern
if response.stop_reason == "end_turn" and not any(
    block.type == "tool_use" for block in response.content
):
    thinking_block = next((b for b in response.content if b.type == "thinking"), None)
    if thinking_block and ("call" in thinking_block.thinking.lower() or "tool" in thinking_block.thinking.lower()):
        # Model intended to call tools but didn't — retry with stronger prompt
        messages.append({"role": "user", "content": "Please proceed with the tool calls you described."})
        continue  # retry the turn

Not ideal, but catches ~80% of these cases.

Bug 2: Malformed JSON in final output

The "partial JSON + newlines + valid JSON" pattern suggests the model is starting to generate output, then self-correcting mid-stream. With structured outputs, the grammar constraint should prevent invalid JSON, but it seems like the constraint isn't being applied consistently across the full generation.

Patterns I've observed:

• More frequent with adaptive thinking (vs fixed budget)
• More frequent when the structured schema has optional fields with defaults
• More frequent after tool result injection mid-conversation

Hacky workaround for malformed output:

def extract_valid_json(text: str) -> dict:
    """Find the last valid JSON object in a potentially malformed string"""
    import json
    import re
    
    # Try parsing the whole thing first
    try:
        return json.loads(text.strip())
    except json.JSONDecodeError:
        pass
    
    # Look for JSON objects in the text
    candidates = re.findall(r\{[^{}]*\}|\{(?:[^{}]|\{[^{}]*\})*\}, text, re.DOTALL)
    
    # Return the last valid one
    for candidate in reversed(candidates):
        try:
            return json.loads(candidate)
        except json.JSONDecodeError:
            continue
    
    raise ValueError("No valid JSON found in response")

Root cause hypothesis

I suspect the interaction issue is in how stop_sequences and grammar constraints are handled when thinking is enabled:

1. Thinking mode may be consuming tokens from the "planning budget" that should go to tool emission
2. The grammar constraint for structured output may be interfering with tool_use block serialization
3. The adaptive thinking budget might be depleting before tool calls can be emitted

Would be great if the Anthropic team could clarify the expected behavior when all three features (thinking, tools, structured output) are combined — or document known limitations.

cc @anthropics/sdk-team

[xXMrNidaXx]

These are two distinct issues that both stem from the interaction between thinking, output_config, and tools — a tricky combination.

Bug 1 Analysis: Thinking Block Without Tool Calls

This looks like a decoding/stopping issue where the model's internal planning (in the thinking block) doesn't translate to actual tool_use generation. Possible causes:

1. Token budget exhaustion — if thinking consumes most of max_tokens, there may not be enough left for tool_use blocks
2. Conflicting constraints — output_config.format + thinking + tools creates ambiguity about what the final output should be

Workaround for Bug 1:

# Increase headroom significantly
response = await client.messages.create(
    ...,
    thinking={"type": "enabled", "budget_tokens": N},
    max_tokens=N + 4096,  # Ensure plenty of room after thinking
)

# Or: remove output_config during tool-calling turns
# Only apply structured output on the FINAL turn (after all tools complete)

Bug 2 Analysis: Malformed JSON in Final Block

The double-JSON-in-one-block issue (partial JSON + newlines + valid JSON) suggests the model is "restarting" its response mid-generation. This can happen when:

1. The model backtracks internally but doesn't fully clear its output buffer
2. Context window pressure causes generation anomalies

Workaround for Bug 2:

import re
import json

def extract_valid_json(text: str):
    # Find the LAST complete JSON object (usually the correct one)
    matches = list(re.finditer(r'\{[^{}]*\}', text, re.DOTALL))
    for match in reversed(matches):
        try:
            return json.loads(match.group())
        except json.JSONDecodeError:
            continue
    raise ValueError("No valid JSON found")

Recommended Pattern:

Separate tool-calling turns from structured output turns:

# Tool-calling phase: NO output_config
response = await client.messages.create(
    tools=tools,
    tool_choice={"type": "auto"},
    # thinking enabled
    # NO output_config here
)

# Final structured output phase: NO tools
if response.stop_reason == "end_turn":
    final = await client.messages.parse(
        output_config={"format": {"type": "json_schema", "schema": ...}},
        # NO tools here
        # thinking can stay
    )

This separation avoids the three-way conflict between thinking, tools, and structured output.

[ootkin]

Great analysis.

The core issue is that, in a ReAct loop, the termination condition is owned by the agent itself. The agent decides whether to continue the loop by issuing additional tool calls or to stop and emit the final response.

This is manageable in a single-step interaction, where execution consists of one reasoning phase followed by one final output. However, it becomes much harder to handle in an iterative setting, because the number of reasoning / acting steps is not known in advance.

In other words, the caller can trigger the loop, but it cannot reliably determine from the outside when the agent has actually completed its execution. That decision remains internal to the agent.