[RFC] 088 - Claude Thinking 集成

[arvinxx]

背景

集成 Claude 3.7 Sonnet 模型 Extended Thinking 能力：https://www.anthropic.com/news/claude-3-7-sonnet

设计思路

Thinking 集成

由于 Claude 3.7 具有 DeepSeek 类模型不具备的特殊字段，因此设计还略有不同。

Claude 对于推理的态度是一个可控的思路，提供了专门的 thinking 字段用于开启这个功能，同时提供了 budget_tokens 用于控制思考深度，比起 OpenAI 和 DeepSeek 都对开发者更友好：

const params = {
  thinking: {
    type: "enabled",
    budget_tokens: 4096
  }
};

正常来说只要开启这个配置后，Claude 就具有了 Thinking 能力，这也是 PR #6515 中做的工作。

Thinking + Tool Use

但事实上当我测试了同时开启 Thinking + Search 插件时，事情就变得没有那么简单了。默认 Thinking + 联网插件时，直接使用就会遇到一个报错：

{
  "error": {
    "type": "error",
    "error": {
      "type": "invalid_request_error",
      "message": "messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking (request id: 2025022709391110067949266409883)"
    }
  }
}

报错提到的问题点是，assistant 如果需要使用 tool calling，那么必须在 assistants 中包含 thinking 或者 redacted_thinking 块。

即构造出来的数据结构必须得是这样：

todo

这里非常值得说的就是 signature 字段，我在下面贴了中间和 Claude 3.7 聊的点，它提到了为什么需要 signature

进展

• Step 1： 完成 Thinking 配置能力集成与实现 ✨ feat: support claude sonnet 3.7 thinking #6515
• Step 2: 针对 Thinking 场景完成相关 edge case 修正
  • 🐛 fix: fix claude 3.7 sonnet thinking with tool use #6528
  • 🐛 fix: fix anthropic tools issue  #6566

[arvinxx]

litellm 示例数据：

{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505568,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"我","ignature_delta":null}],"reasoning_content":"我","content":"","role":"assistant"}}],"thinking_blocks":[{"type":"thinking","thinking":"我","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"需要找94的所有质","signature_delta":null}],"reasoning_content":"需要找出394的所有质","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"需要找出394的所有质","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"因数。\n质因数是","signature_delta":null}],"reasoning_content":"因数。\n\n质因数是","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"因数。\n\n质因数是","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"指能整数的质","signature_delta":null}],"reasoning_content":"指能整除该数的质","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"指能整除该数的质","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"数。我检查394能","signature_delta":null}],"reasoning_content":"数。我先来检查394能","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"数。我先来检查394能","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"被哪些","signature_delta":null}],"reasoning_content":"被哪些小质","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"被哪些小质","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"数整除,"signature_delta":null}],"reasoning_content":"数整除。","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"数整除。","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"\n\n先尝试2：\n394 ","signature_delta":null}],"reasoning_content":"\n\n先尝试2：\n394 ","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"\n\n先尝试2：\n394 ","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"÷ 2 = 197","signature_delta":null}],"reasoning_content":"÷ 2 = 197","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"÷ 2 = 197","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"\n197是质数吗？我","signature_delta":null}],"reasoning_content":"\n197是质数吗？我","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"\n197是质数吗？我","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"来检查n197不","signature_delta":null}],"reasoning_content":"来检查：\n197不","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"来检查：\n197不","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"能被2\n197不能被3","signature_delta":null}],"reasoning_content":"能被2整除\n197不能被3","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"能被2整除\n197不能被3","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"整除\97不能被5","signature_delta":null}],"reasoning_content":"整除\n197不能被5","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"整除\n197不能被5","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505569,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"整除\97不能被7整除","signature_delta":null}],"reasoning_content":"整除\n197不能被7整除","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"整除\n197不能被7整除","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"\n197不能被11整除\n197","signature_delta":null}],"reasoning_content":"\n197不能被11整除\n197","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"\n197不能被11整除\n197","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"不能被整除","signature_delta":null}],"reasoning_content":"不能被13整除","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"不能被13整除","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"\n\n要确定197是不是质数，我","signature_delta":null}],"reasoning_content":"\n\n要确定197是不是质数，我","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"\n\n要确定197是不是质数，我","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"需要检是否能","signature_delta":null}],"reasoning_content":"需要检查它是否能","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"需要检查它是否能","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"被不超197的质数整除。√","signature_delta":null}],"reasoning_content":"被不超过√197的质数整除。√","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"被不超过√197的质数整除。√","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"197 ≈ ","signature_delta":null}],"reasoning_content":"197 ≈ ","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"197 ≈ ","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"14，所我已经检查了","signature_delta":null}],"reasoning_content":"14，所以我已经检查了","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"14，所以我已经检查了","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505570,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"所有需质数，","signature_delta":null}],"reasoning_content":"所有需要的质数，","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"所有需要的质数，","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"197没更小的因","signature_delta":null}],"reasoning_content":"197没有更小的因","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"197没有更小的因","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"数，所97是质数。","signature_delta":null}],"reasoning_content":"数，所以197是质数。","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"数，所以197是质数。","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"\n\n因此，394的质因数是2和","signature_delta":null}],"reasoning_content":"\n\n因此，394的质因数是2和","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"\n\n因此，394的质因数是2和","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"197。n394 = 2 ","signature_delta":null}],"reasoning_content":"197。\n394 = 2 ","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"197。\n394 = 2 ","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"thinking_blocks":[{"type":"thinking","thinking":"× 197","signature_delta":null}],"reasoning_content":"× 197","content":""}}],"thinking_blocks":[{"type":"thinking","thinking":"× 197","signature_delta":null}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"","tool_calls":[{"function":{"arguments":"{}"},"type":"function","index":-1}]}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"要找出394的质因数，我需要将"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"394分解为质数的乘积"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505571,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"。\n\n我先用最"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"小的质数2"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"尝试除394：\n394 "}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"÷ 2 = 197"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n\n现在我需要判"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"断197是否为质数。\n-"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" 197不能被2整除"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n- 197不能被3整除"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n- 对于更"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"大的质数，我"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505572,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"只需检查到"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"√197 ≈ 14"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n- 5, 7, "}}]}
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"11, 13都不能整除197"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n\n因此197是质数。\n\n所"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"以394的质因数只有两"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"个：2和197"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"\n\n答案：394 = 2"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" × 197，质因数是"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"2和197。"}}]},
{"id":"chatcmpl-1c04a1f1-d58c-4008-abd7-84f043a222af","created":1740505573,"model":"claude-3-7-sonnet-latest","object":"chat.completion.chunk","choices":[{"finish_reason":"stop","index":0,"delta":{}}]}

[arvinxx]

遗留问题：在持续搜索后，达到历史记录限制带来的报错。

相关问题：

• sonnet3.7频繁调用搜索插件，而无法正常回答 #6547
• [Bug] 使用插件进行联网搜索，多次询问后报错 #5455
• [Bug] Claude 使用 function call + 开启限制历史消息数功能，报错：messages.0: tool_use blocks can only be in assistant messages #6556

这里存在的问题应该之前实现的时候没有做准整个 message 转换逻辑。需要做一下修正:

• 🐛 fix: fix anthropic tools issue  #6566

[arvinxx]

开启思考后，需要关闭历史记录限制，否则会报错：

{
  "body": {
    "error": {
      "status": 400,
      "headers": {
        "connection": "keep-alive",
        "content-length": "579",
        "content-type": "application/json; charset=utf-8",
        "date": "Thu, 27 Feb 2025 08:47:58 GMT",
        "x-aihubmix-request-id": "2025022708475795031093801986038"
      },
      "error": {
        "error": {
          "type": "invalid_request_error",
          "message": "messages.1.content.0.type: Expected `thinking` or `redacted_thinking`, but found `text`. When `thinking` is enabled, a final `assistant` message must start with a thinking block (preceeding the lastmost set of `tool_use` and `tool_result` blocks). We recommend you include thinking blocks from previous turns. To avoid this requirement, disable `thinking`. Please consult our documentation at https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking (request id: 2025022708475795031093801986038)"
        },
        "type": "error"
      }
    },
    "endpoint": "https://ai****ix.com/",
    "provider": "anthropic"
  },
  "errorType": "ProviderBizError"
}

从这个角度来考虑，感觉深度思考不能默认开启，需要手动触发，并提示用户开启深度思考后会自动取消历史记录条数限制