推理输出

vLLM 为 DeepSeek R1 等推理模型提供支持，这些模型旨在生成包含推理步骤和最终结论的输出。

推理模型会在输出中返回一个额外的 reasoning 字段，其中包含得出最终结论的推理步骤。其他模型的输出中不包含此字段。

警告

reasoning 过去被称为 reasoning_content。如需迁移，请直接将 reasoning_content 替换为 reasoning。

支持的模型

vLLM 目前支持以下推理模型：

模型系列	解析器名称	结构化输出支持	工具调用
DeepSeek R1 系列	deepseek_r1	json, regex	❌
DeepSeek-V3.1	deepseek_v3	json, regex	❌
ERNIE-4.5-VL 系列	ernie45	json, regex	❌
ERNIE-4.5-21B-A3B-Thinking	ernie45	json, regex	✅
GLM-4.5 系列	glm45	json, regex	✅
Holo2 系列	holo2	json, regex	✅
Hunyuan A13B 系列	hunyuan_a13b	json, regex	✅
IBM Granite 3.2 语言模型	granite	❌	❌
MiniMax-M2	minimax_m2_append_think	json, regex	✅
Qwen3 系列	qwen3	json, regex	✅
QwQ-32B	deepseek_r1	json, regex	✅

注意

IBM Granite 3.2 和 DeepSeek-V3.1 的推理功能默认处于禁用状态；要启用它，您必须在 chat_template_kwargs 中传入 thinking=True。Qwen3 系列的推理功能默认启用，如需禁用，必须在 chat_template_kwargs 中传入 enable_thinking=False。DeepSeek-V3.1 在非思维模式下支持工具调用。Holo2 的推理功能默认启用，如需禁用，必须在 chat_template_kwargs 中传入 thinking=False。

快速入门

要使用推理模型，需要在向聊天补全端点发送请求时指定 --reasoning-parser 标志。--reasoning-parser 标志指定了用于从模型输出中提取推理内容的推理解析器。

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
    --reasoning-parser deepseek_r1

接下来，向模型发送请求，该模型应在响应中返回推理内容。

代码

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "https://:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
response = client.chat.completions.create(model=model, messages=messages)

reasoning = response.choices[0].message.reasoning
content = response.choices[0].message.content

print("reasoning:", reasoning)
print("content:", content)

reasoning 字段包含得出最终结论的推理步骤，而 content 字段包含最终结论。

流式聊天补全

推理模型同样支持流式聊天补全。reasoning 字段可在 聊天补全响应块 的 delta 字段中获取。

Json

{
    "id": "chatcmpl-123",
    "object": "chat.completion.chunk",
    "created": 1694268190,
    "model": "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    "system_fingerprint": "fp_44709d6fcb",
    "choices": [
        {
            "index": 0,
            "delta": {
                "role": "assistant",
                "reasoning": "is",
            },
            "logprobs": null,
            "finish_reason": null
        }
    ]
}

OpenAI Python 客户端库并未官方支持流式输出的 reasoning 属性。但客户端支持响应中的额外属性。您可以使用 hasattr 来检查响应中是否存在 reasoning 属性。例如：

代码

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "https://:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
# For Qwen3 series, if you want to disable thinking in reasoning mode, add:
# extra_body={"chat_template_kwargs": {"enable_thinking": False}}
stream = client.chat.completions.create(
    model=model,
    messages=messages,
    stream=True,
)

print("client: Start streaming chat completions...")
printed_reasoning = False
printed_content = False

for chunk in stream:
    # Safely extract reasoning and content from delta,
    # defaulting to None if attributes don't exist or are empty strings
    reasoning = (
        getattr(chunk.choices[0].delta, "reasoning", None) or None
    )
    content = getattr(chunk.choices[0].delta, "content", None) or None

    if reasoning is not None:
        if not printed_reasoning:
            printed_reasoning = True
            print("reasoning:", end="", flush=True)
        print(reasoning, end="", flush=True)
    elif content is not None:
        if not printed_content:
            printed_content = True
            print("\ncontent:", end="", flush=True)
        # Extract and print the content
        print(content, end="", flush=True)

请记住在访问 reasoning 字段前检查其是否存在。您可以查看此 示例。

工具调用

当同时启用工具调用和推理解析器时，也可以获取推理内容。此外，工具调用仅解析 content 字段中的函数，不会解析 reasoning 字段。

代码

from openai import OpenAI

client = OpenAI(base_url="https://:8000/v1", api_key="dummy")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location", "unit"],
            }
        },
    }
]

response = client.chat.completions.create(
    model=client.models.list().data[0].id,
    messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
    tools=tools,
    tool_choice="auto",
)

print(response)
tool_call = response.choices[0].message.tool_calls[0].function

print(f"reasoning: {response.choices[0].message.reasoning}")
print(f"Function called: {tool_call.name}")
print(f"Arguments: {tool_call.arguments}")

更多示例，请参阅 examples/online_serving/openai_chat_completion_tool_calls_with_reasoning.py。

服务器级默认聊天模板参数

您可以使用 --default-chat-template-kwargs 命令行参数在服务器级别设置默认的 chat_template_kwargs。这对于在所有请求中配置推理行为非常有用，无需客户端在每个请求中重复指定。

默认禁用思维模式

对于像 Qwen3 这样默认启用思维模式的模型，您可以全服务器禁用它：

vllm serve Qwen/Qwen3-8B \
    --reasoning-parser qwen3 \
    --default-chat-template-kwargs '{"enable_thinking": false}'

默认启用思维模式

对于像 IBM Granite 3.2 或 DeepSeek-V3.1 这样默认禁用思维模式的模型，您可以全服务器启用它：

vllm serve ibm-granite/granite-3.2-2b-instruct \
    --reasoning-parser granite \
    --default-chat-template-kwargs '{"thinking": true}'

请求级覆盖

请求级的 chat_template_kwargs 总是优先于服务器默认设置。例如，如果服务器启动时使用了 enable_thinking=false，客户端仍然可以在特定请求中启用它：

response = client.chat.completions.create(
    model=model,
    messages=messages,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}}  # Overrides server default
)

推理预算控制

某些模型，如 Qwen3, DeepSeek 和 Nemotron3，支持推理预算，用于限制推理所使用的最大标记 (token) 数量。

标记计数从 reasoning_start_str 开始。一旦推理标记数量达到配置的 thinking_token_budget，vLLM 会强制模型生成 reasoning_end_str，从而有效地终止推理块。

要使用此功能：

• --reasoning-parser 启用推理提取。
• --reasoning-config 定义推理边界标记（例如 reasoning_start_str, reasoning_end_str）。如果未设置，vLLM 将尝试从推理解析器中自动初始化这些标记。
• thinking_token_budget（采样参数）设置每个请求的推理标记限制。

如果未指定 thinking_token_budget，则除了常规生成约束（如 max_tokens）外，不会应用显式的推理限制。

--reasoning-config 接受一个 JSON 对象，对应于：
ReasoningConfig，包含以下字段：

字段	类型	描述
reasoning_start_str	str | null	标记推理内容开始的字符串
reasoning_end_str	str | null	标记推理内容结束的字符串

注意

reasoning_end_str 可以包含推理结束标记前的过渡短语。例如，将 reasoning_end_str 设置为 "I have to give the solution based on the reasoning directly now.</think>" 会指示模型在预算耗尽时发出该短语，使推理终止过程更加自然。

在线服务

vllm serve Qwen/Qwen3-0.6B \
    --reasoning-parser qwen3 \
    --reasoning-config '{"reasoning_start_str": "<think>", "reasoning_end_str": "I have to give the solution based on the reasoning directly now.</think>"}'

然后，发送带有 thinking_token_budget 的请求来限制推理标记数量：

curl https://:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [
      { "role": "user", "content": "9.11 and 9.8, which is greater?" }
    ],
    "extra_body": {
      "thinking_token_budget": 10
    }
  }'

离线推理

from vllm import LLM, SamplingParams
from vllm.config import ReasoningConfig

llm = LLM(
    model="Qwen/Qwen3-0.6B",
    reasoning_config=ReasoningConfig(
        reasoning_start_str="<think>",
        reasoning_end_str="I have to give the solution based on the thinking directly now.</think>",
    ),
)

sampling_params = SamplingParams(thinking_token_budget=10)

messages = [
    {"role": "user", "content": "9.11 and 9.8, which is greater?"},
]

outputs = llm.chat(messages, sampling_params=sampling_params)

for output in outputs:
    print("text:", output.outputs[0].text)

限制

• 推理内容仅适用于在线服务的聊天补全端点 (/v1/chat/completions)。

如何支持新的推理模型

您可以参考 vllm/reasoning/deepseek_r1_reasoning_parser.py 添加一个新的 ReasoningParser。

代码

# import the required packages

from vllm.reasoning import ReasoningParser, ReasoningParserManager
from vllm.entrypoints.openai.chat_completion.protocol import ChatCompletionRequest
from vllm.entrypoints.openai.engine.protocol import DeltaMessage

# define a reasoning parser and register it to vllm
# the name list in register_module can be used
# in --reasoning-parser.
class ExampleParser(ReasoningParser):
    def __init__(self, tokenizer: TokenizerLike):
        super().__init__(tokenizer)

    def extract_reasoning_streaming(
        self,
        previous_text: str,
        current_text: str,
        delta_text: str,
        previous_token_ids: Sequence[int],
        current_token_ids: Sequence[int],
        delta_token_ids: Sequence[int],
    ) -> DeltaMessage | None:
        """
        Instance method that should be implemented for extracting reasoning
        from an incomplete response; for use when handling reasoning calls and
        streaming. Has to be an instance method because  it requires state -
        the current tokens/diffs, but also the information about what has
        previously been parsed and extracted (see constructor)
        """

    def extract_reasoning(
        self,
        model_output: str,
        request: ChatCompletionRequest | ResponsesRequest,
    ) -> tuple[str | None, str | None]:
        """
        Extract reasoning content from a complete model-generated string.

        Used for non-streaming responses where we have the entire model response
        available before sending to the client.

        Parameters:
        model_output: str
            The model-generated string to extract reasoning content from.

        request: ChatCompletionRequest
            The request object that was used to generate the model_output.

        Returns:
        tuple[Optional[str], Optional[str]]
            A tuple containing the reasoning content and the content.
        """
# Register the reasoning parser
ReasoningParserManager.register_lazy_module(
    name="example",
    module_path="vllm.reasoning.example_reasoning_parser",
    class_name="ExampleParser",
)

此外，若要启用结构化输出，您需要创建一个新的 Reasoner，类似于 vllm/reasoning/deepseek_r1_reasoning_parser.py 中的实现。

代码

@dataclass
class DeepSeekReasoner(Reasoner):
    """
    Reasoner for DeepSeek R series models.
    """
    start_token_id: int
    end_token_id: int

    start_token: str = "<think>"
    end_token: str = "</think>"

    @classmethod
    def from_tokenizer(cls, tokenizer: PreTrainedTokenizer) -> Reasoner:
        return cls(
            start_token_id=tokenizer.encode("<think>", add_special_tokens=False)[0],
            end_token_id=tokenizer.encode("</think>", add_special_tokens=False)[0],
        )

    def is_reasoning_end(self, input_ids: list[int]) -> bool:
        return self.end_token_id in input_ids

    def is_reasoning_end_streaming(self, input_ids: list[int], delta_ids: list[int]) -> bool:
        return self.end_token_id in delta_token_ids
    ...

结构化输出引擎（如 xgrammar）将使用 end_token_id 来检查模型输出中是否存在推理内容，如果存在，则跳过结构化输出。

最后，您可以使用 --reasoning-parser 标志为模型启用推理功能。

vllm serve <model_tag> --reasoning-parser example