跳到内容
vLLM
Meetups
正在初始化搜索
GitHub
主页
用户指南
开发者指南
基准测试
API 参考
CLI 参考
社区
vLLM
GitHub
主页
用户指南
用户指南
入门
入门
快速入门
安装
安装
GPU
CPU
TPU
示例
示例
应用
应用
API 服务器
聊天机器人
RAG
基础
基础
离线推理
在线服务
部署
部署
异步 LLM 流式传输
Helm Charts
LLM引擎示例
Sagemaker-Entrypoint
解耦
解耦
解耦式编码器
解耦服务
Ec Both 编码器
解耦式 Prefill V1
Flexkv 连接器
KV 加载失败恢复测试
LMCache 示例
Mooncake 连接器
功能
功能
自动前缀缓存
批处理不变性
上下文扩展
数据并行
KV 事件
日志配置
自定义 Logits 处理器
LoRA
使用 OpenAI 批量文件格式进行离线推理
暂停/恢复
性能分析
提示词嵌入
重置 KV
分片状态
推测解码
结构化输出
张量化 vLLM 模型
Torchrun
生成
生成
在线批量聊天补全
多模态
Qwen 1M 离线
可观测性
可观测性
监控仪表盘
指标
设置 OpenTelemetry POC
Prometheus 和 Grafana
池化
池化
分类
嵌入
插件
奖励
评分
令牌分类
令牌嵌入
Ray Serving
Ray Serving
批量LLM推理
弹性端点
多节点服务
Ray Serve Deepseek
运行集群
推理
推理
带推理功能的 OpenAI 聊天补全工具调用
带推理功能的 OpenAI 聊天补全
带推理功能的 OpenAI 聊天补全流式传输
OpenAI 响应客户端
强化学习 (RL)
强化学习 (RL)
RLHF 异步新 API
RLHF HTTP IPC
RLHF HTTP NCCL
RLHF IPC
RLHF IPC Fsdp Ep
RLHF NCCL
RLHF NCCL Fsdp Ep
RLHF 稀疏 NCCL
路由专家模型端到端 (Routed Experts E2E)
引擎初始化时跳过加载权重
横向扩展
横向扩展
初始化
示例 Mm 服务
令牌生成客户端
语音转文本
语音转文本
Lid
OpenAI
实时
工具调用
工具调用
离线工具调用对话
带工具的 OpenAI 聊天补全客户端
带所需工具的 OpenAI 聊天补全客户端
带工具 Xlam 的 OpenAI 聊天补全客户端
带工具 Xlam 流式传输的 OpenAI 聊天补全客户端
带 Mcp 工具的 OpenAI 响应客户端
带工具的 OpenAI 响应客户端
通用
通用
vLLM V1
常见问题
生产指标
可复现性
安全
故障排除
使用统计收集
推理与服务
推理与服务
离线推理
在线服务
在线服务
生成式评分
兼容 OpenAI 的服务器
渲染器 API
语音转文本 API
上下文并行部署
数据并行部署
分布式部署故障排查
专家并行部署
并行与扩展
集成
集成
Claude 代码
Codex
LangChain
LlamaIndex
部署
部署
使用 Docker
使用 Kubernetes
使用 Nginx
框架
框架
Anyscale
AnythingLLM
AutoGen
BentoML
Cerebrium
Chatbox
Dify
dstack
Haystack
Helm
Hugging Face 推理端点
LiteLLM
Lobe Chat
LWS
Modal
Open WebUI
检索增强生成
RunPod
SkyPilot
Streamlit
NVIDIA Triton
集成
集成
AIBrix
NVIDIA Dynamo
KAITO
KServe
Kthena
KubeAI
KubeRay
Llama Stack
llm-d
llmaz
生产堆栈
训练
训练
异步强化学习
什么是逐层(重新)加载?
人类反馈强化学习
Transformer 强化学习
权重迁移
权重迁移
基类与自定义引擎
IPC 引擎
NCCL 引擎
配置
配置
内存节约
引擎参数
环境变量
模型解析
优化与调优
服务器参数
TPU
模型
模型
支持的模型
生成模型
池化模型
池化模型
分类用法
嵌入用法
奖励模型用法
评分用法
特定模型示例
Token 分类用法
Token 嵌入用法
扩展
扩展
使用 fastsafetensors 加载模型权重
使用 InstantTensor 加载模型权重
使用 Run:ai Model Streamer 加载模型
使用 CoreWeave 的 Tensorizer 加载模型
硬件支持的模型
硬件支持的模型
CPU - 英特尔® 至强®
XPU - 英特尔® GPU
TPU
特性
功能
自动前缀缓存
批处理不变性
上下文扩展
自定义参数
自定义 Logits 处理器
解耦式编码器
解耦预填充(实验性)
IndexCache
交错式思考
KV 卸载使用指南
LoRA 适配器
MooncakeConnector 使用指南
MooncakeStoreConnector 使用指南
MoRIIOConnector 使用指南
多模态输入
NixlConnector 兼容性矩阵
NixlConnector 使用指南
按请求指标
提示词嵌入输入
推理输出
睡眠模式
结构化输出
工具调用
量化
量化
AutoAWQ
BitsAndBytes
FP8 ViT 编码器注意力
GGUF
GPTQModel
Intel 量化支持
NVIDIA 模型优化器
在线量化
量化 KV 缓存
AMD Quark
TorchAO
LLM 压缩器
LLM 压缩器
FP8 W8A8
INT4 W4A16
INT8 W4A8
INT8 W8A8
推测解码
推测解码 (Speculative Decoding)
草稿模型
动态推测解码
EAGLE 草稿模型
隐藏状态提取
MLP 草稿模型
MTP (多 Token 预测)
N-Gram 推测
并行草稿模型
vLLM-Project/Speculators
后缀解码
开发者指南
开发者指南
通用
通用
弃用策略
Dockerfile
编辑 Agent 指令
增量编译工作流程
对 vLLM 进行性能分析
漏洞管理
模型实现
模型实现
基本模型
注册模型
单元测试
多模态支持
语音转文本(转录/翻译)支持
CI
持续集成 (CI)
CI 失败
vLLM 轮子 (Wheels) 的夜间构建版
更新 vLLM OSS CI/CD 上的 PyTorch 版本
设计文档
设计文档
插件
插件
端点插件
IO 处理器插件
LoRA 解析器插件
插件系统
架构概述
Attention 后端特性支持
CUDA 图
视觉编码器 (ViT) CUDA 图
CustomOp
双批次重叠
如何调试 vLLM-torch.compile 集成
融合 MoE 模块化内核
融合 torch.compile 传递
与 Hugging Face 集成
混合 KV 缓存管理器
Logits 处理器
指标
多模态数据处理
模型运行器 V2 设计文档
融合 MoE 内核特性
Python 多进程
NIXL KV 缓存租约续期
NIXL 推送模式 KV 传输
优化级别
Paged Attention
自动前缀缓存
torch.compile 集成
torch.compile 与多模态编码器
vLLM IR:函数式中间表示
基准测试
基准测试
基准测试 CLI
参数扫描
性能仪表盘
API 参考
API 参考
vllm
vllm
collect_env
connections
env_override
envs
exceptions
forward_context
logger
logits_process
logprobs
model_inspection
outputs
pooling_params
sampling_params
scalar_type
scripts
sequence
tasks
version
assets
assets
audio
base
image
video
benchmarks
benchmarks
latency
mm_processor
plot
serve
startup
throughput
datasets
datasets
create_txt_slices_dataset
datasets
utils
lib
lib
endpoint_request_func
ready_checker
utils
sweep
sweep
cli
param_sweep
plot
plot_pareto
serve
serve_workload
server
startup
utils
compilation
compilation
backends
base_static_graph
breakable_cudagraph
caching
codegen
compiler_interface
counter
cuda_graph
decorators
monitor
partition_rules
piecewise_backend
wrapper
passes
passes
fx_utils
inductor_pass
pass_manager
vllm_inductor_pass
fusion
fusion
act_quant_fusion
allreduce_rms_fusion
attn_quant_fusion
collective_fusion
matcher_utils
mla_attn_quant_fusion
mla_rope_kvcache_cat_fusion
qk_norm_rope_fusion
rms_quant_fusion
rocm_aiter_fusion
rope_kvcache_fusion
sequence_parallelism
ir
ir
clone_elimination
inplace_functionalization
lowering_pass
utils
utility
utility
fix_functionalization
noop_elimination
post_cleanup
scatter_split_replace
split_coalescing
config
config
attention
cache
compilation
device
diffusion
ec_transfer
kernel
kv_events
kv_transfer
load
lora
mamba
model
model_arch
multimodal
observability
卸载
parallel
pooler
profiler
quantization
reasoning
scheduler
speculative
speech_to_text
structured_outputs
utils
vllm
weight_transfer
cute_utils
cute_utils
cvt
device_allocator
device_allocator
cumem
sleep_mode_backend
xpumem
distributed
distributed
communication_op
kv_events
nixl_utils
parallel_state
stateless_coordinator
utils
device_communicators
device_communicators
aiter_custom_all_reduce
all2all
all_reduce_utils
base_device_communicator
cpu_communicator
cuda_communicator
cuda_wrapper
custom_all_reduce
flashinfer_all_reduce
mnnvl_compat
pynccl
pynccl_allocator
pynccl_wrapper
quick_all_reduce
ray_communicator
shm_broadcast
shm_object_storage
symm_mem
xpu_communicator
ec_transfer
ec_transfer
ec_transfer_state
ec_connector
ec_connector
base
example_connector
factory
cpu
cpu
common
connector
ec_shared_region
scheduler
调度器
embedding_cache
step_tracker
worker
worker
descriptor_buffers
elastic_ep
elastic_ep
elastic_execute
elastic_state
standby_state
eplb
eplb
async_worker
eplb_communicator
eplb_state
eplb_utils
rebalance_execute
policy
policy
abstract
default
kv_transfer
kv_transfer
kv_transfer_state
kv_connector
kv_connector
base
factory
utils
v1
v1
base
decode_bench_connector
example_connector
example_hidden_states_connector
flexkv_connector
lmcache_connector
lmcache_mp_connector
metrics
multi_connector
offloading_connector
simple_cpu_offload_connector
ssm_conv_transfer_utils
hf3fs
hf3fs
hf3fs_client
hf3fs_connector
hf3fs_metadata_server
utils
utils
common
gather_scatter_helper
hf3fs_mock_client
lmcache_integration
lmcache_integration
multi_process_adapter
utils
vllm_v1_adapter
mooncake
mooncake
mooncake_connector
mooncake_utils
rdma_utils
stats
store
store
connector
coordinator
data
metrics
protocol
scheduler
worker
moriio
moriio
moriio_common
moriio_connector
moriio_engine
moriio_layout
nixl
nixl
base_scheduler
base_worker
connector
metadata
pull_scheduler
pull_worker
push_scheduler
push_worker
scheduler
stats
tp_mapping
utils
worker
offloading
offloading
common
events
metrics
scheduler
worker
weight_transfer
weight_transfer
base
factory
ipc_engine
nccl_common
nccl_engine
packed_tensor
sparse_nccl_engine
engine
engine
arg_utils
async_llm_engine
llm_engine
protocol
entrypoints
entrypoints
chat_utils
grpc_server
launcher
llm
offline_utils
anthropic
anthropic
api_router
protocol
serving
cli
cli
collect_env
launch
main
openai
run_batch
serve
types
benchmark
benchmark
base
latency
main
mm_processor
serve
startup
sweep
throughput
generate
generate
api_router
factories
base
base
serving
beam_search
beam_search
offline
online
utils
generative_scoring
generative_scoring
api_router
serving
mcp
mcp
tool
tool_server
openai
openai
api_server
cli_args
dp_supervisor
run_batch
chat_completion
chat_completion
api_router
batch_serving
protocol
serving
completion
completion
api_router
protocol
serving
engine
engine
protocol
models
models
api_router
protocol
serving
parser
parser
harmony_utils
responses
responses
api_router
context
harmony
protocol
serving
streaming_events
utils
pooling
pooling
factories
offline
typing
utils
base
base
io_processor
protocol
serving
classify
classify
api_router
io_processor
protocol
serving
embed
embed
api_router
io_processor
protocol
serving
pooling
pooling
api_router
io_processor
protocol
serving
scoring
scoring
api_router
io_processor
protocol
serving
typing
utils
scale_out
scale_out
factories
derender
derender
api_router
serving
render
render
api_router
serving
token_in_token_out
token_in_token_out
api_router
mm_serde
protocol
serving
serve
serve
dev
dev
cache
cache
api_router
rlhf
rlhf
api_router
rpc
rpc
api_router
server_info
server_info
api_router
sleep
sleep
api_router
elastic_ep
elastic_ep
api_router
middleware
engine
engine
serving
typing
instrumentator
instrumentator
basic
health
metrics
offline_docs
lora
lora
api_router
protocol
profile
profile
api_router
sagemaker
sagemaker
api_router
tokenize
tokenize
api_router
protocol
serving
utils
utils
api_utils
constants
error_response
fingerprint
orca_metrics
request_logger
server_utils
ssl
tool_calls_utils
speech_to_text
speech_to_text
factories
base
base
protocol
serving
utils
realtime
realtime
api_router
connection
metrics
protocol
serving
transcription
transcription
api_router
protocol
serving
translation
translation
api_router
protocol
serving
inputs
inputs
engine
llm
preprocess
ir
ir
op
tolerances
util
ops
ops
layernorm
kernels
kernels
aiter_ops
oink_ops
vllm_c
xpu_ops
helion
helion
case_key
config_manager
register
utils
ops
ops
dynamic_per_token_scaled_fp8_quant
fused_qk_norm_rope
per_token_group_fp8_quant
rms_norm_dynamic_per_token_quant
rms_norm_per_block_quant
silu_and_mul_per_block_quant
silu_mul_fp8
triton
triton
qkv_padded_fp8_quant
logging_utils
logging_utils
access_log_filter
dump_input
formatter
lazy
log_time
torch_tensor
lora
lora
lora_model
lora_weights
model_manager
peft_helper
request
resolver
utils
worker_manager
layers
layers
base
base_linear
column_parallel_linear
fused_moe
logits_processor
replicated_linear
row_parallel_linear
utils
vocal_parallel_embedding
ops
ops
torch_ops
torch_ops
lora_ops
triton_ops
triton_ops
fp8_kernel_utils
fused_moe_lora_fp8_op
fused_moe_lora_op
kernel_utils
lora_expand_fp8_op
lora_expand_op
lora_kernel_metadata
lora_shrink_fp8_op
lora_shrink_op
utils
xpu_ops
xpu_ops
lora_ops
punica_wrapper
punica_wrapper
punica_base
punica_cpu
punica_gpu
punica_selector
punica_xpu
utils
model_executor
model_executor
custom_op
parameter
utils
kernels
kernels
attention
attention
dsa
dsa
dcp_indexer_cutedsl
linear
linear
base
zentorch_utils
cute_dsl
cute_dsl
ll_bf16
mixed_precision
mixed_precision
allspark
conch
cpu
cutlass
dynamic_4bit
exllama
humming
MPLinearKernel
machete
marlin
rdna3_w4a16
rdna_hybrid_w4a16
triton_w4a16
xpu
zentorch
mxfp4
mxfp4
base
flashinfer
humming
marlin
xpu
mxfp8
mxfp8
emulation
flashinfer
humming
Mxfp8LinearKernel
marlin
rocm_native
xpu
nvfp4
nvfp4
base
cutlass
emulation
fbgemm
flashinfer
humming
marlin
scaled_mm
scaled_mm
aiter
BlockScaledMMLinearKernel
cpu
cutlass
deep_gemm
flashinfer
humming
marlin
pytorch
rocm
ScaledMMLinearKernel
triton
xpu
zentorch
mhc
mhc
aiter
tilelang
tilelang_kernels
torch
triton
layers
layers
activation
attention_layer_base
batch_invariant
conv
fused_allreduce_gemma_rms_norm
fused_qk_norm_rope
layernorm
lightning_attn
linear
logits_processor
mhc
mla
resampler
sparse_attn_indexer
utils
vocab_parallel_embedding
attention
attention
attention
chunked_local_attention
cross_attention
encoder_only_attention
kv_transfer_utils
mla_attention
mm_encoder_attention
prefill_prefix_lm_attention
rswa_attention
sparse_mla_attention
static_sink_attention
fla
fla
ops
ops
chunk
chunk_delta_h
chunk_o
chunk_scaled_dot_kkt
cumsum
fused_gdn_prefill_post_conv
fused_recurrent
fused_sigmoid_gating
index
kda
l2norm
layernorm_guard
op
solve_tril
utils
wy_fast
fused_moe
fused_moe
activation
all2all_utils
config
cpu_fused_moe
deep_gemm_utils
eep_reconfigure
expert_map_manager
fused_flydsl_moe
fused_moe
fused_moe_method_base
fused_moe_modular_method
hpc_moe
layer
modular_kernel
moe_align_block_size
moe_fused_mul_sum
moe_permute_unpermute
routed_experts
routed_experts_capturer
topk_weight_and_reduce
unquantized_fused_moe_method
utils
experts
experts
aiter_mxfp4_w4a8_moe
aiter_mxfp8_moe
batched_deep_gemm_moe
cpu_int4_moe
cpu_moe
cutlass_moe
deep_gemm_moe
fallback
flashinfer_b12x_moe
flashinfer_cutedsl_batched_moe
flashinfer_cutedsl_moe
flashinfer_cutlass_moe
fused_batched_moe
fused_humming_moe
gpt_oss_triton_kernels_moe
lora_context
lora_experts_mixin
marlin_moe
mxfp8_emulation_moe
mxfp8_native_moe
nvfp4_emulation_moe
ocp_mx_emulation_moe
rocm_aiter_moe
triton_cutlass_moe
triton_deep_gemm_moe
triton_moe
trtllm_bf16_moe
trtllm_fp8_moe
trtllm_mxfp4_moe
trtllm_mxint4_moe
trtllm_nvfp4_moe
xpu_moe
oracle
oracle
base
fp8
int8
int_wna16
mxfp4
mxfp8
nvfp4
unquantized
w4a8
w4a8_int8
prepare_finalize
prepare_finalize
batched
deepep_ht
deepep_ll
deepep_v2
flashinfer_nvlink_one_sided
flashinfer_nvlink_two_sided
mori
naive_dp_ep
nixl_ep
no_dp_ep
router
router
aiter_shared_routed_fused_moe_router
base_router
custom_routing_router
fused_moe_router
fused_topk_bias_router
fused_topk_router
gate_linear
grouped_topk_router
router_factory
routing_simulator_router
zero_expert_router
runner
runner
moe_runner
moe_runner_interface
shared_experts
fusion
fusion
quant_activation
hpc
hpc
hpc_module
rope_norm
mamba
mamba
abstract
mamba_mixer
mamba_mixer2
mamba_utils
short_conv
gdn
gdn
base
kimi_gdn_linear_attn
olmo_gdn_linear_attn
qwen_gdn_linear_attn
linear
linear
bailing_linear_attn
base
minimax_linear_attn
ops
ops
causal_conv1d
layernorm_gated
mamba_ssm
ssd_bmm
ssd_chunk_scan
ssd_chunk_state
ssd_combined
ssd_state_passing
ssu_dispatch
triton_helpers
cpu
cpu
causal_conv1d
gdn_attention
gdn_chunk_cutedsl
gdn_chunk_cutedsl
kernel_h
kernel_kkt_inv_uw
kernel_o
minimax_rms_norm
minimax_rms_norm
lamport_workspace
rms_norm_tp
pooler
pooler
abstract
activations
common
special
seqwise
seqwise
heads
methods
poolers
tokwise
tokwise
heads
methods
poolers
quantization
quantization
auto_awq
auto_gptq
awq_triton
base_config
bitsandbytes
experts_int8
fbgemm_fp8
fp8
fp_quant
humming
input_quant_fp8
kv_cache
modelopt
moe_wna16
mxfp4
qutlass_utils
torchao
compressed_tensors
compressed_tensors
compressed_tensors
compressed_tensors_embedding
triton_scaled_mm
utils
compressed_tensors_moe
compressed_tensors_moe
compressed_tensors_moe
compressed_tensors_moe_w4a4_mxfp4
compressed_tensors_moe_w4a4_nvfp4
compressed_tensors_moe_w4a8_fp8
compressed_tensors_moe_w4a8_int8
compressed_tensors_moe_w4a16_flydsl
compressed_tensors_moe_w8a8_fp8
compressed_tensors_moe_w8a8_int8
compressed_tensors_moe_w8a8_mxfp8
compressed_tensors_moe_wna16
compressed_tensors_moe_wna16_marlin
compressed_tensors_moe_wna16_rdna3
rocm_moe_rdna
schemes
schemes
compressed_tensors_scheme
compressed_tensors_w4a4_mxfp4
compressed_tensors_w4a4_nvfp4
compressed_tensors_w4a8_fp8
compressed_tensors_w4a8_int
compressed_tensors_w8a8_fp8
compressed_tensors_w8a8_int8
compressed_tensors_w8a8_mxfp8
compressed_tensors_w8a16_fp8
compressed_tensors_wNa8o8
compressed_tensors_wNa16
transform
transform
linear
module
utils
schemes
schemes
linear_qutlass_nvfp4
inc
inc
config_parser
inc
inc_linear
schemes
schemes
factory
inc_ark_ops
inc_scheme
inc_wna16_linear
inc_wna16_scheme
online
online
base
fp8
int8
moe_base
mxfp8
quark
quark
quark
quark_moe
utils
schemes
schemes
quark_nvfp4
quark_ocp_mx
quark_scheme
quark_w4a8_mxfp4_fp8
quark_w8a8_fp8
quark_w8a8_int8
turboquant
turboquant
centroids
config
utils
utils
allspark_utils
flashinfer_fp4_moe
flashinfer_mxint4_moe
flashinfer_utils
fp8_utils
gptq_utils
humming_utils
int8_utils
layer_utils
machete_utils
marlin_utils
marlin_utils_fp4
marlin_utils_fp8
marlin_utils_test
mxfp4_utils
mxfp6_utils
mxfp8_utils
nvfp4_emulation_utils
nvfp4_utils
ocp_mx_utils
quant_utils
w8a8_utils
rotary_embedding
rotary_embedding
base
common
deepseek_scaling_rope
dual_chunk_rope
dynamic_ntk_alpha_rope
dynamic_ntk_scaling_rope
ernie45_vl_rope
fope
gemma4_rope
linear_scaling_rope
llama3_rope
llama4_vision_rope
mrope
mrope_interleaved
ntk_scaling_rope
phi3_long_rope_scaled_rope
telechat3_scaling_rope
xdrope
yarn_scaling_rope
model_loader
model_loader
base_loader
bitsandbytes_loader
default_loader
dummy_loader
ep_weight_filter
modelexpress_loader
runai_streamer_loader
sharded_state_loader
tensorizer
tensorizer_loader
utils
weight_utils
reload
reload
layerwise
meta
sanitize
torchao_decorator
types
utils
models
models
AXK1
adapters
afmoe
aimv2
apertus
arcee
arctic
aria
audioflamingo3
bagel
bailing_moe
bailing_moe_linear
bailing_moe_mtp
bee
bert
bert_with_rope
blip
blip2
bloom
chameleon
chatglm
cheers
clip
cohere2_moe
cohere2_vision
cohere_asr
cohere_eagle
colbert
colmodernvbert
colpali
colqwen3
colqwen3_5
commandr
config
conformer_encoder
cosmos3
cosmos3_edge
dbrx
deepencoder
deepencoder2
deepseek_eagle
deepseek_eagle3
deepseek_mtp
deepseek_ocr
deepseek_ocr2
deepseek_v2
deepseek_vl2
diffusion_gemma
dots_ocr
eagle2_5_vl
ernie45
ernie45_moe
ernie45_vl
ernie45_vl_moe
ernie_mtp
exaone
exaone4
exaone4_5
exaone4_5_mtp
exaone_moe
exaone_moe_mtp
extract_hidden_states
fairseq2_llama
falcon
falcon_h1
fireredasr2
fireredlid
flex_olmo
funasr
funaudiochat
gemma
gemma2
gemma3
gemma3_mm
gemma3n
gemma3n_audio_utils
gemma3n_mm
gemma4
gemma4_mm
gemma4_mtp
gemma4_unified
glm
glm4
glm4_1v
glm4_moe
glm4_moe_lite
glm4_moe_lite_mtp
glm4_moe_mtp
glm4v
glm_ocr
glm_ocr_mtp
glmasr
glmasr_utils
gpt2
gpt_j
gpt_neox
gpt_oss
granite
granite4_vision
granite_speech
granite_speech_plus
granitemoe
granitemoehybrid
granitemoeshared
gritlm
h2ovl
hrm_text
hunyuan_v1
hunyuan_vision
hy_v3
hy_v3_mtp
hyperclovax
hyperclovax_vision
hyperclovax_vision_v2
idefics2_vision_model
idefics3
interfaces
interfaces_base
intern_vit
internlm2
interns1
interns1_pro
interns1_vit
interns2_preview
internvl
iquest_loopcoder
isaac
jais2
jamba
jina
jina_vl
kanana_v
keye
keye_vl1_5
kimi_audio
kimi_k25
kimi_k25_vit
kimi_linear
kimi_vl
laguna
laguna_dflash
lfm2
lfm2_moe
lfm2_siglip2
lfm2_vl
lightonocr
llama
llama4
llama4_eagle
llama_eagle
llama_eagle3
llava
llava_next
llava_next_video
llava_onevision
llava_onevision2
longcat_flash
longcat_flash_mtp
longcat_flash_ngram
mamba
mamba2
medusa
mellum
midashenglm
mimo
mimo_audio
mimo_mtp
mimo_v2
mimo_v2_mtp
mimo_v2_omni
minicpm
minicpm3
minicpm_eagle
minicpmo
minicpmv
minicpmv4_6
minimax_m2
mistral
mistral3
mistral_eagle
mistral_large_3
mistral_large_3_eagle
mixtral
mllama4
mlp_speculator
modernbert
module_mapping
molmo
molmo2
moondream3
moonvit
moss_audio
moss_transcribe_diarize
mpt
nano_nemotron_vl
nemotron
nemotron_h
nemotron_h_mtp
nemotron_nas
nemotron_parse
nemotron_vl
nvlm_d
olmo3
olmo_hybrid
olmoe
openai_privacy_filter
opencua
openpangu
openpangu_mtp
openpangu_vl
openvla
opt
orion
ouro
ovis
ovis2_5
paddleocr_vl
paligemma
parakeet
param2moe
phi
phi3
phi3v
phi4mm
phi4mm_audio
phi4mm_utils
phi4siglip
phimoe
pixtral
plamo2
plamo3
qianfan_ocr
qwen2
qwen2_5_omni_thinker
qwen2_5_vl
qwen2_audio
qwen2_moe
qwen2_rm
qwen2_vl
qwen3
qwen3_5
qwen3_5_mtp
qwen3_asr
qwen3_asr_forced_aligner
qwen3_asr_realtime
qwen3_dflash
qwen3_dspark
qwen3_eagle3
qwen3_moe
qwen3_next
qwen3_next_mtp
qwen3_omni_moe_thinker
qwen3_vl
qwen3_vl_moe
radio
registry
rnj1
roberta
rvl
sarvam
seed_oss
siglip
siglip2navit
skyworkr1v
smolvlm
solar
stablelm
step1
step3_text
step3_vl
step3p5
step3p5_mtp
step3p7
step_vl
telechat2
teleflm
terratorch
ultravox
unlimited_ocr
utils
vision
voxtral
voxtral_realtime
voyage
whisper
whisper_causal
whisper_utils
zamba2
transformers
transformers
base
causal
fuser
fx_utils
legacy
moe
multimodal
pooling
utils
fusers
fusers
base
glu
moe
qkv
rms_norm
offloader
offloader
base
prefetch
prefetch_ops
uva
warmup
warmup
cutedsl_warmup
deep_gemm_warmup
deepseek_v4_mhc_warmup
fa4_cutedsl_config
flashinfer_autotune_cache
flashinfer_sparse_mla_warmup
kernel_warmup
minimax_m3_msa_warmup
qwen_triton_warmup
sparse_mla_triton_warmup
v1_block_table_warmup
models
models
deepseek_v4
deepseek_v4
attention
compressor
quant_config
sparse_mla
amd
amd
dspark
model
mtp
rocm
common
common
rope
ops
ops
cache_utils
fused_compress_quant_cache
fused_indexer_q
fused_inv_rope_fp8_quant
fused_mtp_input_rmsnorm
fused_qk_rmsnorm
save_partial_states
nvidia
nvidia
dspark
flashinfer_sparse
flashmla
model
mtp
ops
ops
dequant_gather_k_cutedsl
fused_indexer_q_cutedsl
o_proj
prepare_megamoe
sparse_attn_compress_cutedsl
xpu
xpu
model
mtp
xpu_qnorm_rope_kv_fp8_insert
xpu_sparse
xpu_sparse_decode_fp8
deepseek_v32
deepseek_v32
nvidia
nvidia
attention
fused_ops
kernels
model
mtp
minimax_m3
minimax_m3
amd
amd
model
mtp
sparse_attention_msa
ops
ops
gemma_rmsnorm
index_topk
sparse_attn
sparse_pa
swiglu_oai
common
common
indexer
mm_preprocess
sparse_attention
vision_tower
ops
ops
index_topk
sparse_attn
nvidia
nvidia
indexer_msa
model
mtp
sparse_attention_msa
multimodal
multimodal
audio
cache
encoder_budget
evs
gpu_ipc_memory
hasher
image
inputs
parse
registry
utils
video
media
media
audio
base
connector
image
video
processing
processing
context
dummy_inputs
inputs
processor
parser
parser
abstract_parser
deepseek_v4
deepseek_v32
gemma4
glm47_moe
harmony
kimi_k2
metrics
minimax_m2
mistral
nemotron_v3
parser_manager
qwen3
seed_oss
utils
engine
engine
adapters
events
incremental_lexer
parser_engine
parser_engine_config
registered_adapters
streaming_parser_engine
token_id_scanner
platforms
platforms
cpu
cuda
interface
rocm
tpu
xpu
zen_cpu
plugins
plugins
endpoint_plugins
endpoint_plugins
interface
io_processors
io_processors
interface
lora_resolvers
lora_resolvers
filesystem_resolver
hf_hub_resolver
profiler
profiler
layerwise_profile
utils
wrapper
ray
ray
lazy_utils
ray_env
reasoning
reasoning
abs_reasoning_parsers
basic_parsers
cohere_command_reasoning_parser
deepseek_r1_reasoning_parser
deepseek_v3_reasoning_parser
deepseek_v4_engine_reasoning_parser
ernie45_reasoning_parser
gemma4_engine_reasoning_parser
gemma4_utils
glm47_moe_reasoning_parser
gptoss_reasoning_parser
granite_reasoning_parser
hunyuan_a13b_reasoning_parser
hy_v3_reasoning_parser
identity_reasoning_parser
kimi_k2_reasoning_parser
minimax_m2_reasoning_parser
minimax_m3_reasoning_parser
mistral_reasoning_parser
nemotron_v3_engine_reasoning_parser
olmo3_reasoning_parser
poolside_v1_reasoning_parser
qwen3_engine_reasoning_parser
seed_oss_engine_reasoning_parser
step3_reasoning_parser
step3p5_reasoning_parser
renderers
renderers
base
deepseek_v4
deepseek_v32
embed_utils
hf
mistral
online_derenderer
online_renderer
params
registry
terratorch
inputs
inputs
preprocess
tokenize
tokenizers
tokenizers
deepseek_v4
deepseek_v4_encoding
deepseek_v32
deepseek_v32_encoding
detokenizer_utils
fastokens
hf
kimi_audio
mistral
protocol
registry
tool_parsers
tool_parsers
abstract_tool_parser
apertus_tool_parser
cohere_command_tool_parser
deepseekv3_tool_parser
deepseekv4_engine_tool_parser
deepseekv31_tool_parser
deepseekv32_engine_tool_parser
ernie45_tool_parser
functiongemma_tool_parser
gemma4_engine_tool_parser
gemma4_utils
gigachat3_tool_parser
glm47_moe_tool_parser
gptoss_tool_parser
granite4_tool_parser
granite_20b_fc_tool_parser
granite_tool_parser
hermes_tool_parser
hunyuan_a13b_tool_parser
hy_v3_tool_parser
internlm2_tool_parser
jamba_tool_parser
kimi_k2_tool_parser
lfm2_tool_parser
llama4_pythonic_tool_parser
llama_tool_parser
longcat_tool_parser
minicpm5xml_tool_parser
minimax_m2_tool_parser
minimax_m3_tool_parser
mistral_tool_parser
olmo3_tool_parser
phi4mini_tool_parser
poolside_v1_tool_parser
pythonic_tool_parser
qwen3_engine_tool_parser
rust_tool_parser
seed_oss_engine_tool_parser
step3_tool_parser
step3p5_tool_parser
streaming
structural_tag_registry
utils
xlam_tool_parser
tracing
tracing
otel
utils
transformers_utils
transformers_utils
config
config_parser_base
dynamic_module
model_arch_config_convertor
processor
repo_utils
runai_utils
s3_utils
utils
chat_templates
chat_templates
registry
triton_utils
triton_utils
allocation
force_first_config
importing
usage
usage
usage_lib
utils
utils
argparse_utils
async_utils
cache
collection_utils
counter
cpu_resource_utils
cpu_triton_utils
deep_gemm
flashinfer
func_utils
gc_utils
gpu_sync_debug
hashing
hpc
humming
import_utils
jit_monitor
jsontree
math_utils
mem_constants
mem_utils
mistral
multi_stream_utils
nccl
network_utils
numa_utils
nvtx_pytorch_hooks
ompmultiprocessing
platform_utils
print_utils
registry
serial_utils
sparse_utils
system_utils
tensor_schema
torch_utils
tqdm_utils
v1
v1
cudagraph_dispatcher
kv_cache_interface
kv_cache_spec_registry
outputs
request
serial_utils
utils
attention
attention
backend
selector
backends
backends
cpu_attn
fa_utils
flash_attn
flash_attn_diffkv
flashinfer
flex_attention
gdn_attn
hpc_attn
linear_attn
mamba1_attn
mamba2_attn
mamba_attn
registry
rocm_aiter_fa
rocm_aiter_unified_attn
rocm_attn
short_conv_attn
triton_attn
triton_attn_diffkv
turboquant_attn
utils
mla
mla
aiter_triton_mla
compressor_utils
cutlass_mla
flashattn_mla
flashattn_mla_sparse
flashinfer_mla
flashinfer_mla_sparse
flashinfer_mla_sparse_sm120
flashmla
flashmla_sparse
indexer
rocm_aiter_mla
rocm_aiter_mla_sparse
sparse_swa
sparse_utils
tokenspeed_mla
triton_mla
xpu_mla_sparse
prefill
prefill
aiter_flash_attn
base
flash_attn
flashinfer
registry
selector
tokenspeed_mla
trtllm_ragged
ops
ops
chunked_prefill_paged_decode
common
dcp_alltoall
flashmla
int4_per_token_head
merge_attn_states
paged_attn
prefix_prefill
rocm_aiter_mla_sparse
triton_attention_helpers
triton_decode_attention
triton_fp8_mqa_logits
triton_merge_attn_states
triton_prefill_attention
triton_reshape_and_cache_flash
triton_turboquant_decode
triton_turboquant_store
triton_unified_attention
triton_unified_attention_diffkv
vit_attn_wrappers
xpu_mla_sparse
core
core
block_pool
encoder_cache_manager
kv_cache_coordinator
kv_cache_manager
kv_cache_metrics
kv_cache_utils
single_type_kv_cache_manager
sched
sched
async_scheduler
interface
output
request_queue
scheduler
utils
engine
engine
async_llm
coordinator
core
core_client
detokenizer
exceptions
input_processor
llm_engine
logprobs
output_processor
parallel_sampling
tensor_ipc
utils
executor
executor
abstract
multiproc_executor
ray_env_utils
ray_executor
ray_executor_v2
ray_utils
uniproc_executor
vllm_net_devices
kv_offload
kv_offload
base
factory
file_mapper
cpu
cpu
common
gpu_worker
manager
shared_offload_region
spec
swap_blocks_triton
policies
policies
arc
base
lru
tiering
tiering
async_lookup
base
factory
manager
spec
example
example
manager
fs
fs
io
manager
thread_pool
obj
obj
config
manager
p2p
p2p
manager
控制
控制
base
zmq
data
data
base
nixl
session
session
客户端
protocol
server
session
metrics
metrics
loggers
perf
prometheus
ray_wrappers
reader
stats
utils
pool
pool
late_interaction
late_interaction_runner
metadata
sample
sample
metadata
rejection_sampler
sampler
thinking_budget_state
logits_processor
logits_processor
builtin
interface
state
ops
ops
bad_words
logprobs
penalties
topk_topp_sampler
topk_topp_triton
simple_kv_offload
simple_kv_offload
copy_backend
cuda_mem_ops
manager
metadata
worker
spec_decode
spec_decode
custom_class_proposer
dflash
draft_model
eagle
extract_hidden_states
gemma4
llm_base_proposer
medusa
metadata
metrics
ngram_proposer
ngram_proposer_gpu
step3p5
suffix_decoding
utils
vocab_mapping
动态
动态
utils
structured_output
structured_output
backend_guidance
backend_lm_format_enforcer
backend_outlines
backend_types
backend_xgrammar
request
utils
worker
worker
block_table
cp_utils
cpu_model_runner
cpu_worker
dp_utils
ec_connector_model_runner_mixin
encoder_cudagraph
encoder_cudagraph_defs
gpu_input_batch
gpu_model_runner
gpu_ubatch_wrapper
gpu_worker
kv_connector_model_runner_mixin
lora_model_runner_mixin
mamba_utils
启动计划
tpu_input_batch
ubatch_utils
ubatching
utils
worker_base
workspace
xpu_model_runner
xpu_worker
cpu
cpu
buffer_utils
model_runner
shm
gpu
gpu
async_utils
attn_utils
block_table
buffer_utils
cp_utils
cudagraph_utils
dp_utils
eplb_utils
input_batch
kv_connector
lora_utils
model_runner
pp_utils
shutdown
states
structured_outputs
warmup
metrics
metrics
logits
mm
mm
encoder_cache
encoder_runner
lora
rope
model_states
model_states
default
编码器-解码器
interface
mamba_hybrid
mm_pruning
pool
pool
pooling_runner
sample
sample
bad_words
gumbel
logit_bias
logprob
min_p
output
penalties
prompt_logprob
sampler
states
spec_decode
spec_decode
rejection_sampler
rejection_sampler_utils
speculator
utils
autoregressive
autoregressive
cudagraph_utils
speculator
dflash
dflash
cudagraph
speculator
utils
dspark
dspark
speculator
utils
eagle
eagle
eagle3_utils
speculator
utils
gemma4
gemma4
speculator
mtp
mtp
speculator
CLI 参考
CLI 参考
vllm serve
vllm chat
vllm complete
vllm run-batch
vllm bench
vllm bench
vllm bench latency
vllm bench mm-processor
vllm bench serve
vllm bench sweep plot
vllm bench sweep plot_pareto
vllm bench sweep serve
vllm bench sweep serve_workload
vllm bench throughput
vllm launch
vllm launch
vllm launch render
社区
社区
联系我们
Meetups
赞助商
治理
治理
协作策略
贡献者
治理流程
博客
论坛
Slack
主页
社区
聚会
¶
我们会在全球各地定期举办聚会。我们将分享来自 vLLM 团队的项目更新,并邀请行业嘉宾分享他们的经验和见解。
请访问
vllm.ai/events
了解更多信息。
回到顶部