llmcompressor.modeling.qwen3_moe

类

• CalibrationQwen3MoeSparseMoeBlock –

  Qwen3MoeSparseMoeBlock 的校准版本，该版本将所有 token 发送到所有 expert。

CalibrationQwen3MoeSparseMoeBlock

CalibrationQwen3MoeSparseMoeBlock(
    original: Qwen3MoeSparseMoeBlock,
    config: Qwen3MoeConfig,
    calibrate_all_experts: bool = True,
)

基类: MoECalibrationModule

Qwen3MoeSparseMoeBlock 的校准版本，该版本将所有 token 发送到所有 expert。

源代码在 llmcompressor/modeling/qwen3_moe.py

def __init__(
    self,
    original: OriginalQwen3MoeSparseMoeBlock,
    config: Qwen3MoeConfig,
    calibrate_all_experts: bool = True,
):
    super().__init__()
    self.num_experts = config.num_experts
    self.top_k = config.num_experts_per_tok
    self.norm_topk_prob = config.norm_topk_prob

    self.calibrate_all_experts = calibrate_all_experts
    self.gate = original.gate
    self.experts = original.experts