跳到内容

llmcompressor.transformers.tracing.debug

函数

  • trace –

    通过将预训练模型跟踪到子图来调试可追溯性

trace

trace(
    model_id: str,
    model_class: Type[PreTrainedModel],
    sequential_targets: list[str] | str | None = None,
    ignore: list[str]
    | str = DatasetArguments().tracing_ignore,
    modality: str = "text",
    trust_remote_code: bool = True,
    skip_weights: bool = True,
    device_map: str | dict = "cpu",
) -> Tuple[
    PreTrainedModel, list[Subgraph], dict[str, torch.Tensor]
]

通过将预训练模型跟踪到子图来调试可追溯性

参数

  • model_id

    (str) –

    要加载的模型存根

  • model_class

    (Type[PreTrainedModel]) –

    预训练模型的类构造函数。可以使用 HF transformers 类或 LLM Compressor 定义的 Traceable 类

  • sequential_targets

    (list[str] | str | None, default: None ) –

    顺序跟踪的目标，默认为自动推断

  • ignore

    (list[str] | str, default: tracing_ignore ) –

    跟踪期间要忽略的模式

  • modality

    (str, default: 'text' ) –

    用于模拟跟踪数据的模式，默认为 'text'

  • trust_remote_code

    (bool, 默认值: True ) –

    信任远程模型代码 CLI 使用示例 llmcompressor.trace --model_id Qwen/Qwen2-VL-2B-Instruct --model_class Qwen2VLForConditionalGeneration --sequential_targets Qwen2VLDecoderLayer --ignore "lm_head" "re:visual.*" --modality text

Source code in llmcompressor/transformers/tracing/debug.py 
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def trace(
    model_id: str,
    model_class: Type[PreTrainedModel],
    sequential_targets: list[str] | str | None = None,
    ignore: list[str] | str = DatasetArguments().tracing_ignore,
    modality: str = "text",
    trust_remote_code: bool = True,
    skip_weights: bool = True,
    device_map: str | dict = "cpu",
) -> Tuple[PreTrainedModel, list[Subgraph], dict[str, torch.Tensor]]:
    """
    Debug traceability by tracing a pre-trained model into subgraphs

    :param model_id: stub of the model to load
    :param model_class: class constructor of the pre-trained model. Can use either
        HF transformers classes or `Traceable` classes defined by LLM Compressor
    :param sequential_targets: targets for sequential tracing, defaults to automatic
        inference
    :param ignore: patterns to ignore during tracing
    :param modality: data modality for dummy tracing data, defaults to 'text'
    :param trust_remote_code: trust remote model code

    Example usage from CLI
    llmcompressor.trace \
        --model_id Qwen/Qwen2-VL-2B-Instruct \
        --model_class Qwen2VLForConditionalGeneration \
        --sequential_targets Qwen2VLDecoderLayer \
        --ignore "lm_head" "re:visual.*" \
        --modality text
    """
    # Load model
    with skip_weights_download(model_class) if skip_weights else nullcontext():
        model = model_class.from_pretrained(
            model_id,
            device_map=device_map,
            torch_dtype="auto",
            trust_remote_code=trust_remote_code,
        )
    processor = AutoProcessor.from_pretrained(
        model_id, trust_remote_code=trust_remote_code
    )
    print("Loaded model")

    # Prepare sample data
    dataset_args = DatasetArguments(**get_dataset_kwargs(modality, ignore))
    dataset = TextGenerationDataset.load_from_registry(
        dataset_args.dataset,
        dataset_args=dataset_args,
        split=dataset_args.splits["calibration"],
        processor=processor,
    )(add_labels=False)
    sample = next(iter(dataset))
    sample = collate_sample(sample, device=model.device)
    print("Loaded sample data")

    # infer sequential targets
    if sequential_targets is None:
        sequential_targets = get_no_split_params(model)
    if isinstance(sequential_targets, str):
        sequential_targets = [sequential_targets]

    # Attempt trace
    print(
        "\nAttempting trace\n"
        f"    model_id={model_id}\n"
        f"    model_class={model_class.__name__}\n"
        f"    dataset={dataset_args.dataset}\n"
        f"    split={dataset.split}\n"
        f"    inputs={sample.keys()}\n"
        f"    sequential_targets={sequential_targets}\n"
        f"    ignore={dataset_args.tracing_ignore}\n"
    )
    subgraphs = trace_subgraphs(
        model, sample, sequential_targets, dataset_args.tracing_ignore
    )
    print(f"Successfully traced model into {len(subgraphs)} subgraphs!\n")

    return model, subgraphs, sample