跳到内容

llmcompressor.transformers.utils.helpers

用于将 LLM Compressor 与 huggingface/transformers 流集成相关的助手变量和函数

函数

infer_recipe_from_model_path

infer_recipe_from_model_path(
    model_path: str | Path,
) -> str | None

从 model_path 推断 recipe。

参数

  • model_path

    (str | Path) –

    要加载模型的路径。它可以是以下之一： - 模型目录的路径 - 模型文件的路径 - Hugging Face 模型 ID

返回

  • str | None –

    如果找到 recipe 文件，则返回该文件路径，否则返回 None。

源代码位于 llmcompressor/transformers/utils/helpers.py 
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
def infer_recipe_from_model_path(model_path: str | Path) -> str | None:
    """
    Infer the recipe from the model_path.

    :param model_path: The path to the model to load. It can be one of the following:
        - a path to the model directory
        - a path to the model file
        - Hugging face model ID
    :return: The path to the recipe file if found, None otherwise.
    """
    model_path = (
        model_path.as_posix() if isinstance(model_path, Path) else model_path.strip()
    )
    if model_path == "":
        logger.debug("got path_or_name=<empty string>" "unable to find recipe")
        return None

    if os.path.isdir(model_path) or os.path.isfile(model_path):
        # Model path is a local path to the model directory or file
        model_path = (
            os.path.dirname(model_path) if os.path.isfile(model_path) else model_path
        )
        recipe = os.path.join(model_path, RECIPE_FILE_NAME)

        if os.path.isfile(recipe):
            logger.info(f"Found recipe in the model_path: {recipe}")
            return recipe
        logger.debug(f"No recipe found in the model_path: {model_path}")
        return None

    # Try to resolve HF model ID to cached location first
    cached_recipe = try_to_load_from_cache(
        repo_id=model_path,
        filename=RECIPE_FILE_NAME,
    )

    if cached_recipe and cached_recipe is not _CACHED_NO_EXIST:
        # Recipe found in cached model
        logger.info(f"Found recipe in cached model: {cached_recipe}")
        return cached_recipe
    # No recipe in cache - fall through to network check

    # If the model path is a Hugging Face model ID
    recipe = recipe_from_huggingface_model_id(hf_stub=model_path)

    if recipe is None:
        logger.debug("Failed to infer the recipe from the model_path")

    return recipe

is_model_ct_quantized_from_path

is_model_ct_quantized_from_path(path: str) -> bool

根据配置确定模型是否已量化

参数

  • 路径

    (str) –

    模型或 HF 存根的路径

返回

  • bool –

    如果配置包含来自给定路径的 quantization_config，则为 True

源代码位于 llmcompressor/transformers/utils/helpers.py 
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
def is_model_ct_quantized_from_path(path: str) -> bool:
    """
    Determine if model from path is quantized based
    on the config

    :param path: path to the model or HF stub
    :return: True if config contains quantization_config from the given path

    """
    config = AutoConfig.from_pretrained(path)
    if config is not None:
        if (
            hasattr(config, "quantization_config")
            and config.quantization_config["quant_method"] == "compressed-tensors"
        ):
            return True
    return False

recipe_from_huggingface_model_id

recipe_from_huggingface_model_id(
    hf_stub: str, recipe_file_name: str = RECIPE_FILE_NAME
) -> str | None

尝试从 Hugging Face 模型 ID 下载 recipe。

参数

  • hf_stub

    (str) –

    假定为 Hugging Face 模型 ID。

  • recipe_file_name

    (str, default: RECIPE_FILE_NAME ) –

    要下载的 recipe 文件的名称。默认为 RECIPE_FILE_NAME。

返回

  • str | None –

    一个元组： - 如果找到 recipe 文件，则返回该文件路径，否则返回 None。 - 如果 hf_stub 是有效的 Hugging Face 模型 ID，则返回 True，否则返回 False。

源代码位于 llmcompressor/transformers/utils/helpers.py 
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
def recipe_from_huggingface_model_id(
    hf_stub: str, recipe_file_name: str = RECIPE_FILE_NAME
) -> str | None:
    """
    Attempts to download the recipe from the Hugging Face model ID.

    :param hf_stub: Assumed to be the Hugging Face model ID.
    :param recipe_file_name: The name of the recipe file to download.
     Defaults to RECIPE_FILE_NAME.
    :return: A tuple:
        - The path to the recipe file if found, None otherwise.
        - True if hf_stub is a valid Hugging Face model ID, False otherwise.
    """
    # Check if offline mode is enabled
    if os.getenv("HF_HUB_OFFLINE") == "1":
        logger.debug("HF_HUB_OFFLINE is set, skipping recipe download from HuggingFace")
        return None

    # Use custom HF_ENDPOINT
    hf_api = HfApi()
    model_id_url = f"{hf_api.endpoint.rstrip('/')}/{hf_stub}"
    request = requests.head(model_id_url)

    if request.status_code != 200:
        logger.debug(
            (
                "hf_stub is not a valid Hugging Face model ID. ",
                "Skipping recipe resolution.",
            )
        )
        return None

    try:
        recipe = hf_hub_download(repo_id=hf_stub, filename=recipe_file_name)
        logger.info(f"Found recipe: {recipe_file_name} for model ID: {hf_stub}.")
    except Exception as e:  # TODO: narrow acceptable exceptions
        logger.debug(
            (
                f"Unable to find recipe {recipe_file_name} "
                f"for model ID: {hf_stub}: {e}."
                "Skipping recipe resolution."
            )
        )
        recipe = None

    return recipe