跳到内容

llmcompressor.modifiers.smoothquant.utils

函数

get_layer_mappings_from_architecture

get_layer_mappings_from_architecture(
    architecture: str,
) -> List[LayerMap]

参数

  • architecture

    (str) –

    str: 模型的架构

返回

  • List[LayerMap] –

    list: 给定架构的层映射

源代码在 llmcompressor/modifiers/smoothquant/utils.py 
86
87
88
89
90
91
92
93
94
95
96
97
98
def get_layer_mappings_from_architecture(architecture: str) -> List[LayerMap]:
    """
    :param architecture: str: The architecture of the model
    :return: list: The layer mappings for the given architecture
    """

    if architecture not in MAPPINGS_REGISTRY:
        logger.info(
            f"Architecture {architecture} not found in mappings. "
            f"Using default mappings: {DEFAULT_SMOOTHQUANT_MAPPINGS}"
        )

    return MAPPINGS_REGISTRY.get(architecture, DEFAULT_SMOOTHQUANT_MAPPINGS)

handle_mapping_resolution_errors

handle_mapping_resolution_errors(func)

装饰器，用于捕获解析映射时发生的任何错误，并向用户提供有用的错误消息，指向 README 文件。

源代码在 llmcompressor/modifiers/smoothquant/utils.py 
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
def handle_mapping_resolution_errors(func):
    """
    Decorator to catch any errors that occur when resolving mappings and provide a
    helpful error message to the user pointing them to the README
    """

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as original_exception:
            readme_location = (
                "https://github.com/vllm-project/llm-compressor/tree/main/"
                "src/llmcompressor/modifiers/smoothquant"
            )
            raise RuntimeError(
                f"Error resolving mappings for given architecture."
                f"Please refer to the README at {readme_location} for more information."
            ) from original_exception

    return wrapper