llmcompressor.transformers.data.evolcodealpaca

类

• EvolCodeAlpacaDataset –

  Evol Code Alpaca 数据集的子文本生成类

EvolCodeAlpacaDataset

EvolCodeAlpacaDataset(
    dataset_args: DatasetArguments,
    split: str,
    processor: Processor,
)

基类：TextGenerationDataset

Evol Code Alpaca 数据集的子文本生成类

参数

• dataset_args

  (DatasetArguments) –

  数据集加载的配置设置

• split

  (str) –

  从数据集中加载的拆分，例如 test 或 train[:5%]

• processor

  (Processor) –

  要在数据集上使用的处理器或分词器

源代码在 llmcompressor/transformers/data/evolcodealpaca.py

def __init__(
    self, dataset_args: "DatasetArguments", split: str, processor: Processor
):
    dataset_args = deepcopy(dataset_args)
    dataset_args.dataset = "theblackcat102/evol-codealpaca-v1"
    dataset_args.text_column = "text"

    super().__init__(dataset_args, split=split, processor=processor)