Lighteval documentation

Pipeline

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.7.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Pipeline

Pipeline

class lighteval.pipeline.Pipeline

< >

( tasks: str pipeline_parameters: PipelineParameters evaluation_tracker: EvaluationTracker model_config = None model = None )

PipelineParameters

class lighteval.pipeline.PipelineParameters

< >

( launcher_type: ParallelismManager env_config: EnvConfig = <factory> job_id: int = 0 dataset_loading_processes: int = 1 nanotron_checkpoint_path: str | None = None custom_tasks_directory: str | None = None override_batch_size: int | None = None num_fewshot_seeds: int = 1 max_samples: int | None = None use_chat_template: bool = False system_prompt: str | None = None )

ParallelismManager

class lighteval.pipeline.ParallelismManager

< >

( value names = None module = None qualname = None type = None start = 1 )

An enumeration.

< > Update on GitHub