diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 000000000..55e349a08 --- /dev/null +++ b/skills/README.md @@ -0,0 +1,26 @@ +# Olive Agent Skills + +This directory contains portable [Agent Skills](https://agentskills.io/) for AI assistants. + +| Skill | Purpose | +| --- | --- | +| [`olive`](olive/SKILL.md) | Use the native Olive CLI and YAML/JSON workflows to optimize AI models. | + +## Install + +With GitHub CLI 2.90.0 or later: + +```shell +gh skill install microsoft/Olive olive +``` + +For a manual installation, copy the complete `skills/olive` directory to one of the locations supported by +your AI assistant: + +- Personal GitHub Copilot skill: `~/.copilot/skills/olive` +- Project GitHub Copilot skill: `.github/skills/olive` +- Cross-agent project skill: `.agents/skills/olive` +- Claude project skill: `.claude/skills/olive` + +The skill requires the `olive` command from the `olive-ai` Python package. It does not require an MCP +server. diff --git a/skills/olive/SKILL.md b/skills/olive/SKILL.md new file mode 100644 index 000000000..efcfb0cb2 --- /dev/null +++ b/skills/olive/SKILL.md @@ -0,0 +1,154 @@ +--- +name: olive +description: Use Microsoft Olive through its native CLI and YAML/JSON workflow configuration files to optimize, export, quantize, fine-tune, tune, evaluate, and package AI models for ONNX Runtime. Use when a user mentions Olive, olive-ai, olive optimize, Olive passes, model conversion, ONNX optimization, execution providers, or asks to create, explain, validate, or run an Olive workflow config. +license: MIT +compatibility: Requires Python 3.10 or later and the olive-ai package. Model downloads and some dependency installations require network access; GPU, NPU, and vendor-specific workflows require matching hardware and runtimes. +metadata: + author: microsoft + version: "2.0.0" +--- + +# Microsoft Olive + +Use the native `olive` command and Olive workflow files. This skill does not require or assume an MCP +server. + +Treat the installed Olive version as the source of truth. Before using unfamiliar options, run: + +```shell +olive --help +olive --help +``` + +Do not invent command flags, pass names, pass parameters, model types, or execution providers. If an +existing project already has an Olive config, preserve its conventions and make the smallest necessary +change. + +## Choose the right interface + +| User goal | Preferred interface | +| --- | --- | +| Wants guided setup or does not know which operation to choose | `olive init` | +| Wants an end-to-end optimized model | `olive optimize` | +| Wants only ONNX export | `olive capture-onnx-graph` | +| Wants only quantization | `olive quantize` | +| Wants text LoRA or QLoRA training | `olive finetune` | +| Wants diffusion LoRA training | `olive diffusion-lora` | +| Wants one known Olive pass | `olive run-pass` | +| Wants a repeatable multi-pass pipeline, evaluation, search, or custom data | `olive run --config ...` | +| Wants ONNX Runtime session tuning | `olive tune-session-params` | +| Wants lm-eval benchmarking | `olive benchmark` | + +Do not use `olive auto-opt`; it is deprecated in favor of `olive optimize`. + +Read [the CLI guide](references/cli.md) for installation, command examples, dry runs, test mode, and +provider selection. + +## Execution workflow + +1. Identify the input model format and path or Hugging Face ID. +2. Identify the desired output: optimized ONNX, quantized model, adapter, benchmark, or reusable workflow. +3. Identify the target device and execution provider only when the user has not already specified them. +4. Check `olive --help` in the active environment. +5. Use an explicit output directory and `--log_level 1` for meaningful progress logs. +6. For expensive or unfamiliar high-level commands, add `--dry_run`. Inspect the generated + `/config.json`, then run it with `olive run --config ` after it is correct. +7. Run the requested operation. Do not claim success until the process exits successfully and the expected + output exists. +8. Report the output path, selected provider and precision, passes that ran, and metrics that Olive actually + returned. + +Use `--save_config_file` when the user wants both execution and a saved recipe. It saves +`olive_config.json` while the command continues. Use `--dry_run` when the user wants configuration +generation without optimization; it saves `config.json` and stops. It does not perform full workflow or +pass-schema validation. + +## Write workflow configuration + +Use a YAML or JSON workflow when the user needs multiple passes, reusable configuration, custom data, +evaluation, search, custom scripts, remote systems, or settings not exposed by a high-level command. + +Read [the workflow configuration guide](references/workflow-config.md) before creating or editing a +workflow. For a model- and provider-specific workflow, first look for the same model or a close architecture +in [microsoft/olive-recipes](https://github.com/microsoft/olive-recipes). Read that recipe's README and use +the executable workflow JSON named in its command; `info.yml` and `info.yaml` are recipe catalog metadata, +not files for `olive run --config`. + +If there is no close recipe, generate a version-specific config with a high-level command: + +```shell +olive optimize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --precision int4 \ + --provider CPUExecutionProvider \ + --output_path olive-output \ + --dry_run +``` + +When authoring a workflow: + +- Keep `input_model`, `passes`, and output settings explicit. +- Define pass entries in execution order; mapping order is pipeline order. +- Reference systems, data configs, and evaluators by their declared names. +- Use a unique `name` for every data config. +- Olive accepts JSON and YAML. Recipes normally use JSON for executable workflows; YAML remains useful for + hand-maintained workflows with comments. +- Use [the bundled workflow template](assets/workflow.yaml) only as a generic Hugging Face-to-ONNX example. + Current generative LLM recipes often use `ModelBuilder`; follow the closest recipe instead of substituting + `OnnxConversion`. +- Never copy a model's Hugging Face `config.json` and treat it as an Olive workflow. +- Never place tokens, credentials, or secrets in a workflow file. + +Before using a nontrivial pass, inspect its installed schema from this skill's root: + +```shell +python scripts/inspect_pass.py OnnxConversion +``` + +Validate a workflow without running model optimization: + +```shell +python scripts/validate_config.py workflow.yaml +olive run --config workflow.yaml --list_required_packages +``` + +The second command writes `olive_requirements.txt` in the current directory. Review it, install the listed +packages into the intended environment, and check model-loader and exporter requirements described in the +CLI guide before running: + +```shell +olive run --config workflow.yaml +``` + +Structural validation cannot prove that remote models are accessible, local data is semantically correct, +the model is supported by every pass, or the target hardware has enough memory. Surface those constraints +instead of presenting validation as execution success. + +## Dependency and hardware rules + +- Reuse the user's active environment when it already contains the required Olive and runtime packages. +- Install dependencies only when Olive or a required optional package is missing. +- Use one ONNX Runtime variant per environment. Do not combine CPU, CUDA, DirectML, OpenVINO, or QNN + runtime packages in the same environment unless the installed package documentation explicitly supports + it. +- Match `device`, execution provider, and runtime: CPU with `CPUExecutionProvider`, NVIDIA GPU with + `CUDAExecutionProvider`, WebGPU with `WebGpuExecutionProvider`, Windows DirectX GPU with + `DmlExecutionProvider`, and Qualcomm NPU with `QNNExecutionProvider`. Configure only one execution + provider per accelerator. +- Do not select fp16 for CPU merely to reduce model size. Prefer int4 or int8 when supported. +- Calibration-based quantization and fine-tuning may require datasets and substantial compute. Do not + silently replace the user's dataset or algorithm. + +## Safety and correctness + +- Use `--trust_remote_code` only when the user explicitly trusts the model repository or existing project + configuration already requires it. +- Use `HF_TOKEN` or the Hugging Face credential store for gated models. Never write a token into a command, + config, script, log, or committed file. +- Do not enable `clean_cache`, delete outputs, overwrite a nonempty output directory, or remove generated + artifacts without user intent. +- A `--test` run uses a small randomly initialized model with the same architecture. It checks pipeline + compatibility, not real model quality. +- Do not claim quality, latency, memory, or size improvements without comparing actual outputs or metrics. +- If network access or hardware is unavailable, complete local config and dry-run validation and state + exactly what remains unverified. diff --git a/skills/olive/assets/workflow.yaml b/skills/olive/assets/workflow.yaml new file mode 100644 index 000000000..6757ac2fa --- /dev/null +++ b/skills/olive/assets/workflow.yaml @@ -0,0 +1,31 @@ +workflow_id: distilbert_cpu_onnx + +input_model: + type: HfModel + model_path: distilbert/distilbert-base-uncased-finetuned-sst-2-english + task: text-classification + +systems: + local_cpu: + type: LocalSystem + accelerators: + - device: cpu + execution_providers: + - CPUExecutionProvider + +passes: + conversion: + type: OnnxConversion + target_opset: 20 + save_as_external_data: true + all_tensors_to_one_file: true + graph_optimization: + type: OrtTransformersOptimization + +search_strategy: false +evaluate_input_model: false +host: local_cpu +target: local_cpu +output_dir: olive-output +cache_dir: .olive-cache +log_severity_level: 1 diff --git a/skills/olive/references/cli.md b/skills/olive/references/cli.md new file mode 100644 index 000000000..ef489bb2e --- /dev/null +++ b/skills/olive/references/cli.md @@ -0,0 +1,308 @@ +# Olive CLI Guide + +The installed CLI is authoritative: + +```shell +olive --help +olive --help +``` + +Examples below use POSIX line continuation. Use backticks in PowerShell or place the command on one line. + +## Installation + +Create or reuse a virtual environment. Install exactly one ONNX Runtime provider extra: + +| Target | Olive package | Common LLM Model Builder package | +| --- | --- | --- | +| CPU | `olive-ai[cpu]` | `onnxruntime-genai` | +| NVIDIA CUDA | `olive-ai[gpu]` | `onnxruntime-genai-cuda` | +| Windows DirectML | `olive-ai[directml]` | `onnxruntime-genai-directml` | +| OpenVINO | `olive-ai[openvino]` | Check the selected exporter | +| Qualcomm QNN | `olive-ai[qnn]` | Check the selected exporter | + +Hugging Face models normally also need `transformers`. Combine extras when appropriate: + +```shell +python -m pip install "olive-ai[cpu,capture-onnx-graph]" transformers onnxruntime-genai +``` + +Common operation extras: + +| Operation | Extra | +| --- | --- | +| ONNX capture through Optimum | `olive-ai[capture-onnx-graph]` | +| LoRA or QLoRA fine-tuning | `olive-ai[finetune]` | +| Diffusion pipelines | `olive-ai[diffusers]` and `datasets` | +| OpenVINO passes | `olive-ai[openvino]` | +| Docker system | `olive-ai[docker]` | + +Some quantizers and vendor passes have additional dependencies. For a workflow file, use +`olive run --config --list_required_packages`. For a high-level command, inspect its help and the +error from the selected implementation rather than installing every optional package. + +Do not install multiple ONNX Runtime variants into one environment. Use separate environments when +switching between CPU, CUDA, DirectML, OpenVINO, and QNN. + +For Olive source development: + +```shell +python -m pip install -r requirements.txt +python -m pip install -e . +``` + +## Guided setup + +`olive init` interactively selects a model, operation, target, and output. It can generate a CLI command, +generate a reusable config, or run immediately: + +```shell +olive init --output_path olive-output +``` + +Use it when requirements are vague or the user wants to explore supported choices. + +## End-to-end optimization + +`optimize` is the default all-in-one command. It schedules export, quantization, graph optimization, and +provider-specific passes: + +```shell +olive optimize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --device cpu \ + --provider CPUExecutionProvider \ + --precision int4 \ + --output_path models/qwen-int4 \ + --log_level 1 +``` + +Useful options include: + +- `--exporter model_builder|dynamo_exporter|torchscript_exporter|optimum_exporter` +- `--act_precision int8` for activation quantization where supported +- `--block_size ` for block-wise quantization +- `--surgeries ` for graph surgeries +- `--num_split ` and `--memory ` for model splitting +- `--enable_aot --qnn_env_path ` for supported QNN AOT workflows + +Never use deprecated `olive auto-opt`. + +## Generate a config before running + +All high-level commands that expose `--dry_run` can generate a workflow from their command-line arguments +without executing passes: + +```shell +olive optimize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --precision int4 \ + --provider CPUExecutionProvider \ + --output_path recipes/qwen-int4 \ + --dry_run +``` + +This writes `recipes/qwen-int4/config.json`. Review or edit it, then execute: + +```shell +olive run --config recipes/qwen-int4/config.json +``` + +Dry run checks command-line argument handling, but it does not perform full `RunConfig` or pass-schema +validation. Use `scripts/validate_config.py` and `olive run --list_required_packages` before execution. + +`--save_config_file` is different: it saves `olive_config.json` and still executes the command. + +## Export only + +Use `capture-onnx-graph` when the user wants conversion without the full optimization schedule: + +```shell +olive capture-onnx-graph \ + --model_name_or_path microsoft/Phi-4-mini-instruct \ + --use_model_builder \ + --precision int4 \ + --use_ort_genai \ + --output_path models/phi-onnx \ + --log_level 1 +``` + +Exporter choices are mutually exclusive: + +- `--use_model_builder` for supported generative models +- `--use_dynamo_exporter` for PyTorch Dynamo export +- `--use_mobius_builder` for supported multi-component or multimodal models; requires `mobius-ai` +- No exporter flag uses the command's default PyTorch/Optimum route + +Use `--target_opset`, `--torch_dtype`, and `--fixed_param_dict` only when the exporter supports them. + +## Quantize only + +For a Hugging Face or PyTorch model: + +```shell +olive quantize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --algorithm rtn \ + --precision int4 \ + --implementation olive \ + --output_path models/qwen-rtn \ + --log_level 1 +``` + +For an ONNX model using ONNX Runtime dynamic int8 quantization: + +```shell +olive quantize \ + --model_name_or_path model.onnx \ + --algorithm rtn \ + --precision int8 \ + --implementation ort \ + --output_path models/model-int8 \ + --log_level 1 +``` + +Supported combinations depend on input format, implementation, precision, QDQ encoding, and whether a +dataset is supplied. Run `olive quantize --help`; do not assume every algorithm works with every backend. +AWQ and GPTQ commonly need GPU resources and calibration data. + +## Fine-tune + +Text LoRA or QLoRA: + +```shell +olive finetune \ + --model_name_or_path microsoft/Phi-4-mini-instruct \ + --method qlora \ + --data_name tatsu-lab/alpaca \ + --use_chat_template \ + --max_seq_len 1024 \ + --max_samples 256 \ + --output_path models/phi-adapter \ + --log_level 1 +``` + +Use exactly one of `--text_field`, `--text_template`, or `--use_chat_template`. For local files, set +`--data_name` to the format (`json`, `csv`, or `parquet`) and use `--data_files`. Multiple CLI files are a +comma-separated list without spaces; split mappings require a workflow config. + +Diffusion LoRA: + +```shell +olive diffusion-lora \ + --model_name_or_path stabilityai/stable-diffusion-xl-base-1.0 \ + --model_variant sdxl \ + --data_dir training-images \ + --max_train_steps 1000 \ + --output_path models/sdxl-adapter \ + --log_level 1 +``` + +DreamBooth requires `--dreambooth` and `--instance_prompt`. Prior preservation additionally requires +`--with_prior_preservation` and `--class_prompt`. + +## Run one pass + +List pass names: + +```shell +olive run-pass --list-passes +``` + +Run one pass with a JSON parameter object: + +```shell +olive run-pass \ + --pass-name OnnxConversion \ + --model_name_or_path microsoft/Phi-4-mini-instruct \ + --pass-config '{"target_opset": 20}' \ + --output_path models/phi-onnx \ + --dry_run +``` + +Use `scripts/inspect_pass.py ` from the skill directory before constructing nontrivial +`--pass-config` JSON. + +## Run a workflow file + +Olive accepts `.json`, `.yaml`, and `.yml`: + +```shell +python scripts/validate_config.py workflow.yaml +olive run --config workflow.yaml --list_required_packages +python -m pip install -r olive_requirements.txt +olive run --config workflow.yaml --log_level 1 +``` + +The `run` command can override selected model, output, and logging fields: + +```shell +olive run \ + --config workflow.yaml \ + --model_name_or_path another/model \ + --output_path another-output \ + --log_level 1 +``` + +Only use overrides when intentional; otherwise the workflow file should remain the reproducible source of +truth. + +`--list_required_packages` reports dependencies declared by passes and the selected runtime. It may not +include every model-loader or exporter dependency. For example, a Hugging Face `OnnxConversion` workflow +normally also needs `transformers` and either `optimum` or a separately configured exporter. + +## Tune and benchmark + +Tune ONNX Runtime session settings: + +```shell +olive tune-session-params \ + --model_name_or_path model.onnx \ + --device cpu \ + --providers_list CPUExecutionProvider \ + --output_path tuned-session \ + --log_level 1 +``` + +Evaluate with lm-eval: + +```shell +olive benchmark \ + --model_name_or_path model-or-hf-id \ + --tasks hellaswag \ + --device cpu \ + --limit 0.1 \ + --output_path benchmark-output \ + --log_level 1 +``` + +`ort` and `ortgenai` benchmark backends require ONNX input; `ortgenai` also requires generated +`genai_config.json` assets. + +## Pipeline test mode + +For Hugging Face inputs, `--test` replaces downloaded weights with a small randomly initialized model of +the same architecture: + +```shell +olive optimize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --precision int4 \ + --output_path test-output \ + --test \ + --test_metrics mae speedup +``` + +Use a dedicated output directory. Test mode validates architecture and pipeline compatibility; it does not +measure real model quality. + +## Authentication and remote code + +Authenticate gated models through the Hugging Face credential store or an environment variable: + +```shell +huggingface-cli login +``` + +Never put a token in an Olive config or a committed command script. Add `--trust_remote_code` only after the +user has explicitly accepted the model repository's code. diff --git a/skills/olive/references/workflow-config.md b/skills/olive/references/workflow-config.md new file mode 100644 index 000000000..2b9b7ecee --- /dev/null +++ b/skills/olive/references/workflow-config.md @@ -0,0 +1,413 @@ +# Olive Workflow Configuration + +Olive workflows are YAML or JSON files consumed by: + +```shell +olive run --config workflow.yaml +``` + +Olive accepts both YAML and JSON. Hand-maintained workflows may use YAML for comments, while +[microsoft/olive-recipes](https://github.com/microsoft/olive-recipes) normally uses JSON for executable +workflows. JSON does not allow comments or trailing commas. + +## Choose a starting point + +For a model- and provider-specific workflow, first find the same model or a close architecture in +`microsoft/olive-recipes`. Read the recipe README and use the JSON file named in its `olive run --config` +command. Files named `info.yml` or `info.yaml` describe the recipe for catalog and automation purposes; they +are not Olive workflow configs. + +Reuse the closest recipe's model type, pass chain, system, provider, and data shape, then change only the +fields required for the user's model and output. Recipes can require backend-specific packages or target a +different Olive version, so validate the result against the active installation. + +When no close recipe exists, generate a workflow for the installed Olive version: + +```shell +olive optimize \ + --model_name_or_path Qwen/Qwen2.5-0.5B-Instruct \ + --precision int4 \ + --provider CPUExecutionProvider \ + --output_path generated-workflow \ + --dry_run +``` + +Olive writes `generated-workflow/config.json`. Edit that file rather than rebuilding a complex provider +recipe from memory. + +For a hand-authored starting point, copy `assets/workflow.yaml` from this skill. It is a classic Hugging +Face-to-ONNX conversion and graph-optimization example, not a universal template for current generative +LLMs. It normally needs `olive-ai[cpu,optimum]` and `transformers`. + +## Top-level structure + +```yaml +workflow_id: my_workflow +input_model: {} +systems: {} +data_configs: [] +evaluators: {} +passes: {} +search_strategy: false +host: null +target: null +evaluator: null +output_dir: olive-output +cache_dir: .olive-cache +log_severity_level: 1 +``` + +`input_model` is required. Other sections are optional. Engine settings can be written under an `engine` +object or flattened at the top level; prefer the flattened form emitted by current high-level CLI commands. + +## Input models + +### Hugging Face model + +```yaml +input_model: + type: HfModel + model_path: Qwen/Qwen2.5-0.5B-Instruct + task: text-generation-with-past + adapter_path: null + load_kwargs: + trust_remote_code: false + attn_implementation: eager +``` + +`model_path` and `adapter_path` may be local paths or Hub IDs. Avoid `trust_remote_code: true` unless the +user explicitly trusts the repository. + +### Local PyTorch model + +```yaml +input_model: + type: PyTorchModel + model_path: model.pt + model_script: model_loader.py + model_loader: load_model + io_config: + input_names: [input_ids, attention_mask] + output_names: [logits] + input_shapes: + - [1, 128] + - [1, 128] + dummy_inputs_func: create_dummy_inputs +``` + +`model_loader`, `io_config`, and `dummy_inputs_func` can refer to functions in `model_script`. Supply +`script_dir` when the script imports local modules from another directory. + +### ONNX model + +Single file: + +```yaml +input_model: + type: ONNXModel + model_path: models/model.onnx +``` + +Model with external data: + +```yaml +input_model: + type: ONNXModel + model_path: models/model + onnx_file_name: model.onnx +``` + +For external data, `model_path` is the containing directory and `onnx_file_name` identifies the graph. + +## Systems and execution providers + +Use explicit systems when the target provider matters: + +```yaml +systems: + local_cpu: + type: LocalSystem + accelerators: + - device: cpu + execution_providers: + - CPUExecutionProvider +host: local_cpu +target: local_cpu +``` + +Common mappings: + +| Device | Execution provider | +| --- | --- | +| CPU | `CPUExecutionProvider` | +| NVIDIA GPU | `CUDAExecutionProvider` | +| WebGPU | `WebGpuExecutionProvider` | +| Windows DirectX GPU or NPU | `DmlExecutionProvider` | +| Intel OpenVINO | `OpenVINOExecutionProvider` | +| Qualcomm NPU | `QNNExecutionProvider` | + +Only one accelerator is currently supported per system, and each accelerator accepts one execution +provider. If systems are omitted, Olive defaults to the local system and infers available providers where +possible. Do not copy older recipes that list multiple providers under one accelerator. + +`host` is where passes execute. `target` is where evaluation and target-side passes execute. They may +refer to the same system or different systems. + +## Passes + +Pass keys are unique user-chosen labels. `type` is the registered Olive pass class. Entries execute in +mapping order: + +```yaml +passes: + conversion: + type: OnnxConversion + target_opset: 20 + save_as_external_data: true + all_tensors_to_one_file: true + graph_optimization: + type: OrtTransformersOptimization + session_tuning: + type: OrtSessionParamsTuning + data_config: token_data + io_bind: true +``` + +Pass parameters may be flattened beside `type`, as above. `host` and `evaluator` can override the +workflow-level values for one pass. A nested `config` object is also accepted, but current recipes generally +use flattened pass parameters. Follow the selected recipe and installed pass schema rather than moving +fields between the two forms blindly. + +Many current generative LLM recipes use `ModelBuilder` rather than the traditional `OnnxConversion` pass: + +```yaml +passes: + model_builder: + type: ModelBuilder + precision: int4 +``` + +Some backends add `GraphSurgeries` or backend-specific passes. Copy the complete chain from the closest +model/provider recipe; do not infer it from this minimal example. + +Inspect available passes and exact parameters in the active Olive installation: + +```shell +olive run-pass --list-passes +python scripts/inspect_pass.py OnnxConversion +python scripts/inspect_pass.py OnnxBlockWiseRtnQuantization \ + --device cpu \ + --provider CPUExecutionProvider +``` + +Do not use a pass merely because its name is plausible. Check its supported model format, accelerator, +provider, precision, algorithm, dataset requirement, optional packages, and parameter schema. + +## Data configs + +Data configs are declared once and referenced by name from passes and metrics: + +```yaml +data_configs: + - name: calibration_data + type: HuggingfaceContainer + load_dataset_config: + data_name: Salesforce/wikitext + subset: wikitext-2-raw-v1 + split: train + pre_process_data_config: + strategy: line-by-line + max_samples: 128 + max_seq_len: 512 + dataloader_config: + batch_size: 1 +``` + +Every data config name must be unique and contain letters, numbers, and underscores. + +Common container types: + +- `HuggingfaceContainer` for Hub datasets and local CSV, JSON, JSONL, or Parquet files +- `DummyDataContainer` for explicit input names, shapes, and dtypes +- `TransformersPromptDummyDataContainer` for prompt-phase transformer inputs +- `TransformersTokenDummyDataContainer` for token-phase transformer inputs with KV cache +- `RawDataContainer` for raw tensor files + +Local JSONL example: + +```yaml +data_configs: + - name: local_training_data + type: HuggingfaceContainer + load_dataset_config: + data_name: json + data_files: + train: data/train.jsonl + validation: data/validation.jsonl + split: train + pre_process_data_config: + type: text_generation_huggingface_pre_process + text_cols: text + dataloader_config: + batch_size: 1 +``` + +File-to-split mappings are supported in workflow files even though the high-level CLI accepts only one +file or a comma-separated file list. + +Reference a data config from a pass: + +```yaml +passes: + static_quantization: + type: OnnxStaticQuantization + data_config: calibration_data +``` + +Inline data configs in pass or metric fields are not supported; use the declared name. + +## Evaluators and metrics + +Evaluators contain metrics and can be attached to the engine or an individual pass: + +```yaml +evaluators: + common_evaluator: + metrics: + - name: accuracy + type: accuracy + data_config: evaluation_data + sub_types: + - name: accuracy_score + priority: 1 + goal: + type: max-degradation + value: 0.01 + - name: latency + type: latency + data_config: latency_data + sub_types: + - name: avg + priority: 2 + goal: + type: percent-min-improvement + value: 20 +evaluator: common_evaluator +``` + +Built-in metric types include `accuracy`, `latency`, `throughput`, `size_on_disk`, and `custom`. Multiple +objectives need distinct priorities so Olive can rank candidates. Set `higher_is_better` explicitly for a +custom or unusual metric. + +An evaluator is required for parameter search. Without an evaluator, omit search and use fixed pass +parameters. + +Current language-model recipes may use `LMEvaluator` directly instead of a `metrics` list: + +```yaml +evaluators: + mmlu: + type: LMEvaluator + tasks: [mmlu] + batch_size: 8 +evaluator: mmlu +``` + +## Search + +Disable search for a deterministic ordered pipeline: + +```yaml +search_strategy: false +``` + +Enable search with an explicit strategy: + +```yaml +search_strategy: + execution_order: joint + sampler: tpe + max_samples: 5 + seed: 0 + stop_when_goals_met: true +evaluator: common_evaluator +``` + +Do not use `search_strategy: true`; current Olive versions require an explicit strategy mapping. Only enable +search after defining metrics, priorities, and goals. A config with search enabled but no evaluator is +invalid. + +Pass parameters can use fixed values, `DEFAULT_VALUE`, `SEARCHABLE_VALUES`, or pass-supported lists that +form a categorical search space. Inspect the pass schema before using searchable values. + +## Output, cache, logging, and packaging + +```yaml +output_dir: outputs/my-workflow +cache_dir: .olive-cache +clean_cache: false +clean_evaluation_cache: false +evaluate_input_model: false +log_severity_level: 1 +ort_log_severity_level: 3 +ort_py_log_severity_level: 3 +log_to_file: false +packaging_config: + type: Zipfile + name: optimized-model +``` + +Logging levels are `0` debug, `1` info, `2` warning, `3` error, and `4` critical. Keep `clean_cache` false +unless the user intends to discard cached intermediate results. `no_artifacts: true` suppresses auxiliary +metrics, footprints, and run-history artifacts; it does not prevent the final model from being written to +`output_dir`. + +## Custom code + +Custom script fields belong to the object that consumes them; `user_script` and `script_dir` are not +top-level `RunConfig` fields. + +| Purpose | Configuration scope | +| --- | --- | +| PyTorch model loader and model helpers | `input_model.model_script` and `input_model.script_dir` | +| Registered dataset or data-processing components | `data_configs[*].user_script` and `script_dir` | +| Pass-specific helper code | The pass's `user_script` and `script_dir` fields, only when its schema exposes them | +| Custom metric code | The metric's `user_config.user_script` and `user_config.script_dir` | +| Custom evaluator implementation | `evaluators..user_script` and `evaluators..script_dir` | + +Use registered function names in the corresponding model, data, pass, metric, or evaluator fields. Do not +embed arbitrary Python code in YAML or JSON. + +## Validation and execution + +From the skill root, validate structure, references, pass names, pass parameters, and declared local +packages: + +```shell +python scripts/validate_config.py workflow.yaml +``` + +Ask Olive to generate a dependency file without running passes: + +```shell +olive run --config workflow.yaml --list_required_packages +``` + +This writes `olive_requirements.txt` in the current directory. Review and install it: + +```shell +python -m pip install -r olive_requirements.txt +``` + +This file contains dependencies declared by passes and the selected runtime. Check model-loader and +exporter requirements separately; Hugging Face conversion commonly also requires `transformers` and +`optimum`, while Model Builder workflows require the matching `onnxruntime-genai` package. + +Then execute: + +```shell +olive run --config workflow.yaml --log_level 1 +``` + +Validation does not download the model or dataset and cannot guarantee runtime memory, hardware support, +or model/pass semantic compatibility. diff --git a/skills/olive/scripts/inspect_pass.py b/skills/olive/scripts/inspect_pass.py new file mode 100644 index 000000000..edd1004ca --- /dev/null +++ b/skills/olive/scripts/inspect_pass.py @@ -0,0 +1,127 @@ +#!/usr/bin/env python +# ruff: noqa: T201 + +import argparse +import json +from pathlib import Path +from typing import Any + +from olive.hardware import AcceleratorSpec +from olive.package_config import OlivePackageConfig + + +def _serialize(value: Any): + if value is None or isinstance(value, (str, int, float, bool)): + return value + if isinstance(value, Path): + return str(value) + if isinstance(value, (list, tuple, set)): + return [_serialize(item) for item in value] + if isinstance(value, dict): + return {str(key): _serialize(item) for key, item in value.items()} + if hasattr(value, "to_json"): + return _serialize(value.to_json()) + return str(value) + + +def _type_name(annotation: Any) -> str: + if annotation is None: + return "Any" + if getattr(annotation, "__module__", None) == "builtins" and (name := getattr(annotation, "__name__", None)): + return name + return str(annotation).replace("typing.", "") + + +def _matches(values: set[str], requested: str | None) -> bool: + return requested is None or "*" in values or requested in values + + +def _list_passes(package_config: OlivePackageConfig, args: argparse.Namespace): + passes = [] + for module_config in package_config.passes.values(): + if not _matches(module_config.supported_accelerators, args.device): + continue + if not _matches(module_config.supported_providers, args.provider): + continue + if not _matches(module_config.supported_precisions, args.precision): + continue + + passes.append( + { + "name": module_config.module_path.rsplit(".", 1)[-1], + "accelerators": sorted(module_config.supported_accelerators), + "providers": sorted(module_config.supported_providers), + "precisions": sorted(module_config.supported_precisions), + "algorithms": sorted(module_config.supported_algorithms), + "dataset": str(module_config.dataset), + } + ) + + print(json.dumps({"passes": passes, "total": len(passes)}, indent=2)) + + +def _inspect_pass(package_config: OlivePackageConfig, args: argparse.Namespace): + module_config = package_config.get_pass_module_config(args.pass_name) + pass_class = package_config.import_pass_module(args.pass_name) + accelerator = AcceleratorSpec(args.device or "cpu", args.provider or "CPUExecutionProvider") + default_config = pass_class.default_config(accelerator) + + packages = list(module_config.module_dependencies) + for extra in module_config.extra_dependencies: + packages.extend(package_config.extra_dependencies.get(extra, [f"olive-ai[{extra}]"])) + + parameters = {} + for name, parameter in default_config.items(): + parameters[name] = { + "type": _type_name(parameter.type_), + "required": parameter.required, + "default": _serialize(parameter.default_value), + "search_defaults": _serialize(parameter.search_defaults), + "description": parameter.description or "", + } + + result = { + "name": pass_class.__name__, + "module_path": module_config.module_path, + "accelerator": accelerator.to_json(), + "supported_accelerators": sorted(module_config.supported_accelerators), + "supported_providers": sorted(module_config.supported_providers), + "supported_precisions": sorted(module_config.supported_precisions), + "supported_algorithms": sorted(module_config.supported_algorithms), + "supported_quantization_encodings": sorted(module_config.supported_quantization_encodings), + "dataset": str(module_config.dataset), + "packages": sorted(set(packages)), + "parameters": parameters, + } + print(json.dumps(result, indent=2)) + + +def main(): + parser = argparse.ArgumentParser(description="Inspect passes from the active Microsoft Olive installation.") + parser.add_argument("pass_name", nargs="?", help="Pass type to inspect, for example OnnxConversion.") + parser.add_argument("--list", action="store_true", help="List passes instead of inspecting one pass.") + parser.add_argument( + "--device", + help="Filter list output by device; defaults to cpu when inspecting one pass.", + ) + parser.add_argument( + "--provider", + help="Filter list output by provider; defaults to CPUExecutionProvider when inspecting one pass.", + ) + parser.add_argument("--precision", help="Filter list output by supported precision.") + args = parser.parse_args() + + if not args.list and not args.pass_name: + parser.error("pass_name is required unless --list is used") + if not args.list and bool(args.device) != bool(args.provider): + parser.error("--device and --provider must be provided together when inspecting one pass") + + package_config = OlivePackageConfig.load_default_config() + if args.list: + _list_passes(package_config, args) + else: + _inspect_pass(package_config, args) + + +if __name__ == "__main__": + main() diff --git a/skills/olive/scripts/validate_config.py b/skills/olive/scripts/validate_config.py new file mode 100644 index 000000000..e64fa70dd --- /dev/null +++ b/skills/olive/scripts/validate_config.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python +# ruff: noqa: T201 + +import argparse + +from olive.hardware import DEFAULT_CPU_ACCELERATOR, AcceleratorSpec, Device +from olive.hardware.constants import DEVICE_TO_EXECUTION_PROVIDERS +from olive.package_config import OlivePackageConfig +from olive.workflows.run.config import RunConfig +from olive.workflows.run.run import get_required_packages + + +def _accelerator_from_config(run_config: RunConfig) -> AcceleratorSpec: + system = run_config.engine.target or run_config.engine.host + if not system or not system.config or not system.config.accelerators: + return DEFAULT_CPU_ACCELERATOR + + accelerator = system.config.accelerators[0] + providers = accelerator.get_ep_strs() or [] + provider = providers[0] if providers else None + device = accelerator.device + + if device is None and provider: + device = next( + ( + candidate + for candidate, candidate_providers in DEVICE_TO_EXECUTION_PROVIDERS.items() + if provider in candidate_providers + ), + Device.CPU, + ) + + return AcceleratorSpec(device or Device.CPU, provider, accelerator.memory) + + +def _validate_passes( + run_config: RunConfig, + package_config: OlivePackageConfig, + accelerator: AcceleratorSpec, +): + search_enabled = bool(run_config.engine.search_strategy) + validated = [] + + for name, pass_configs in run_config.passes.items(): + for index, pass_config in enumerate(pass_configs): + pass_class = package_config.import_pass_module(pass_config.type) + provided_config = pass_config.config or {} + known_parameters = set(pass_class.default_config(accelerator)) + unknown_parameters = set(provided_config) - known_parameters + if unknown_parameters: + unknown = ", ".join(sorted(unknown_parameters)) + raise ValueError(f"Pass '{name}' has unknown parameters: {unknown}") + + pass_class.get_config_params( + accelerator, + provided_config, + disable_search=not search_enabled, + ) + display_name = name if len(pass_configs) == 1 else f"{name}[{index}]" + validated.append((display_name, pass_class.__name__)) + + return validated + + +def main(): + parser = argparse.ArgumentParser(description="Validate a Microsoft Olive YAML or JSON workflow.") + parser.add_argument("config", help="Path to an Olive .yaml, .yml, or .json workflow.") + args = parser.parse_args() + + package_config = OlivePackageConfig.load_default_config() + run_config = RunConfig.parse_file_or_obj(args.config) + accelerator = _accelerator_from_config(run_config) + passes = _validate_passes(run_config, package_config, accelerator) + required_packages = sorted(get_required_packages(package_config, run_config)) + + print(f"Valid Olive workflow: {args.config}") + print(f"Workflow ID: {run_config.workflow_id}") + print(f"Input model type: {run_config.input_model.type}") + print(f"Target accelerator: {accelerator}") + print("Pass order:") + if passes: + for index, (name, pass_type) in enumerate(passes, start=1): + print(f" {index}. {name}: {pass_type}") + else: + print(" No explicit passes") + print("Declared local packages:") + if required_packages: + for package in required_packages: + print(f" - {package}") + else: + print(" None") + + +if __name__ == "__main__": + main()