-
Notifications
You must be signed in to change notification settings - Fork 238
Eagle3 demo #3920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Eagle3 demo #3920
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds support for EAGLE3 speculative decoding in the OpenVINO Model Server demos by introducing dedicated documentation, configuration updates, and model export enhancements.
Changes:
- Added comprehensive EAGLE3 setup documentation with model preparation and deployment instructions
- Added
datasetsdependency to support EAGLE3 model export requirements - Enhanced export script with EAGLE3-specific flags and draft device configuration
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| demos/continuous_batching/speculative_decoding/README.md | Added EAGLE3 section with model preparation steps and deployment instructions for Qwen3-8B models |
| demos/common/export_models/requirements.txt | Added datasets package dependency for EAGLE3 support |
| demos/common/export_models/export_model.py | Added --draft_eagle3 flag and draft device configuration in template |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if args["draft_eagle3"]: | ||
| print("Using eagle3 option for the draft model export") | ||
| additional_options += " --eagle3 --task text-generation-with-past" | ||
| optimum_command = "optimum-cli export openvino --model {} --weight-format {} --trust-remote-code {} {}".format(draft_source_model, precision, additional_options, draft_llm_model_path) |
Copilot
AI
Jan 22, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The format string placeholders are misaligned with the arguments. The order should be: draft_source_model, precision, draft_llm_model_path, and additional_options should be inserted before draft_llm_model_path in the command string, not as a separate placeholder.
| optimum_command = "optimum-cli export openvino --model {} --weight-format {} --trust-remote-code {} {}".format(draft_source_model, precision, additional_options, draft_llm_model_path) | |
| optimum_command = ( | |
| "optimum-cli export openvino " | |
| f"--model {draft_source_model} " | |
| f"--weight-format {precision} " | |
| "--trust-remote-code" | |
| f"{additional_options} " | |
| f"{draft_llm_model_path}" | |
| ) |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
No description provided.