Skip to content

Conversation

@mzegla
Copy link
Collaborator

@mzegla mzegla commented Jan 22, 2026

No description provided.

Copilot AI review requested due to automatic review settings January 22, 2026 16:16
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for EAGLE3 speculative decoding in the OpenVINO Model Server demos by introducing dedicated documentation, configuration updates, and model export enhancements.

Changes:

  • Added comprehensive EAGLE3 setup documentation with model preparation and deployment instructions
  • Added datasets dependency to support EAGLE3 model export requirements
  • Enhanced export script with EAGLE3-specific flags and draft device configuration

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
demos/continuous_batching/speculative_decoding/README.md Added EAGLE3 section with model preparation steps and deployment instructions for Qwen3-8B models
demos/common/export_models/requirements.txt Added datasets package dependency for EAGLE3 support
demos/common/export_models/export_model.py Added --draft_eagle3 flag and draft device configuration in template

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

if args["draft_eagle3"]:
print("Using eagle3 option for the draft model export")
additional_options += " --eagle3 --task text-generation-with-past"
optimum_command = "optimum-cli export openvino --model {} --weight-format {} --trust-remote-code {} {}".format(draft_source_model, precision, additional_options, draft_llm_model_path)
Copy link

Copilot AI Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format string placeholders are misaligned with the arguments. The order should be: draft_source_model, precision, draft_llm_model_path, and additional_options should be inserted before draft_llm_model_path in the command string, not as a separate placeholder.

Suggested change
optimum_command = "optimum-cli export openvino --model {} --weight-format {} --trust-remote-code {} {}".format(draft_source_model, precision, additional_options, draft_llm_model_path)
optimum_command = (
"optimum-cli export openvino "
f"--model {draft_source_model} "
f"--weight-format {precision} "
"--trust-remote-code"
f"{additional_options} "
f"{draft_llm_model_path}"
)

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants