Skip to content

[AMD] Add algo-config option in Quark#2579

Merged
xiaoyu-work merged 2 commits into
microsoft:mainfrom
poganesh:8D-MS
Jul 24, 2026
Merged

[AMD] Add algo-config option in Quark#2579
xiaoyu-work merged 2 commits into
microsoft:mainfrom
poganesh:8D-MS

Conversation

@poganesh

Copy link
Copy Markdown
Contributor
  • Adds an algo_configs parameter to the QuarkQuantization pass (torch path), letting a recipe pass a custom quantization algorithm config (e.g. AWQ scaling_layers / model_decoder_layers) through to LLMTemplate.get_config(algo_configs=...).

  • This is needed for architectures that Quark 0.12.post1 does not ship a built-in algorithm config for, e.g. qwen3, which is absent from Quark's AWQ_MAP and otherwise raises NotImplementedError: No built-in awq configuration is available for the 'qwen3' architecture. With this change, the recipe can supply the config (including scaling_layers: [] for auto-detection), unblocking AWQ quantization for such models.

  • The parameter is optional and defaults to None; existing recipes and architectures with built-in configs are unaffected.

Copilot AI review requested due to automatic review settings July 21, 2026 01:37
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Olive’s Quark Torch quantization pass to accept user-supplied algorithm configuration (“algo_configs”) from a recipe and forward it into Quark’s LLMTemplate.get_config(...), enabling AWQ quantization for model architectures that Quark doesn’t yet ship built-in configs for (e.g., qwen3).

Changes:

  • Add an optional algo_configs pass parameter to QuarkQuantization (torch path) to accept per-algorithm config dictionaries from recipes.
  • In the torch quantization runner, translate recipe-provided AWQ config dicts into Quark’s AWQConfig objects and pass them into template.get_config(algo_configs=...).
  • Remove the explicit processor=None argument from the calibration dataloader call.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
olive/passes/quark_quantizer/torch/quark_torch_quantization.py Builds algo_configs (currently AWQ-only) from recipe JSON into AWQConfig and forwards it to LLMTemplate.get_config; removes processor=None from the calibration dataloader call.
olive/passes/quark_quantizer/quark_quantization.py Adds the optional algo_configs PassConfigParam for the torch pathway so recipes can supply algorithm configs.

@poganesh

Copy link
Copy Markdown
Contributor Author

@thpereir, @thiagocrepaldi - Please help review the quark changes.

@thiagocrepaldi thiagocrepaldi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code itself looks OK. Are you planning to add tests? any extra documentation planned, with maybe a real usage example?

Comment thread test/passes/quark_quantizer/test_quark_torch_quantization.py Fixed
Comment thread test/passes/quark_quantizer/test_quark_torch_quantization.py Fixed
Comment thread test/passes/quark_quantizer/test_quark_torch_quantization.py Fixed
Comment thread test/passes/quark_quantizer/test_quark_torch_quantization.py Fixed
@poganesh

Copy link
Copy Markdown
Contributor Author

Code itself looks OK. Are you planning to add tests? any extra documentation planned, with maybe a real usage example?

@thiagocrepaldi

  • The new Quark 0.12 torch path had no existing tests (only the ONNX path in test_quark_onnx_quantization.py), so I added a new test_quark_torch_quantization.py covering the algo_configs handling introduced in the PR.

  • The primary use case is olive-recipes (e.g. qwen3, which Quark has no built-in AWQ config for). detailed config semantics are deferred to the Quark docs, in the olive-recipes README.

@xiaoyu-work
xiaoyu-work merged commit f12401e into microsoft:main Jul 24, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants