[AMD] Add algo-config option in Quark#2579
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
This PR extends Olive’s Quark Torch quantization pass to accept user-supplied algorithm configuration (“algo_configs”) from a recipe and forward it into Quark’s LLMTemplate.get_config(...), enabling AWQ quantization for model architectures that Quark doesn’t yet ship built-in configs for (e.g., qwen3).
Changes:
- Add an optional
algo_configspass parameter toQuarkQuantization(torch path) to accept per-algorithm config dictionaries from recipes. - In the torch quantization runner, translate recipe-provided AWQ config dicts into Quark’s
AWQConfigobjects and pass them intotemplate.get_config(algo_configs=...). - Remove the explicit
processor=Noneargument from the calibration dataloader call.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| olive/passes/quark_quantizer/torch/quark_torch_quantization.py | Builds algo_configs (currently AWQ-only) from recipe JSON into AWQConfig and forwards it to LLMTemplate.get_config; removes processor=None from the calibration dataloader call. |
| olive/passes/quark_quantizer/quark_quantization.py | Adds the optional algo_configs PassConfigParam for the torch pathway so recipes can supply algorithm configs. |
|
@thpereir, @thiagocrepaldi - Please help review the quark changes. |
thiagocrepaldi
left a comment
There was a problem hiding this comment.
Code itself looks OK. Are you planning to add tests? any extra documentation planned, with maybe a real usage example?
|
Adds an algo_configs parameter to the QuarkQuantization pass (torch path), letting a recipe pass a custom quantization algorithm config (e.g. AWQ scaling_layers / model_decoder_layers) through to LLMTemplate.get_config(algo_configs=...).
This is needed for architectures that Quark 0.12.post1 does not ship a built-in algorithm config for, e.g. qwen3, which is absent from Quark's AWQ_MAP and otherwise raises NotImplementedError: No built-in awq configuration is available for the 'qwen3' architecture. With this change, the recipe can supply the config (including scaling_layers: [] for auto-detection), unblocking AWQ quantization for such models.
The parameter is optional and defaults to None; existing recipes and architectures with built-in configs are unaffected.