fix(parallelizer): resolve NemotronH decoder blocks for Nemotron-V3#2638
Merged
Conversation
…on-V3 layout Two classes share the name NemotronHForCausalLM: the HF model keeps its decoder blocks at model.backbone.layers (a ModuleList), while the native Nemotron-V3 model (NemotronV3Model) keeps them at model.model.layers (a ModuleDict). The parallelizer registry routes by class name, so the native model reached NemotronHParallelizationStrategy, which hard-coded model.backbone.layers -> AttributeError: 'NemotronHForCausalLM' object has no attribute 'backbone' (nemotron_nano_v3_singlegpu_lora, single GPU LoRA). - Add _nemotronh_decoder_blocks() resolving blocks for both layouts (iterating ModuleDict values); use it in the strategy incl. the activation-checkpointing write-back (by ModuleDict key / ModuleList index). - Add model.layers to the NemotronHForCausalLM entry in LLM_MODEL_CLS_TO_LAYERS (_reduce_attrs already skips the absent backbone.layers via getattr). - Add regression tests for both layouts. Fixes AM-448. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
akoumpa
added
the
r0.5.0
akoumpa
changed the title
Contributor
Author
|
/ok to test 3b32a53 |
HuiyingLi
reviewed
Jun 19, 2026
HuiyingLi
approved these changes
Jun 19, 2026
akoumpa
added a commit
that referenced
this pull request
Jun 19, 2026
…-V3 (2638)` into `r0.5.0` (#2642) fix(parallelizer): resolve NemotronH decoder blocks for Nemotron-V3 (#2638) fix(parallelizer): resolve NemotronH decoder blocks for native Nemotron-V3 layout Two classes share the name NemotronHForCausalLM: the HF model keeps its decoder blocks at model.backbone.layers (a ModuleList), while the native Nemotron-V3 model (NemotronV3Model) keeps them at model.model.layers (a ModuleDict). The parallelizer registry routes by class name, so the native model reached NemotronHParallelizationStrategy, which hard-coded model.backbone.layers -> AttributeError: 'NemotronHForCausalLM' object has no attribute 'backbone' (nemotron_nano_v3_singlegpu_lora, single GPU LoRA). - Add _nemotronh_decoder_blocks() resolving blocks for both layouts (iterating ModuleDict values); use it in the strategy incl. the activation-checkpointing write-back (by ModuleDict key / ModuleList index). - Add model.layers to the NemotronHForCausalLM entry in LLM_MODEL_CLS_TO_LAYERS (_reduce_attrs already skips the absent backbone.layers via getattr). - Add regression tests for both layouts. Fixes AM-448. Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com> Signed-off-by: NeMo Bot <nemo-bot@nvidia.com> Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Fixes
AttributeError: 'NemotronHForCausalLM' object has no attribute 'backbone'that crashes FSDP2 parallelization for the native Nemotron-V3 model (e.g.nemotron_nano_v3_singlegpu_lora).Two distinct classes share the name
NemotronHForCausalLM:model.backbone.layers(annn.ModuleList), whileNemotronV3Model) keeps them atmodel.model.layers(annn.ModuleDictkeyed"0".."N-1").NemotronHParallelizationStrategy(selected by the class-name string) and theLLM_MODEL_CLS_TO_LAYERSregistry assumed the HF layout, so the native model raised atlayers = model.backbone.layers.Changelog
_nemotronh_decoder_blocks(model)resolving the decoder-block container for both layouts (backbone.layersModuleList vsmodel.layersModuleDict) and returning the ordered block list.NemotronHParallelizationStrategy.parallelize(TP iteration, CP setup, FSDP wrap), and make the activation-checkpointing rewrap write back into the real container by ModuleDict key / ModuleList index."model.layers"to theNemotronHForCausalLMentry ofLLM_MODEL_CLS_TO_LAYERS._reduce_attrsalready skips the absentbackbone.layersviagetattr(..., None), so both paths are covered and neither raises.TestNemotronHLayoutResolution) for both layouts (helper +_extract_model_layers).Before your PR is "Ready for review"
tests/unit_tests/distributed/test_parallelization_strategies.py::TestNemotronHLayoutResolution(10 NemotronH tests pass locally)Additional Information
nemotron_nano_v3_singlegpu_lora(single-GPU LoRA), surfaced in nemo-ci job 343744282.tp>1/ep>1, this strategy's TP plan does not shard MoE experts; proper multi-GPU native-v3 parallelism should use the MoE parallelizer. This PR fixes the single-GPU / DP-only crash.