Skip to content

refactor(moe): remove enable_deepep, switch failing ep recipes to hybridep #2630

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
HuiyingLi merged 8 commits into from
Jun 19, 2026
Merged

refactor(moe): remove enable_deepep, switch failing ep recipes to hybridep #2630

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
7ef87f3
refactor(moe): remove enable_deepep, switch failing ep recipes to hyb…
akoumpa Jun 18, 2026
8c83a5b
refactor(moe): use deepep/hybridep over torch for migrated EP recipes
akoumpa Jun 18, 2026
c29d447
fix(moe): switch failing ep_size>8 deepep recipes to hybridep
akoumpa Jun 18, 2026
4091051
ci(nemotron): run nemotron_ultra_v3_squad on 8 nodes
akoumpa Jun 18, 2026
1f2c43d
ci(moe): add ci sections to ep_size>8 recipes that lacked one
akoumpa Jun 18, 2026
31af3b2
fix(moe): switch VLM ep_size>8 deepep recipes to hybridep
akoumpa Jun 18, 2026
4d5abb1
fix(ci): move ci section to end of minimax_m3_vl_sft_ep32pp4
akoumpa Jun 18, 2026
70fb43e
fix(ci): correct recipe_owner on the added ci sections per review
akoumpa Jun 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/guides/vlm/nemotron-omni.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: false
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
3 changes: 2 additions & 1 deletion docs/guides/vlm/nemotron-omni.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,8 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: false
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
4 changes: 2 additions & 2 deletions examples/audio_finetune/qwen3_omni_asr/ami_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
4 changes: 2 additions & 2 deletions examples/audio_finetune/qwen3_omni_asr/multi_en_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
2 changes: 1 addition & 1 deletion examples/llm_benchmark/glm/glm_4.7_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ model:
rms_norm: te
rope_fusion: true
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: true
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_benchmark/qwen/qwen3_next_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ model:
rms_norm: te
rope_fusion: true
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: true
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/deepseek_v32/deepseek_v32_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_packed_sequence_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion ...les/llm_finetune/deepseek_v4/deepseek_v4_pro_hellaswag_all_tilelang_pp8_ep64_20steps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion ...seek_v4/deepseek_v4_pro_packed_sequence_hellaswag_all_tilelang_pp8_ep64_1024_20steps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 3 additions & 0 deletions examples/llm_finetune/glm/glm_4.7_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ rng:
model:
_target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
pretrained_model_name_or_path: zai-org/GLM-4.7
backend:
_target_: nemo_automodel.components.models.common.BackendConfig
dispatcher: hybridep

checkpoint:
enabled: false
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/glm/glm_5.1_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/glm/glm_5.1_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/glm/glm_5_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_full_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_full_sft_chat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
dispatcher: torch

@HuiyingLi HuiyingLi Jun 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intended?

@akoumpa akoumpa Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, intentional. This PEFT recipe is single-GPU (ep_size: null, dp_size: 1, nproc_per_node: 1), so there is no expert parallelism — deepep/hybridep are multi-rank all-to-all dispatchers and do not apply, so torch is correct here. (The *_full_sft* variants are ep_size: 8 and did move to deepep.)

distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 1
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_peft_packing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
dispatcher: torch

@HuiyingLi HuiyingLi Jun 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

@akoumpa akoumpa Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same — customizer_gpt_oss_peft_packing is also single-GPU (ep_size: null, nproc_per_node: 1), so torch is intentional (no EP to dispatch).

distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 1
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/hy_v3/hy3_preview_deepep_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ model:
linear: torch
rms_norm: torch_fp32
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
gate_precision: float32
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/ling/ling_flash_2_0_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ model:
linear: torch
rms_norm: torch_fp32
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/mimo_v2_flash/mimo_v2_flash_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
gate_precision: float32
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.1_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.5_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ model:
rms_norm: torch_fp32
rope_fusion: true
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.7_hellaswag_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: true
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.7_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ model:
rms_norm: torch_fp32
rope_fusion: false
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_full_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_full_sft_chat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: null
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_peft_packing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: null
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/nemotron/nemotron_ultra_v3_hellaswag_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: te
rms_norm: te
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_fsdp_optimizations: false

Expand Down
5 changes: 5 additions & 0 deletions examples/llm_finetune/nemotron/nemotron_ultra_v3_hellaswag_peft_gb200.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,8 @@ lr_scheduler:
lr_decay_style: cosine
lr_warmup_steps: 10
min_lr: 1.0e-5

ci:
# pp_size(1) * ep_size(16) = 16 GPUs => 4 nodes (4 GB200/node).
recipe_owner: adil-a
nodes: 4
5 changes: 5 additions & 0 deletions examples/llm_finetune/nemotron/nemotron_ultra_v3_squad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ lr_scheduler:
lr_warmup_steps: 10
min_lr: 1.0e-6

ci:
# pp_size(1) * ep_size(64) = 64 GPUs => 8 nodes (8 H100/node).
recipe_owner: adil-a
nodes: 8

# wandb:
# project: my-project
# entity: my-entity
Expand Down
3 changes: 3 additions & 0 deletions examples/llm_finetune/qwen/qwen3_next_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ rng:
model:
_target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
pretrained_model_name_or_path: Qwen/Qwen3-Next-80B-A3B-Instruct
backend:
_target_: nemo_automodel.components.models.common.BackendConfig
dispatcher: hybridep

checkpoint:
enabled: false
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/stepfun/step_3.5_flash_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
5 changes: 5 additions & 0 deletions examples/llm_pretrain/deepseekv3_pretrain.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,3 +152,8 @@ lr_scheduler:
lr_decay_style: cosine
lr_warmup_steps: 500
min_lr: 0.0

ci:
# pp_size(8) * ep_size(16) = 128 GPUs => 16 nodes (8 H100/node).
recipe_owner: hemildesai
nodes: 16
3 changes: 2 additions & 1 deletion examples/vlm_finetune/kimi/kimi25vl_medpix.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/vlm_finetune/kimi/kimi2vl_cordv2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
7 changes: 6 additions & 1 deletion examples/vlm_finetune/minimax_m3/minimax_m3_vl_sft_ep32pp4.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ model:
rms_norm: torch_fp32
rope_fusion: false
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down Expand Up @@ -141,3 +141,8 @@ wandb:
entity: <your_wandb_entity>
name: <your_wandb_name>
dir: <your_wandb_save_dir>

ci:
# pp_size(4) * ep_size(32) = 128 GPUs => 16 nodes (8 H100/node).
recipe_owner: athitten
nodes: 16
Loading
Loading