Skip to content

cp: refactor(moe): remove enable_deepep, switch failing ep recipes to hybridep (2630) into r0.5.0 #2644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
akoumpa merged 1 commit into from
Jun 19, 2026
Merged

cp: refactor(moe): remove enable_deepep, switch failing ep recipes to hybridep (2630) into r0.5.0 #2644

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/guides/vlm/nemotron-omni.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,8 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: false
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
4 changes: 2 additions & 2 deletions examples/audio_finetune/qwen3_omni_asr/ami_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
4 changes: 2 additions & 2 deletions examples/audio_finetune/qwen3_omni_asr/multi_en_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
2 changes: 1 addition & 1 deletion examples/llm_benchmark/glm/glm_4.7_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ model:
rms_norm: te
rope_fusion: true
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: true
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_benchmark/qwen/qwen3_next_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ model:
rms_norm: te
rope_fusion: true
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: true
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/deepseek_v32/deepseek_v32_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_hellaswag_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/deepseek_v4/deepseek_v4_flash_packed_sequence_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion ...les/llm_finetune/deepseek_v4/deepseek_v4_pro_hellaswag_all_tilelang_pp8_ep64_20steps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion ...seek_v4/deepseek_v4_pro_packed_sequence_hellaswag_all_tilelang_pp8_ep64_1024_20steps.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 3 additions & 0 deletions examples/llm_finetune/glm/glm_4.7_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ rng:
model:
_target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
pretrained_model_name_or_path: zai-org/GLM-4.7
backend:
_target_: nemo_automodel.components.models.common.BackendConfig
dispatcher: hybridep

checkpoint:
enabled: false
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/glm/glm_5.1_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/glm/glm_5.1_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/glm/glm_5_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_full_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_full_sft_chat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
dispatcher: torch
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 1
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/gpt_oss/customizer_gpt_oss_peft_packing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
dispatcher: torch
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 1
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/hy_v3/hy3_preview_deepep_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ model:
linear: torch
rms_norm: torch_fp32
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
gate_precision: float32
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/ling/ling_flash_2_0_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ model:
linear: torch
rms_norm: torch_fp32
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/mimo_v2_flash/mimo_v2_flash_hellaswag.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
dispatcher: deepep
dispatcher: hybridep
experts: torch_mm
gate_precision: float32
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.1_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.5_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ model:
rms_norm: torch_fp32
rope_fusion: true
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.7_hellaswag_lora.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: true
dispatcher: deepep
dispatcher: hybridep
experts: gmm
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/minimax_m2/minimax_m2.7_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ model:
rms_norm: torch_fp32
rope_fusion: false
experts: torch_mm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_full_sft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_full_sft_chat.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: 8
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: null
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/nemotron/customizer_nemotron_nano_peft_packing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ model:
attn_implementation: sdpa
backend:
_target_: nemo_automodel.components.models.common.utils.BackendConfig
enable_deepep: false
experts: gmm
dispatcher: deepep
distributed:
_target_: nemo_automodel.components.distributed.fsdp2.FSDP2Manager
dp_size: null
Expand Down
2 changes: 1 addition & 1 deletion examples/llm_finetune/nemotron/nemotron_ultra_v3_hellaswag_peft.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ model:
linear: te
rms_norm: te
experts: gmm
dispatcher: deepep
dispatcher: hybridep
fake_balanced_gate: false
enable_fsdp_optimizations: false

Expand Down
5 changes: 5 additions & 0 deletions examples/llm_finetune/nemotron/nemotron_ultra_v3_hellaswag_peft_gb200.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,8 @@ lr_scheduler:
lr_decay_style: cosine
lr_warmup_steps: 10
min_lr: 1.0e-5

ci:
# pp_size(1) * ep_size(16) = 16 GPUs => 4 nodes (4 GB200/node).
recipe_owner: adil-a
nodes: 4
5 changes: 5 additions & 0 deletions examples/llm_finetune/nemotron/nemotron_ultra_v3_squad.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,11 @@ lr_scheduler:
lr_warmup_steps: 10
min_lr: 1.0e-6

ci:
# pp_size(1) * ep_size(64) = 64 GPUs => 8 nodes (8 H100/node).
recipe_owner: adil-a
nodes: 8

# wandb:
# project: my-project
# entity: my-entity
Expand Down
3 changes: 3 additions & 0 deletions examples/llm_finetune/qwen/qwen3_next_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ rng:
model:
_target_: nemo_automodel.NeMoAutoModelForCausalLM.from_pretrained
pretrained_model_name_or_path: Qwen/Qwen3-Next-80B-A3B-Instruct
backend:
_target_: nemo_automodel.components.models.common.BackendConfig
dispatcher: hybridep

checkpoint:
enabled: false
Expand Down
3 changes: 2 additions & 1 deletion examples/llm_finetune/stepfun/step_3.5_flash_hellaswag_pp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
5 changes: 5 additions & 0 deletions examples/llm_pretrain/deepseekv3_pretrain.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,3 +152,8 @@ lr_scheduler:
lr_decay_style: cosine
lr_warmup_steps: 500
min_lr: 0.0

ci:
# pp_size(8) * ep_size(16) = 128 GPUs => 16 nodes (8 H100/node).
recipe_owner: hemildesai
nodes: 16
3 changes: 2 additions & 1 deletion examples/vlm_finetune/kimi/kimi25vl_medpix.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ model:
linear: torch
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
3 changes: 2 additions & 1 deletion examples/vlm_finetune/kimi/kimi2vl_cordv2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: true
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
4 changes: 2 additions & 2 deletions examples/vlm_finetune/qwen3/qwen3_omni_moe_30b_te_deepep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
3 changes: 2 additions & 1 deletion examples/vlm_finetune/qwen3/qwen3_vl_moe_235b.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ model:
linear: te
rms_norm: torch_fp32
rope_fusion: false
enable_deepep: true
experts: gmm
dispatcher: hybridep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true
enable_fsdp_optimizations: true
Expand Down
4 changes: 2 additions & 2 deletions examples/vlm_finetune/qwen3/qwen3_vl_moe_30b_neat_packing.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ model:
linear: te
rms_norm: te
rope_fusion: false
experts: te
enable_deepep: true
experts: gmm
dispatcher: deepep
fake_balanced_gate: false
enable_hf_state_dict_adapter: true

Expand Down
Loading
Loading