[dynamo] Add NVIDIA Dynamo project for Forge

[thameem-abbas]

Add projects/dynamo/ — a Forge project for deploying and benchmarking NVIDIA Dynamo inference workloads on OpenShift. Follows the llm_d project pattern but targets Dynamo's own operator stack (DynamoGraphDeployment CRD, etcd/NATS, EPP-based routing) instead of RHOAI/KServe.

Includes:

• CI/CLI entrypoints with standard Forge phases
• Config schema for Dynamo Helm deploy, deployment profiles (agg/disagg), workloads, and model cache
• DynamoGraphDeployment manifest renderer (aggregated + disaggregated)
• Toolbox commands: deploy_dynamo_platform, capture_dynamo_state, cleanup_dynamo_resources
• Runtime config with RunSpec matrix expansion
• Presets (smoke, disagg-smoke, full)
• Unit tests for manifest rendering and config utilities

Reuses GPU operator, NFD, model cache, guidellm benchmarks, and DSL framework from existing Forge projects.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2dcc556e-d5c6-4844-84fc-2667b76aeb34

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thameem-abbas]

Design: Dynamo Project for Forge

Why a Separate Project

Dynamo uses a fundamentally different deployment stack from llm_d:

	llm_d	dynamo
Operator	RHOAI + KServe	Dynamo operator (Helm chart)
CRD	LLMInferenceService	DynamoGraphDeployment
Infrastructure	Gateway API (DSG)	etcd + NATS (optional) + EPP
Routing	KServe gateway	EPP (Endpoint Picker Plugin) → Inference Gateway
Serving modes	Single pool	Aggregated + Disaggregated (prefill/decode split)

Shared code accessed via cross-project imports — no duplication.

Architecture

prepare phase (cluster-wide, run once)
  ├── verify_oc_access / verify_cluster_version
  ├── prepare_nfd()                          ← reuse projects.cluster
  ├── prepare_gpu_operator()                 ← reuse projects.gpu_operator  
  ├── deploy_dynamo_platform()               ← NEW: helm install (operator + etcd + optional NATS)
  └── wait_for_dynamo_crds()                 ← NEW

prepare phase (per RunSpec)
  ├── ensure_test_namespace()
  ├── prepare_model_cache()                  ← reuse projects.kserve
  └── verify_gpu_nodes()

test phase (per RunSpec)
  ├── render DynamoGraphDeployment manifest  ← NEW: agg or disagg topology
  ├── oc apply DynamoGraphDeployment         ← NEW
  ├── wait_for_dynamo_ready()                ← NEW: poll pods
  ├── discover_endpoint()                    ← NEW: from HTTPRoute/Service
  ├── run_smoke_request()                    ← reuse projects.guidellm
  └── run_guidellm_benchmark()               ← reuse projects.guidellm

Infrastructure Notes

• etcd — required for service discovery and configuration in K8s deployments
• NATS — optional. Default event plane transport for distributed (etcd/kubernetes) backends since Dynamo 1.1.0, but can be replaced with ZMQ (DYN_EVENT_PLANE=zmq) for single-node setups. Helm chart controls via global.nats.install
• EPP — required for production routing. Implements Gateway API InferencePool extension for KV-cache-aware request routing

Reused Code

Component	Source
GPU operator / NFD setup	projects.gpu_operator, projects.cluster
Model cache PVC	projects.kserve.toolbox.prepare_hf_model_cache
Smoke request + benchmarks	projects.guidellm
DSL framework, config system	projects.core
RunSpec matrix expansion	Adapted from llm_d/runtime_config.py
CI/CLI entrypoints, Fournos	Standard Forge patterns

New Components

• render_graph_deployment.py — builds DynamoGraphDeployment manifests from config. Handles aggregated (single worker pool) and disaggregated (separate prefill/decode workers + EPP) topologies. Modeled after llm_d's render_inference_service.py.
• deploy_dynamo_platform/ toolbox — Helm install/upgrade for the Dynamo platform chart (operator + etcd + optional NATS).
• capture_dynamo_state/ toolbox — captures DynamoGraphDeployments, DynamoComponentDeployments, operator/worker/EPP logs, etcd/NATS status.
• Deployment profiles — aggregated and disaggregated with configurable backend framework (vllm/sglang/trtllm), replicas, tensor parallelism, and vllm args.
• Dynamo-specific SCCs — dynamo-frontend-scc (fsGroup RunAsAny) and dynamo-backend-scc (fsGroup RunAsAny + IPC_LOCK + SYS_RAWIO) for OpenShift.

Presets

Preset	Model	Mode	Benchmark
smoke	Qwen3-0.6B	aggregated	short (2min)
disagg-smoke	Qwen3-0.6B	disaggregated	short
full	Llama-3-70B, Qwen3-32B	agg + disagg	concurrent-1k-1k, multi-turn

Testing Status

• ✅ Unit tests pass (5/5 manifest rendering tests)
• ✅ Project auto-discovered by run_cli
• ✅ DynamoGraphDeployment deployed on Janus (OCP 4.21, H200s) — worker 2/2 + EPP 1/1 running
• ⏳ End-to-end HTTP smoke test blocked on Istio CA mismatch (cluster-level issue, not project code)

Next Steps

1. Fix Istio CA on test cluster → complete end-to-end smoke test
2. Add Caliper postprocess plugin for Dynamo-specific metrics
3. Onboard to Fournos hub for CI automation (TBD if needed)

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign harshith-umesh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment