docs(inference_guide): validated MiniMax-M2.5 (W8A8) on Ascend 910B3 (16-card DP2×TP8, plain agg baseline)#287
Open
EdisonSu768 wants to merge 1 commit into
Open
docs(inference_guide): validated MiniMax-M2.5 (W8A8) on Ascend 910B3 (16-card DP2×TP8, plain agg baseline)#287EdisonSu768 wants to merge 1 commit into
EdisonSu768 wants to merge 1 commit into
Conversation
…(16-card DP2×TP8, plain agg baseline) MiniMax-M2.5-w8a8-QuaRot (minimax_m2: 256-expert MoE + 62 layers full attention, 230B-A10B, ~230 GB / 71 shards) served cross-node DP2×TP8+EP16 on 2x8x910B3 64 GB through the InferNex surface, on the release-pinned vLLM-Ascend v0.23.0 engine. Deliberate production config = plain `agg` baseline (FULL_DECODE_ONLY graph + local prefix cache), NO mooncake KV store and NO speculative decoding: because this model is "fast" (~10B activated + full attention + W8A8), both add-ons were measured to lose — eagle3 is a batch-throughput net loss and 500s on 27k-prefill agent requests (built-in MTP unusable: QuaRot weights ship no MTP tensors), and the mooncake store is -13..14% since remote KV fetch >= local recompute. The opposite of the DeepSeek-V4-Flash W8A8 verdict, where the store wins. Headline numbers are the production v0.23.0 config, conc 8/16/32 sweep via MaaS (480 req/tier, 0 error): S1 8k conc32 TPS 36,012 / decode 564 tok/s, S2 17.5k conc32 TPS 34,906 / decode 247 tok/s. Adds the eagle3/mooncake config-decision table and a real coding-agent validation section (OpenCode 6/4/14, Pi 7/8/18; 3,123 tool executions, 0 </parameter> tail pollution; usable for POC, not yet production-stable). Weights staged on a node-local PV/PVC per node (validated path); an OCI ModelCar tar is documented as the cross-environment distribution alternative (>200 GB, excluded from the Cosign table like DeepSeek W8A8). Data from the model-auto validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deploying alauda-ai with
|
| Latest commit: |
4d042a5
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://18bb12f2.alauda-ai.pages.dev |
| Branch Preview URL: | https://docs-minimax-m2-5-w8a8.alauda-ai.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an inference-guide page + deployment manifest for MiniMax-M2.5 (W8A8), following the same shape as the DeepSeek-V4-Flash (W8A8) entry (#283). Data from the
model-autovalidation on 贵安 Ascend 910B3.What's validated
Eco-Tech/MiniMax-M2.5-w8a8-QuaRot—minimax_m2(256-expert MoE, 8 experts/tok, no shared expert; 62 layers full attention, GQA 48/8; native MTP head), 230B-A10B, W8A8 QuaRot, ~230 GB / 71 shards.quay.io/ascend/vllm-ascend:v0.23.0-openeuler(vLLM 0.23.0 + #11505 streaming tool-call fix).Key point — production config is a plain
aggbaselineUnlike DeepSeek-V4-Flash (W8A8), MiniMax ships no mooncake KV store and no speculative decoding. Because the model is "fast" (~10B activated + full attention + W8A8), both add-ons were measured to lose:
What the config keeps is the net-win pair:
FULL_DECODE_ONLYdecode graph + the engine's local prefix cache.Headline benchmark (production v0.23.0, conc 8/16/32 sweep via MaaS, 480 req/tier, 0 error)
Plus an eagle3/mooncake config-decision table and a real coding-agent validation section (OpenCode 6/4/14, Pi 7/8/18 across conc 1/8/32; 3,123 tool executions with 0
</parameter>tail pollution; usable for POC, not yet production-stable).Files
assets/minimax-m2.5-w8a8/minimax-m2.5-w8a8-agg-llmisvc.yaml— self-contained InferNex manifest (LLMISVC leader + worker + hermes-router; node-local PV/PVC weight staging, OCI ModelCar documented as the cross-env alternative).minimax-m2.5-w8a8.mdx— new model page.index.mdx— validated-models table, narrative, runtime-images table (v0.23.0 release row), benchmark-scenarios note, and Cosign note (OCI-tar models excluded from signing).doom lintpasses clean.🤖 Generated with Claude Code