Skip to content

[Docs] Document Nemotron-Parse inference metrics, engine tuning, and Ray fanout #2128

Description

@lbliii

Context

Child of #2118. #2054 added user-configurable engine_kwargs, richer inference metrics, Ray fanout behavior, and improved vLLM startup retries to the Nemotron-Parse PDF pipeline.

Required coverage

  • engine_kwargs passthrough to vllm.LLM, with safe examples such as gpu_memory_utilization and max_num_batched_tokens
  • Metrics: prompt/output tokens, output characters, wall time, truncated/empty outputs, retry count, pages/s, and output-tokens/s
  • How to aggregate metrics using TaskPerfUtils
  • PDFPartitioningStage fanout behavior under Ray Data
  • Port-collision retry behavior and what users should do after retries are exhausted

Acceptance criteria

  • Nemotron-Parse concept/tutorial/API content reflects all new fields
  • A tuning example and a metrics-inspection example are included
  • Backend-specific behavior is clearly labeled
  • Defaults are checked against current code
  • Related benchmark guidance is linked without exposing internal-only infrastructure
  • Fern checks pass

Related PR: #2054

Metadata

Metadata

Assignees

Labels

documentationImprovements or additions to documentation

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions