Context
Child of #2118. #2054 added user-configurable engine_kwargs, richer inference metrics, Ray fanout behavior, and improved vLLM startup retries to the Nemotron-Parse PDF pipeline.
Required coverage
engine_kwargs passthrough to vllm.LLM, with safe examples such as gpu_memory_utilization and max_num_batched_tokens
- Metrics: prompt/output tokens, output characters, wall time, truncated/empty outputs, retry count, pages/s, and output-tokens/s
- How to aggregate metrics using
TaskPerfUtils
PDFPartitioningStage fanout behavior under Ray Data
- Port-collision retry behavior and what users should do after retries are exhausted
Acceptance criteria
Related PR: #2054
Context
Child of #2118. #2054 added user-configurable
engine_kwargs, richer inference metrics, Ray fanout behavior, and improved vLLM startup retries to the Nemotron-Parse PDF pipeline.Required coverage
engine_kwargspassthrough tovllm.LLM, with safe examples such asgpu_memory_utilizationandmax_num_batched_tokensTaskPerfUtilsPDFPartitioningStagefanout behavior under Ray DataAcceptance criteria
Related PR: #2054