feat(openai): capture service_tier in model_parameters

[hassiebp]

Summary

Implements LFE-10828: capture the pricing-relevant service_tier attribute in the OpenAI monkeypatch integration (langfuse/openai.py).

• Request side: service_tier is extracted from kwargs in _get_langfuse_data_from_kwargs (guarding against openai NotGiven sentinels, like the other params) and included in model_parameters only when actually set. Applies to chat completions and the Responses API; embeddings are untouched since they do not support service_tier.
• Response side: _get_langfuse_data_from_default_response now also returns the response's service_tier, which is merged into the request-side model_parameters on the generation update. The response value overrides the request value — OpenAI returns the tier that actually processed the request (e.g. when the request says "auto"), and the actually-used tier is the billing-relevant one.
• Merge semantics: generation.update(model_parameters=...) replaces the whole serialized attribute (it does not merge), so the update merges service_tier into the request-side dict (_merge_service_tier_into_model_parameters) instead of clobbering temperature, top_p, etc. When the response carries no service_tier, the update omits model_parameters entirely, leaving the request-side value intact.
• Streaming: chat-completions streaming reads service_tier from the chunks; Responses API streaming reads it from the final response object emitted by the stream. The request-side model_parameters are threaded through both stream instrumentation paths (openai.Stream/AsyncStream hooks and the LangfuseResponseGenerator{Sync,Async} fallbacks) so the finalized update can merge.
• Sync and async client paths share these helpers; both are covered and tested.

Tests

New unit tests in tests/unit/test_openai.py (written first, confirmed failing, then implemented):

• request kwarg captured into model_parameters
• NOT_GIVEN / absent kwarg leaves the key absent
• response service_tier overrides the request value while preserving other request-side model parameters (sync + async)
• streaming chat path captures service_tier from chunks (sync + async)
• embeddings model_parameters unaffected
• Responses API: non-streaming extraction and streamed final-response extraction return service_tier

Verification

• uv run --frozen pytest tests/unit/test_openai.py tests/unit/test_openai_prompt_extraction.py -q — 48 passed
• uv run --frozen pytest -n auto --dist worksteal tests/unit -q — 622 passed, 2 skipped, 18 errors (the 18 errors are pre-existing tests/unit/test_prompt.py setup errors on main in my local env — missing local Langfuse credentials — verified identical with the change stashed)
• uv run --frozen ruff check . — All checks passed!
• uv run --frozen ruff format --check . — only pre-existing tests/unit/test_experiment.py would be reformatted (also fails on main; not touched by this PR)
• uv run --frozen mypy langfuse --no-error-summary — exit 0

Skipped

• tests/e2e and tests/live_provider — out of scope per task; behavior is fully covered by exporter-local unit tests per the repo's testing policy.

Notes

• Responses API streaming already surfaced service_tier inside the generation metadata (it is collected from the final response object); that behavior is unchanged — the value is now additionally merged into model_parameters.
• Private helper tuple shapes changed (_get_langfuse_data_from_default_response, _extract_streamed_openai_response, _extract_streamed_response_api_response gained a service_tier element); the public stream wrapper classes only gained an optional keyword argument, so the change is backwards compatible.

🤖 Generated with Claude Code

Greptile Summary

This PR adds service_tier capture to the OpenAI monkeypatch integration. On the request side, service_tier is extracted from kwargs (with a NotGiven guard) and stored in model_parameters; on the response side, the resolved tier is read from the non-streaming response, streaming chat chunks, and the final Response API event, then merged back via a new _merge_service_tier_into_model_parameters helper so the authoritative response value wins without clobbering other request-side parameters.

• All three code paths (non-streaming, chat-completion streaming, Responses API streaming) and their async counterparts correctly propagate model_parameters through to _finalize_stream_response, and the helper correctly returns None (suppressing the update) when the response carries no service_tier.
• Tuple shapes for _get_langfuse_data_from_default_response, _extract_streamed_openai_response, and _extract_streamed_response_api_response are updated consistently across every call site.
• A from openai._types import NOT_GIVEN import is placed inside a test function rather than at the top of the module; this should be moved to comply with the repo's import convention.

Confidence Score: 4/5

Safe to merge; the only issue is a function-scoped import in a test that should be at module level.

The core implementation is solid: all three response paths (non-streaming, streaming chat, streaming Responses API) and their async counterparts correctly thread model_parameters through and merge the response-authoritative service_tier without clobbering unrelated parameters. The merge helper's early-return-None design cleanly avoids spurious updates when the response carries no service_tier. The one finding is an import inside a test function body instead of at the module top.

No files require special attention beyond the in-function import in tests/unit/test_openai.py.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant wrap as _wrap / _wrap_async
    participant kwargs as _get_langfuse_data_from_kwargs
    participant resp as _get_langfuse_data_from_default_response
    participant merge as _merge_service_tier_into_model_parameters
    participant gen as LangfuseGeneration

    Caller->>wrap: "openai_client.chat.completions.create(service_tier=auto, ...)"
    wrap->>kwargs: extract request-side model_parameters
    Note over kwargs: parsed_service_tier captured if not NotGiven
    kwargs-->>wrap: langfuse_data (model_parameters incl. service_tier)
    wrap->>gen: "generation.create(model_parameters=...)"

    alt Non-streaming response
        wrap->>resp: _get_langfuse_data_from_default_response(resource, response)
        resp-->>wrap: (model, completion, usage, service_tier)
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: "generation.update(model_parameters=merged)"
    else Streaming response (chat)
        wrap->>wrap: "_instrument_openai_stream(model_parameters=...)"
        Note over wrap: _extract_streamed_openai_response reads service_tier from chunks
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: generation.update via _finalize_stream_response
    else Responses API streaming
        wrap->>wrap: _finalize_stream_response → _extract_streamed_response_api_response
        Note over wrap: service_tier read from response.completed chunk
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: generation.update
    end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant wrap as _wrap / _wrap_async
    participant kwargs as _get_langfuse_data_from_kwargs
    participant resp as _get_langfuse_data_from_default_response
    participant merge as _merge_service_tier_into_model_parameters
    participant gen as LangfuseGeneration

    Caller->>wrap: "openai_client.chat.completions.create(service_tier=auto, ...)"
    wrap->>kwargs: extract request-side model_parameters
    Note over kwargs: parsed_service_tier captured if not NotGiven
    kwargs-->>wrap: langfuse_data (model_parameters incl. service_tier)
    wrap->>gen: "generation.create(model_parameters=...)"

    alt Non-streaming response
        wrap->>resp: _get_langfuse_data_from_default_response(resource, response)
        resp-->>wrap: (model, completion, usage, service_tier)
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: "generation.update(model_parameters=merged)"
    else Streaming response (chat)
        wrap->>wrap: "_instrument_openai_stream(model_parameters=...)"
        Note over wrap: _extract_streamed_openai_response reads service_tier from chunks
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: generation.update via _finalize_stream_response
    else Responses API streaming
        wrap->>wrap: _finalize_stream_response → _extract_streamed_response_api_response
        Note over wrap: service_tier read from response.completed chunk
        wrap->>merge: _merge_service_tier_into_model_parameters(model_parameters, service_tier)
        merge-->>wrap: merged dict
        wrap->>gen: generation.update
    end

Loading

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
tests/unit/test_openai.py:1001-1006
The `NOT_GIVEN` import is placed inside the function body instead of at the top of the module. The repository's convention (and the configured custom instruction) requires module-level imports.

```suggestion
def test_chat_completion_service_tier_not_given_is_absent(
    langfuse_memory_client, get_span, json_attr
):
    openai_client = lf_openai.OpenAI(api_key="test")
```

_{Reviews (1): Last reviewed commit: "feat(openai): capture service_tier in mo..." | Re-trigger Greptile}

Context used:

• Rule used - Move imports to the top of the module instead of p... (source)

Learned From
langfuse/langfuse-python#1387

[github-actions]

@claude review