docs(agent-workflows): plan for custom providers + model auth in Pi#5013
docs(agent-workflows): plan for custom providers + model auth in Pi#5013mmabrouk wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
45f71ea to
c89be77
Compare
New plan-feature workspace for making provider + model auth work end to end on the Pi harness, custom providers included. Builds on provider-model-auth (BUILT) and model-config (DESIGNED). Diagnoses five gaps and sequences five slices. Claude-Session: https://claude.ai/code/session_01HCMtsTWnCdh8fPEzGrda6C
| Normalize `deployment` to the closed set of real access surfaces, and let `endpoint` carry the | ||
| "customness": | ||
|
|
||
| - A `custom_provider` whose kind is in `_PROVIDER_ENV_VARS` (a known single-key direct provider: |
There was a problem hiding this comment.
🤖 The AI agent says:
This still leaves the original pre-resolve failure in place for an arbitrary custom slug such as ollama or my-gateway: it only works when kind is already present in _PROVIDER_ENV_VARS. The requirement is a named OpenAI-compatible connection, not a known built-in provider disguised as custom. Keep custom as the resolved deployment, defer the provider decision until after the trusted vault record is resolved, and allow the Pi (custom, OpenAI-compatible) pair explicitly. A provider-less custom record can default to the OpenAI-compatible family in the service without changing the stored schema.
There was a problem hiding this comment.
Accepted. The revised design.md Section 1 keeps deployment="custom" for a record whose kind is not a known family, defaults a provider-less record to the OpenAI-compatible family after the trusted vault record resolves (no stored-schema change), and adds the (custom, openai-compatible) pair to Pi's capability row so a named endpoint reaches the runner instead of dying at the pre-resolve or post-resolve gate. The known-family normalization to direct already landed (connections.py:330-335) and stays.
|
|
||
| | Field | Role | Owner / lifecycle | Source in our contract | | ||
| | --- | --- | --- | --- | | ||
| | `providers` | config, extensible map (plural, correct) | service, per run | keyed by `resolved_connection.provider` | |
There was a problem hiding this comment.
🤖 The AI agent says:
Key this map by the selected connection slug, not resolved_connection.provider. Two custom connections can both resolve to the OpenAI family, and a provider-less custom connection may not have a provider at all; using the family either collides or cannot produce a Pi provider id. The stable Pi identity should be providers[connection.slug], with the selected model registered under that entry.
There was a problem hiding this comment.
Accepted. models.json is now keyed by providers[], not resolved_connection.provider, and the selected model registers under that entry. This adds one field: slug on ResolvedConnection and its to_wire(), classified in design.md Section 3 as identity/metadata owned by the vault record and changing per connection edit. This drops the first plan's "no new wire field" non-goal, replaced by exactly one new wire field.
| | --- | --- | --- | --- | | ||
| | `providers` | config, extensible map (plural, correct) | service, per run | keyed by `resolved_connection.provider` | | ||
| | `providers.<p>.baseUrl` | routing (the endpoint being contacted, nested under the provider) | service, per run | `resolved_connection.endpoint.baseUrl` | | ||
| | `providers.<p>.api` | config (the wire dialect) | service, per run | derived from provider (default `openai-completions`; Anthropic to `anthropic-messages`) | |
There was a problem hiding this comment.
🤖 The AI agent says:
For this first version, keep the dialect strictly openai-completions. Anthropic Messages has materially different request/response semantics and should not be inferred from a provider label here. A small pure builder can expose a protocol discriminator/union so anthropic-messages can be added later, while v1 rejects unsupported protocols instead of half-supporting them.
There was a problem hiding this comment.
Accepted. v1 speaks strictly openai-completions and does not infer anthropic-messages from a provider label. The model-config builder exposes a protocol discriminator so another dialect can be added later, and v1 rejects any unsupported protocol loudly. See design.md Section 2.
| daemon env. The real key rides `resolved_connection.env` and the runner injects it into the daemon | ||
| env (the same channel `auth.json` uses). So `models.json` on disk carries no secret value. `auth.json` | ||
| follows the same rule: `{ "openrouter": { "type": "api_key", "key": "$OPENROUTER_API_KEY" } }`, | ||
| merged with the login's copied `auth.json` so OAuth still works. |
There was a problem hiding this comment.
🤖 The AI agent says:
Do not merge the operator's copied auth.json into a vault-managed custom-provider run. The existing local helper copies personal login state; reusing it unchanged would let a managed run silently authenticate with an operator subscription or the wrong provider. Create an isolated managed Pi directory that carries only non-credential settings/assets plus models.json with an env reference. The raw key remains solely in the daemon/sandbox environment.
There was a problem hiding this comment.
Accepted. A managed run no longer reuses the operator's copied auth.json. The runner builds an isolated managed Pi directory carrying only non-credential settings plus models.json with a "$ENV" apiKey reference, and the raw key stays solely in the daemon or sandbox environment. This also fixes the existing local path, which copies auth.json today (pi-assets.ts:475-490) while Daytona already refuses (daytona.ts:168-171). See design.md Section 2.
|
|
||
| - The base URL rides `resolved_connection.endpoint.baseUrl`. Already on the wire | ||
| (`wire_resolved_connection()`, `provider-model-auth` Slice 4). | ||
| - The selected model id rides `resolved_connection.model`. Already on the wire. |
There was a problem hiding this comment.
🤖 The AI agent says:
The wire can keep the bare selected model id, but the runner must derive and set the fully qualified Pi id ${connection.slug}/${model}. Pi may advertise multiple providers with the same model suffix, and the current suffix-resolution fallback can select the first wrong provider. Have the model-config builder return both the file content and exact runtime model id, then pass that exact id to setModel.
There was a problem hiding this comment.
Accepted. The wire keeps the bare selected model id, and the model-config builder returns both the file content and the exact runtime id /. The runner passes that exact id to setModel, bypassing the suffix fallback in pickModel (model.ts:49-59) that can select the wrong provider. See design.md Section 2.
| grouped options. A vault custom-provider's models appear only if that provider (and its | ||
| now-`direct` deployment) is reachable by the selected harness, the same policy filter | ||
| `namedConnectionOptions` already applies to the Connection dropdown. | ||
| - `VaultConnectionEntry` (`connectionUtils.ts:302-312`) gains `models?: string[]`, so the array |
There was a problem hiding this comment.
🤖 The AI agent says:
This schema/picker expansion is outside the agreed scope. The existing UI already creates the custom-provider record and sends its selected connection/model; for now the UI change should be limited to renaming the visible type to OpenAI-compatible. Keep the implementation boundary from the service through the runner and remove the VaultConnectionEntry/model-choice contract changes from this plan.
There was a problem hiding this comment.
Accepted. The picker and model-choice contract expansion is cut from this plan and parked in the model-config sibling (Part 3); the VaultConnectionEntry symbol the first plan named does not exist in web/. The only UI change here is renaming the visible type "Custom provider" to "OpenAI-compatible endpoint" at the three locations. See design.md Section 5.
c89be77 to
6eb358c
Compare
Correction: no new wire field is neededThis plan proposed threading a custom-provider connection to Pi by adding a dedicated wire field. That turned out to be unnecessary, and this PR's design text on that point is superseded. What actually ships: the connection Where the implementation landed (both retargeted to
The plan of record is This PR: the corrected copies of the six |
|
Correction: no new wire field was needed; this plan is superseded. This plan claimed the change needs "exactly one new wire field, the connection slug on The feature shipped under the sibling pi-openai-compatible-models project, now implemented and end-to-end verified across the local and Daytona paths:
The corrected doc text (correction banners on README/context/design/plan/status; research unchanged) sits in the working tree awaiting a clean lane apply — the |
1 participant
Goal
Let a user run an OpenAI-compatible model through a Pi agent. The user stores a connection (a base URL, a key, and one or more model ids), picks that connection and a model in the playground, and the run reaches the model. The connection's slug is its identity end to end: the vault record, the resolver, the wire, and the Pi config the runner writes all agree on the same slug, so two connections that speak the same protocol stay distinct.
This is a docs-only plan-feature workspace at
docs/design/agent-workflows/projects/custom-providers-in-pi/. No code changes.What already landed since the plan was written
The plan was written on 2026-07-02. Three of its original slices have since landed in the codebase, so this revision marks them done and re-sequences the rest:
PROVIDER_ENV_VARS(capabilities.py:111-125), withtogether_ai -> TOGETHER_API_KEYandminimax. There is a parity regression test.deployment="direct"(connections.py:330-335) and passes Pi's["direct"]gate.allowedModelsreadsc.value ?? c.id(model.ts:72),applyModelraises a typedModelNotSettableError, andAGENTA_AGENT_MODEL_STRICTis wired for every harness and defaults to true (sandbox_agent.ts:374). This shipped strict-by-default directly, so the plan's staged "flip later" step is moot.What remains, and the revised design
A connection whose kind is not a known family (an Ollama gateway, an in-house proxy, any named OpenAI-compatible endpoint) still fails before it runs, and the runner writes no Pi
models.json. Six review comments on the design reshaped how the plan closes that path. All six were accepted:deployment="custom"for a record whose kind is not a known family, default a provider-less record to the OpenAI-compatible family after the trusted vault record resolves, and let Pi allow the(custom, openai-compatible)pair. The known-familydirectnormalization that landed stays.models.jsonby the connection slug, notresolved_connection.provider, because two custom connections can resolve to the same family and a provider-less one has no provider id.openai-completionsonly in v1. Do not inferanthropic-messagesfrom a provider label. A pure builder exposes a protocol discriminator for later; v1 rejects unsupported protocols loudly.models.jsonwith a"$ENV"apiKey reference. The raw key stays in the daemon or sandbox environment. This also fixes the existing local path, which copies the operatorauth.jsontoday while Daytona already refuses.<slug>/<model>id and sets it directly, bypassing the suffix fallback that can pick the wrong provider.model-configsibling, Part 3). The UI change here is one rename: "Custom provider" to "OpenAI-compatible endpoint".One new wire field
The first plan's non-goal "no new
/runwire field" is dropped. Keyingmodels.jsonby slug needs the slug on the wire. It is replaced by: exactly one new field,slugonResolvedConnection.to_wire(). Everything else the runner still derives fromresolved_connectionandsecrets.The revised slices
slugtoResolvedConnectionand its wire form; a provider-less or unknown-kind custom connection resolves to the OpenAI-compatible family withdeployment="custom"; Pi's capability table allowscustom.models.jsonkeyed byproviders[<slug>](dialectopenai-completions,"$ENV"apiKey, the one selected model) and returns the exact<slug>/<model>id; the runner creates an isolated managed Pi dir on the local and Daytona paths and sets that exact id.Each slice is independently shippable and carries tests (resolver unit tests and a wire golden for slice 1; builder, managed-dir isolation, Daytona parity, and a live acceptance run for slice 2; a render check for slice 3). The Pi startup-banner leak stays a separate lane.
Files
Six files in the plan workspace:
README.md,context.md,research.md,design.md,plan.md,status.md.research.mdgains a dated 2026-07-14 re-verification section.design.mdkeeps the design-interfaces role tables as its spine.https://claude.ai/code/session_01HCMtsTWnCdh8fPEzGrda6C