Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/design/agent-workflows/documentation/adapters/pi.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,23 @@ answers.
For output, Pi streams pure text deltas over ACP (`agent_message_chunk`). The runner
appends them in order to build the final answer.

### OpenAI-compatible custom endpoints

A named `custom` connection (a base URL, a key, and a model id, resolved to provider `openai`
and deployment `custom`) reaches Pi through a `models.json` the runner writes into the per-run
Pi agent dir. The builder (`services/runner/src/engines/sandbox_agent/pi-model-config.ts`) is a
pure function that returns a plan keyed by the connection slug, in the shape
`{"providers": {"<slug>": {baseUrl, api: "openai-completions", apiKey: "$OPENAI_API_KEY",
models: [{id}]}}}`. The `apiKey` is an environment reference, so no secret lands on disk; the
real key rides the sandbox env. Locally the runner writes the file `0600` and atomically into an
isolated per-run dir that carries no operator `auth.json`, so a plan run never authenticates with
the operator's login. On Daytona it uploads the file to `DAYTONA_PI_DIR` before session creation.

For a custom run the runner requests the fully qualified `<slug>/<model>` id rather than the bare
model. This bypasses the suffix match above and its risk of silently routing to `api.openai.com`.
Builder, write, and upload failures are terminal; the run never falls through to a default
provider.

## Daytona notes

Two things differ on Daytona. The sandbox-agent `-full` image ships the `pi-acp` adapter but not the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,15 @@ The service resolves several types, each with a clear job:
one connection deterministically, and fails loud on ambiguity. The run never receives the
whole vault.

**Capability gating.** A per-harness table decides which providers, deployments, and
connection modes a harness can reach. Pi reaches eight direct providers; Claude reaches
Anthropic across direct, custom, bedrock, and vertex. The service checks provider and mode
before resolution and deployment after, and rejects unreachable choices server-side. The
same table ships on `/inspect` so the form can filter ahead of time.
**Capability gating.** A resolved-pair table (`harness_allows_pair`) decides which resolved
`(provider, deployment)` pairs a harness can reach. Pi reaches its direct providers plus the
OpenAI-compatible `custom` deployment (a provider-less `custom` connection resolves to provider
`openai`, deployment `custom`); Claude reaches Anthropic across direct, custom, bedrock, and
vertex. The service checks provider and mode before resolution and deployment after, and rejects
unreachable choices server-side. The flat `deployments` lists (`["direct", "custom"]` for Pi) feed
the form and `/inspect`; the pair table is authoritative for a run. A `custom` connection with an
unusable base URL never falls back to a default endpoint. It raises `EndpointResolutionError`,
which returns HTTP 422, as do `UnsupportedProviderError` and `UnsupportedDeploymentError`.

## Owned by

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,13 @@ The run receives only that candidate's `env` and `endpoint`. It never receives t
vault.

**Capability gating** runs in two halves around resolution. Provider and connection mode are
checked before; deployment is checked after (the resolved connection is what carries it).
Each harness publishes what it can reach: Pi reaches eight direct providers; Claude reaches
Anthropic across direct, custom, bedrock, and vertex. Unnamed default connections degrade
tolerantly to an empty `env` rather than failing the run.
checked before; deployment is checked after (the resolved connection is what carries it). A
resolved-pair table (`harness_allows_pair`) is authoritative: Pi reaches its direct providers
plus the OpenAI-compatible `custom` deployment (a provider-less `custom` connection resolves to
provider `openai`, deployment `custom`); Claude reaches Anthropic across direct, custom, bedrock,
and vertex. Unnamed default connections degrade tolerantly to an empty `env` rather than failing
the run. A `custom` connection with an unusable base URL is the exception: it never resolves to a
default endpoint. It raises `EndpointResolutionError`, which returns HTTP 422.

## Owned by

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Pi OpenAI-compatible models

This project makes a vault `custom_provider` connection usable by the Pi harnesses when the
endpoint implements OpenAI Chat Completions. It keeps the current vault and `/run` schemas, gives
the existing `custom` provider kind the product label **OpenAI-compatible**, and translates the
resolved connection into Pi's native `models.json` format inside the runner.

The implementation is intentionally narrow. It establishes one clean service-to-runner path for a
resolved model connection without introducing a general provider protocol framework. A later
Anthropic Messages implementation can add a second translation beside the OpenAI-compatible one.

## Documents

- [context.md](context.md) explains the problem, goals, non-goals, and boundaries.
- [research.md](research.md) records the current UI, service, wire, runner, and Pi behavior.
- [design.md](design.md) defines the proposed behavior and the decisions behind it.
- [plan.md](plan.md) breaks implementation into reviewable phases.
- [qa.md](qa.md) defines unit, contract, integration, and live acceptance coverage.
- [status.md](status.md) is the source of truth for progress and open decisions.

## Decision summary

1. Keep `custom_provider.data.kind = "custom"`; change only its user-facing label to
**OpenAI-compatible**.
2. Keep the current `ModelRef`, `ResolvedConnection`, `Endpoint`, `secrets`, and `/run` wire
fields. No persistent or wire schema migration is required.
3. Interpret a provider-less named `custom` connection as provider family `openai`. Preserve an
explicit provider family for existing connections, including Anthropic gateway configurations.
4. Validate named connections after vault resolution against the combination of provider family
and deployment. Do not add a wildcard provider capability.
5. For Pi plus `provider=openai` plus `deployment=custom`, generate an isolated `models.json` with
`api: "openai-completions"` and an environment reference to `OPENAI_API_KEY`.
6. Keep Claude's existing connection translation unchanged. Add regression tests around it.
7. Fail the run if a selected custom endpoint cannot be validated or materialized. Never fall back
to the default OpenAI endpoint or a different model.

## Relationship to earlier work

The older [model-config project](../model-config/) established that Pi needs `models.json`. Its
paths and several premises predate the current `services/runner` architecture, strict model
selection, least-privilege connection resolution, and warm Daytona sessions. This workspace is the
current implementation plan for the remaining OpenAI-compatible slice.

Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Context

## Current user experience

A user can create a custom provider in the model registry with a name, base URL, API key, and one
or more model IDs. The vault stores this as a `custom_provider`. The old LLM playground can use the
record, but an agent using `pi_core` or `pi_agenta` cannot.

The agent UI already carries most of the intended connection identity. A vault model option
contains the connection slug, and selecting it writes a structured model reference. The agent
service resolves the named vault record into a model, endpoint, deployment, credential mode, and
least-privilege environment. Those values already cross the `/run` boundary.

The feature stops at the runner. Pi does not use a generic `OPENAI_BASE_URL` variable for ordinary
OpenAI-compatible models. Pi discovers custom providers and arbitrary model IDs through
`PI_CODING_AGENT_DIR/models.json`. The runner never creates that file.

## Goals

- Make an OpenAI-compatible vault connection work automatically with `pi_core` and `pi_agenta`.
- Use the existing vault secret, model reference, resolved connection, and `/run` wire fields.
- Default the existing `custom` provider kind to OpenAI Chat Completions.
- Preserve least-privilege credential delivery and current secret isolation.
- Work identically on the local and Daytona runner paths.
- Keep strict model selection: the requested custom model either runs or fails clearly.
- Keep the implementation easy to extend to Anthropic Messages without designing that extension
now.
- Avoid changing Claude or other harness behavior as part of this feature.

## Non-goals

- No vault schema migration or new secret kind.
- No new `/run` field for provider protocol, API type, or Pi configuration.
- No generic protocol registry.
- No Anthropic Messages support in Pi in this phase.
- No OpenAI Responses support in this phase.
- No Bedrock, Vertex, Azure, or Sagemaker support in Pi.
- No automatic discovery of models from an endpoint.
- No support for unauthenticated local endpoints in the first slice. Users may use a non-secret
placeholder API key for servers that ignore authentication.
- No attempt to make a Daytona sandbox's `localhost` refer to the runner host.

## Constraints

### Security

- Raw API keys remain in `ResolvedConnection.env` and the wire `secrets` field.
- `models.json` contains only an environment reference such as `$OPENAI_API_KEY`, never the key.
- Managed Pi runs must not write model configuration into an operator's shared subscription
directory.
- A blocked or malformed endpoint must fail the named connection. Silently removing the endpoint
would risk sending the request to a provider default.
- Private, loopback, and insecure HTTP endpoints remain disabled by default. Trusted single-tenant
deployments can opt in with the existing egress setting.

### Compatibility

- Existing standard provider keys continue to use Pi's built-in model catalog and env mapping.
- Existing Claude custom-gateway handling continues to use `ANTHROPIC_BASE_URL`.
- Existing explicit provider families on custom records remain authoritative. Only a missing
provider family on `kind=custom` defaults to `openai`.
- Unknown harnesses and unsupported provider/deployment combinations remain closed.

### Ownership

- The UI owns product labels and selection intent.
- The service owns vault resolution, semantic normalization, endpoint validation, and harness
capability validation.
- The runner owns conversion from the neutral resolved connection into harness-native runtime
configuration.
- Pi owns the format and behavior of `models.json`.

Loading
Loading