Skip to content

fix(code-review): add model diagnostics#3990

Merged
alex-alecu merged 5 commits into
mainfrom
fix/code-review-model-list
Jun 15, 2026
Merged

fix(code-review): add model diagnostics#3990
alex-alecu merged 5 commits into
mainfrom
fix/code-review-model-list

Conversation

@alex-alecu

@alex-alecu alex-alecu commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds private runtime diagnostics when code reviews fail because a model is unavailable, so support can see the requested model, live model count, and close matches without showing those details publicly.
  • Keeps GitHub, GitLab, and public Cloud Agent messages generic while the signed-in dashboard gets a clearer private error.
  • Does the live model lookup in the wrapper and gives it a short timeout so a slow lookup cannot block failure reporting.
  • Expands isModelNotFoundMessage regex to also match unknown model and invalid model patterns, keeping it in sync with classifyAssistantFailureMessage in safe-failure-projection.ts so those error variants no longer skip diagnostics.

Verification

Manual staging checks:

  1. Start a code review with a model that is not available.
  2. Confirm the signed-in dashboard shows the missing model, model count, and up to five alternatives.
  3. Confirm GitHub or GitLab only shows the generic model-not-found message.
  4. Search logs and Sentry by the review id and confirm they include the full runtime model list.

Visual Changes

N/A

Reviewer Notes

  • The full model list is private diagnostic data and should not appear in public review output.
  • The wrapper drops diagnostics if the live model lookup is slow, so terminal failure reporting keeps moving.
  • isModelNotFoundMessage now mirrors the regex pattern in classifyAssistantFailureMessage: /\b(model (?:was )?not found|unknown model|invalid model)\b/i.

Built for alex-alecu by Kilo

Comment thread services/cloud-agent-next/wrapper/src/connection.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental commit (957f429) correctly moves modelNotFoundRuntimeDiagnostics into the WrapperTerminalFailure type, adds a guarded async fetch with a 1 s timeout, and wires the diagnostics through the ingest stream event. All previously flagged issues are resolved; no new issues detected.

Resolved Issues
File Line Issue Status
services/cloud-agent-next/wrapper/src/connection.ts 811 isModelNotFoundMessage regex was narrower than classifyAssistantFailureMessageunknown model/invalid model variants now covered ✅ Fixed
services/cloud-agent-next/wrapper/src/connection.ts 773 getTerminalFailure was not inspecting the full error object for billing markers ✅ Fixed
Other Observations (not in diff — carried forward, still open)

These were noted in earlier reviews and are not blocking, but remain unaddressed:

File Note
apps/web/src/app/api/internal/code-review-status/[reviewId]/route.ts (lines 119–198) Contains a local copy of ModelNotFoundRuntimeDiagnostics type and parseModelNotFoundRuntimeDiagnostics with hardcoded constants (MAX_MODEL_ID_LENGTH = 512, MAX_SUGGESTIONS = 5) that duplicate services/cloud-agent-next/src/shared/runtime-model-diagnostics.ts. If the shared module's limits change, the web app's validation silently diverges.
services/cloud-agent-next/src/websocket/ingest.ts:70 and services/cloud-agent-next/src/session/wrapper-supervisor.ts:56 MODEL_NOT_FOUND_SAFE_ERROR_MESSAGE = 'Assistant request failed: model not found' is copy-pasted rather than imported from a single source.
Files Reviewed (24 files)
  • services/cloud-agent-next/wrapper/src/connection.ts — previously 2 issues, now resolved
  • services/cloud-agent-next/wrapper/src/main.ts — no issues
  • services/cloud-agent-next/wrapper/src/model-diagnostics.ts — no issues
  • services/cloud-agent-next/wrapper/src/kilo-api.ts — no issues
  • services/cloud-agent-next/src/shared/protocol.ts — no issues
  • services/cloud-agent-next/src/shared/runtime-model-diagnostics.ts — no issues
  • services/cloud-agent-next/src/shared/index.ts — no issues
  • services/cloud-agent-next/src/session/wrapper-supervisor.ts — no issues
  • services/cloud-agent-next/src/session/message-settlement-outbox.ts — no issues
  • services/cloud-agent-next/src/session/session-message-state.ts — no issues
  • services/cloud-agent-next/src/websocket/ingest.ts — no issues
  • services/cloud-agent-next/src/callbacks/types.ts — no issues
  • services/cloud-agent-next/test/unit/wrapper/model-diagnostics.test.ts — no issues
  • services/cloud-agent-next/test/unit/wrapper/reconnection.test.ts — no issues
  • services/cloud-agent-next/test/unit/wrapper/server.test.ts — no issues
  • services/cloud-agent-next/test/unit/wrapper/snapshot.test.ts — no issues
  • services/cloud-agent-next/test/unit/wrapper/kilo-api.test.ts — no issues
  • services/cloud-agent-next/wrapper/package.json — no issues
  • apps/web/src/app/api/internal/code-review-status/[reviewId]/route.ts — duplicate parser noted above
  • apps/web/src/app/api/internal/code-review-status/[reviewId]/route.test.ts — no issues
  • services/cloud-agent-next/src/session/message-settlement-outbox.test.ts — no issues
  • services/cloud-agent-next/src/session/wrapper-supervisor.test.ts — no issues
  • services/cloud-agent-next/src/websocket/ingest.test.ts — no issues
  • pnpm-lock.yaml — skipped (generated)

Reviewed by claude-4.6-sonnet-20260217 · 3,649,342 tokens

Review guidance: REVIEW.md from base branch main

"dependencies": {
"@kilocode/sdk": "7.3.21"
"@kilocode/sdk": "7.3.21",
"fuzzysort": "3.1.0"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eshurakov how do you feel about adding a new dependency? When a model is not found, I do a fuzzy search on the available runtime (CLI) models against the one set in web and return just a few.

kilo-code-bot Bot and others added 3 commits June 12, 2026 12:04
…-list

# Conflicts:
#	services/cloud-agent-next/src/callbacks/types.ts
#	services/cloud-agent-next/src/session/message-settlement-outbox.ts
#	services/cloud-agent-next/src/session/wrapper-supervisor.ts
#	services/cloud-agent-next/src/websocket/ingest.ts
#	services/cloud-agent-next/test/unit/wrapper/reconnection.test.ts
#	services/cloud-agent-next/wrapper/src/connection.ts
Comment thread services/cloud-agent-next/wrapper/src/connection.ts
@alex-alecu alex-alecu merged commit 1c75249 into main Jun 15, 2026
61 checks passed
@alex-alecu alex-alecu deleted the fix/code-review-model-list branch June 15, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants