Skip to content

chore: update Warden Configuration #5290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jamescrosswell merged 2 commits into from
Jun 21, 2026
Merged

chore: update Warden Configuration #5290

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f6fb115
chore: update Warden Configuration
Flash0ver Jun 15, 2026
8f0188b
fix(ci): use provider/model format for Pi runtime
Flash0ver Jun 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions warden.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,28 @@
# https://github.com/getsentry/warden
#
# Warden reviews code using AI-powered skills triggered by GitHub events.

@sentry sentry Bot Jun 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The model name anthropic/claude-opus-4-5 in warden.toml appears to be invalid or deprecated, which will likely cause API calls to fail.
Severity: HIGH

Suggested Fix

In warden.toml, update the model from "anthropic/claude-opus-4-5" to a valid, currently supported model name, such as "anthropic/claude-opus-4-8". Verify the correct model name from the official Anthropic documentation before applying the change.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: warden.toml#L4

Potential issue: The `warden.toml` configuration sets the default model to
`anthropic/claude-opus-4-5`. According to Anthropic's official API documentation, this
model name is deprecated or invalid. When the Warden tool attempts to use this model for
an API call, the request will likely be rejected by the Anthropic API. This will result
in an API error, causing all Warden skills that rely on this model to fail during their
execution in the CI pipeline, effectively breaking the automated code review process.

@Flash0ver Flash0ver Jun 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/getsentry/sentry-dotnet/pull/5288/checks?check_run_id=80987232263

The CI error indicates that ...

Image

... claude-opus-4-5 seems to be the (current) default of the pi runtime:

Image

But there seems to be a bug with this default: it only declares the model, but missing the provider:

-claude-opus-4-5
+anthropic/claude-opus-4-5

For more information, see the Warden docs:
https://warden.sentry.dev/config/models

@Flash0ver Flash0ver Jun 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify via run: https://github.com/getsentry/sentry-dotnet/pull/5290/checks?check_run_id=81447573521

The workflow run now succeeds with this change:

Image

# Skills live in .agents/skills/ or .claude/skills/
# Built-in skills are available by name. Add more skills as needed.
#
# Add skills with: warden add <skill-name>
# Add built-in reviews with:
# warden add security-review
# warden add code-review

version = 1

# Default settings inherited by all skills
[defaults]
runtime = "pi"
model = "anthropic/claude-opus-4-5"
# Severity levels: critical, high, medium, low, info
# failOn: minimum severity that fails the check
failOn = "high"
# reportOn: minimum severity that creates PR annotations
reportOn = "medium"

# Skills define what to analyze and when to run
# Add skills with: warden add <skill-name>
# Add built-in reviews with:
# warden add security-review
# warden add code-review
#
# Example skill with path filters and triggers:
#
Expand All @@ -28,36 +34,46 @@ reportOn = "medium"
#
# [[skills.triggers]]
# type = "pull_request"
# actions = ["opened", "synchronize", "reopened"]
# actions = ["opened", "synchronize", "reopened", "labeled"]
# draft = false
# labels = ["Warden"]

[[skills]]
name = "code-review"
remote = "getsentry/skills"

[[skills.triggers]]
type = "pull_request"
actions = ["opened", "synchronize", "reopened"]
actions = ["opened", "synchronize", "reopened", "labeled"]
draft = false
labels = ["Warden"]
Comment on lines +47 to +49

@sentry sentry Bot Jun 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The configuration now requires the "Warden" label for reviews to run, but there's no mechanism to automatically apply this label, potentially disabling reviews on new PRs.
Severity: HIGH

Suggested Fix

Add a GitHub Actions workflow to automatically apply the "Warden" label to relevant pull requests. Alternatively, revert the labels = ["Warden"] change in warden.toml if gating reviews by label was not intended. If it is intentional, update developer documentation to instruct contributors to add the label manually.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: warden.toml#L46-L48

Potential issue: The `warden.toml` configuration was updated to require the `labels =
["Warden"]` on pull requests to trigger Warden skills. However, no corresponding
automation was added to apply this label. As a result, Warden reviews for code quality,
security, and bug finding will silently fail to run on new pull requests unless a
developer manually adds the "Warden" label. This creates a significant gap in automated
code review coverage, as developers may be unaware of this new manual requirement.

Also affects:

  • warden.toml:56~58
  • warden.toml:66~68
  • warden.toml:76~78

Did we get this right? 👍 / 👎 to inform future reviews.

@Flash0ver Flash0ver Jun 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite correct:

Image

.. from https://warden.sentry.dev/config/triggers ..

type enum
pull_request, local, or schedule.

actions string[]
Event actions for pull_request triggers.

draft boolean
Draft state for pull_request triggers. Set false to run only on non-draft PRs.

labels string[]
Pull request labels that can also match the trigger. Use with draft = false to let a label opt draft PRs into a run.

This means that for draft Pull Requests, for which Warden is not run automatically, we still can run Warden when applying the "Warden" label to the draft Pull Request.
For non-draft Pull Requests, however, Warden is run regardless (for the actions: "opened", "synchronize", "reopened" and "labeled").
Hence, we add the "labeled" action here too, so that we can run Warden on a draft Pull Request by applying the "Warden" label.

For more information, see the docs at https://warden.sentry.dev/config/triggers


[[skills]]
name = "find-bugs"
remote = "getsentry/skills"

[[skills.triggers]]
type = "pull_request"
actions = ["opened", "synchronize", "reopened"]
actions = ["opened", "synchronize", "reopened", "labeled"]
draft = false
labels = ["Warden"]

[[skills]]
name = "gha-security-review"
remote = "getsentry/skills"

[[skills.triggers]]
type = "pull_request"
actions = ["opened", "synchronize", "reopened"]
actions = ["opened", "synchronize", "reopened", "labeled"]
draft = false
labels = ["Warden"]

[[skills]]
name = "security-review"
remote = "getsentry/skills"

[[skills.triggers]]
type = "pull_request"
actions = ["opened", "synchronize", "reopened"]
actions = ["opened", "synchronize", "reopened", "labeled"]
draft = false
labels = ["Warden"]
Loading