chore: update Warden Configuration#5290
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5290 +/- ##
==========================================
+ Coverage 74.13% 74.14% +0.01%
==========================================
Files 508 508
Lines 18320 18320
Branches 3584 3584
==========================================
+ Hits 13581 13584 +3
+ Misses 3864 3861 -3
Partials 875 875 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| actions = ["opened", "synchronize", "reopened", "labeled"] | ||
| draft = false | ||
| labels = ["Warden"] |
There was a problem hiding this comment.
Bug: The configuration now requires the "Warden" label for reviews to run, but there's no mechanism to automatically apply this label, potentially disabling reviews on new PRs.
Severity: HIGH
Suggested Fix
Add a GitHub Actions workflow to automatically apply the "Warden" label to relevant pull requests. Alternatively, revert the labels = ["Warden"] change in warden.toml if gating reviews by label was not intended. If it is intentional, update developer documentation to instruct contributors to add the label manually.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: warden.toml#L46-L48
Potential issue: The `warden.toml` configuration was updated to require the `labels =
["Warden"]` on pull requests to trigger Warden skills. However, no corresponding
automation was added to apply this label. As a result, Warden reviews for code quality,
security, and bug finding will silently fail to run on new pull requests unless a
developer manually adds the "Warden" label. This creates a significant gap in automated
code review coverage, as developers may be unaware of this new manual requirement.
Also affects:
warden.toml:56~58warden.toml:66~68warden.toml:76~78
Did we get this right? 👍 / 👎 to inform future reviews.
There was a problem hiding this comment.
Not quite correct:
.. from https://warden.sentry.dev/config/triggers ..
type enum
pull_request,local, orschedule.actions string[]
Event actions forpull_requesttriggers.draft boolean
Draft state forpull_requesttriggers. Setfalseto run only on non-draft PRs.labels string[]
Pull request labels that can also match the trigger. Use withdraft = falseto let a label opt draft PRs into a run.
This means that for draft Pull Requests, for which Warden is not run automatically, we still can run Warden when applying the "Warden" label to the draft Pull Request.
For non-draft Pull Requests, however, Warden is run regardless (for the actions: "opened", "synchronize", "reopened" and "labeled").
Hence, we add the "labeled" action here too, so that we can run Warden on a draft Pull Request by applying the "Warden" label.
For more information, see the docs at https://warden.sentry.dev/config/triggers
| @@ -2,22 +2,28 @@ | |||
| # https://github.com/getsentry/warden | |||
| # | |||
| # Warden reviews code using AI-powered skills triggered by GitHub events. | |||
There was a problem hiding this comment.
Bug: The model name anthropic/claude-opus-4-5 in warden.toml appears to be invalid or deprecated, which will likely cause API calls to fail.
Severity: HIGH
Suggested Fix
In warden.toml, update the model from "anthropic/claude-opus-4-5" to a valid, currently supported model name, such as "anthropic/claude-opus-4-8". Verify the correct model name from the official Anthropic documentation before applying the change.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: warden.toml#L4
Potential issue: The `warden.toml` configuration sets the default model to
`anthropic/claude-opus-4-5`. According to Anthropic's official API documentation, this
model name is deprecated or invalid. When the Warden tool attempts to use this model for
an API call, the request will likely be rejected by the Anthropic API. This will result
in an API error, causing all Warden skills that rely on this model to fail during their
execution in the CI pipeline, effectively breaking the automated code review process.
There was a problem hiding this comment.
See https://github.com/getsentry/sentry-dotnet/pull/5288/checks?check_run_id=80987232263
The CI error indicates that ...
... claude-opus-4-5 seems to be the (current) default of the pi runtime:
But there seems to be a bug with this default: it only declares the model, but missing the provider:
-claude-opus-4-5
+anthropic/claude-opus-4-5For more information, see the Warden docs:
https://warden.sentry.dev/config/models
There was a problem hiding this comment.
Verify via run: https://github.com/getsentry/sentry-dotnet/pull/5290/checks?check_run_id=81447573521
The workflow run now succeeds with this change:
fixes #5292
Summary
Update Warden Configuration to the latest version (currently
warden 0.38.1).Remarks
What I did locally to create this PR is:
warden.toml@sentry/wardentoolwarden initwarden.tomlfrom the latest templatemodel = "anthropic/claude-opus-4-5"This fixes the current issue in CI, see