Skip to content

chore: update Warden Configuration#5290

Open
Flash0ver wants to merge 2 commits into
mainfrom
chore/update-warden-configuration
Open

chore: update Warden Configuration#5290
Flash0ver wants to merge 2 commits into
mainfrom
chore/update-warden-configuration

Conversation

@Flash0ver

@Flash0ver Flash0ver commented Jun 15, 2026

Copy link
Copy Markdown
Member

fixes #5292

Summary

Update Warden Configuration to the latest version (currently warden 0.38.1).

Remarks

What I did locally to create this PR is:

  1. delete file warden.toml
  2. update global @sentry/warden tool
    • @sentry/warden@0.38.1

  3. re-initialize Warden via warden init
    • created a new warden.toml from the latest template
  4. restore previous actions and update previous configuration to the new template
  5. add model = "anthropic/claude-opus-4-5"

This fixes the current issue in CI, see

@Flash0ver Flash0ver self-assigned this Jun 15, 2026
@Flash0ver Flash0ver added the skip-changelog Suppress automatic changelog generation via Craft label Jun 15, 2026
@codecov

codecov Bot commented Jun 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.14%. Comparing base (2f2842f) to head (8f0188b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5290      +/-   ##
==========================================
+ Coverage   74.13%   74.14%   +0.01%     
==========================================
  Files         508      508              
  Lines       18320    18320              
  Branches     3584     3584              
==========================================
+ Hits        13581    13584       +3     
+ Misses       3864     3861       -3     
  Partials      875      875              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Flash0ver Flash0ver requested a review from bitsandfoxes June 15, 2026 14:13
@Flash0ver Flash0ver marked this pull request as ready for review June 15, 2026 14:13
Comment thread warden.toml
Comment on lines +46 to +48
actions = ["opened", "synchronize", "reopened", "labeled"]
draft = false
labels = ["Warden"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The configuration now requires the "Warden" label for reviews to run, but there's no mechanism to automatically apply this label, potentially disabling reviews on new PRs.
Severity: HIGH

Suggested Fix

Add a GitHub Actions workflow to automatically apply the "Warden" label to relevant pull requests. Alternatively, revert the labels = ["Warden"] change in warden.toml if gating reviews by label was not intended. If it is intentional, update developer documentation to instruct contributors to add the label manually.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: warden.toml#L46-L48

Potential issue: The `warden.toml` configuration was updated to require the `labels =
["Warden"]` on pull requests to trigger Warden skills. However, no corresponding
automation was added to apply this label. As a result, Warden reviews for code quality,
security, and bug finding will silently fail to run on new pull requests unless a
developer manually adds the "Warden" label. This creates a significant gap in automated
code review coverage, as developers may be unaware of this new manual requirement.

Also affects:

  • warden.toml:56~58
  • warden.toml:66~68
  • warden.toml:76~78

Did we get this right? 👍 / 👎 to inform future reviews.

@Flash0ver Flash0ver Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite correct:

Image

.. from https://warden.sentry.dev/config/triggers ..

type enum
pull_request, local, or schedule.

actions string[]
Event actions for pull_request triggers.

draft boolean
Draft state for pull_request triggers. Set false to run only on non-draft PRs.

labels string[]
Pull request labels that can also match the trigger. Use with draft = false to let a label opt draft PRs into a run.

This means that for draft Pull Requests, for which Warden is not run automatically, we still can run Warden when applying the "Warden" label to the draft Pull Request.
For non-draft Pull Requests, however, Warden is run regardless (for the actions: "opened", "synchronize", "reopened" and "labeled").
Hence, we add the "labeled" action here too, so that we can run Warden on a draft Pull Request by applying the "Warden" label.

For more information, see the docs at https://warden.sentry.dev/config/triggers

Comment thread warden.toml
@@ -2,22 +2,28 @@
# https://github.com/getsentry/warden
#
# Warden reviews code using AI-powered skills triggered by GitHub events.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The model name anthropic/claude-opus-4-5 in warden.toml appears to be invalid or deprecated, which will likely cause API calls to fail.
Severity: HIGH

Suggested Fix

In warden.toml, update the model from "anthropic/claude-opus-4-5" to a valid, currently supported model name, such as "anthropic/claude-opus-4-8". Verify the correct model name from the official Anthropic documentation before applying the change.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: warden.toml#L4

Potential issue: The `warden.toml` configuration sets the default model to
`anthropic/claude-opus-4-5`. According to Anthropic's official API documentation, this
model name is deprecated or invalid. When the Warden tool attempts to use this model for
an API call, the request will likely be rejected by the Anthropic API. This will result
in an API error, causing all Warden skills that rely on this model to fail during their
execution in the CI pipeline, effectively breaking the automated code review process.

@Flash0ver Flash0ver Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://github.com/getsentry/sentry-dotnet/pull/5288/checks?check_run_id=80987232263

The CI error indicates that ...

Image

... claude-opus-4-5 seems to be the (current) default of the pi runtime:

Image

But there seems to be a bug with this default: it only declares the model, but missing the provider:

-claude-opus-4-5
+anthropic/claude-opus-4-5

For more information, see the Warden docs:
https://warden.sentry.dev/config/models

@Flash0ver Flash0ver Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verify via run: https://github.com/getsentry/sentry-dotnet/pull/5290/checks?check_run_id=81447573521

The workflow run now succeeds with this change:

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Suppress automatic changelog generation via Craft

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: fix Warden workflows

1 participant