Skip to content

Make application validation manual-only - #523

Open
msaroufim wants to merge 1 commit into
mainfrom
agent/manual-only-application-validation
Open

Make application validation manual-only#523
msaroufim wants to merge 1 commit into
mainfrom
agent/manual-only-application-validation

Conversation

@msaroufim

Copy link
Copy Markdown
Member

What changed

  • removed the 22:00 America/Los_Angeles application-validation scheduler and its all-leaderboard scan
  • removed the APPLICATION_VALIDATION_ENABLED setting and scheduler startup hooks
  • kept validation behind the authenticated admin API/CLI batch operation
  • preserved top-10-by-user defaults plus --all-users, --only-missing, --wait, and the two-job concurrency bound
  • updated tests and documentation to describe manual-only behavior

Why

Automatic validation would silently expand to every future problem that adds a validation contract, before leaderboard-mode triggering and rate limiting have been designed. Keeping this explicit makes GPU spend and failure handling operator-controlled in the meantime.

Existing validation records and Kernelboard badges are unchanged.

Validation

  • uv run ruff check . --exclude examples/ --line-length 120
  • uv run pytest -q tests/test_application_validation.py tests/test_admin_api.py tests/test_admin_cli.py tests/test_leaderboard_db.py (120 passed)
  • uv run pytest -q -m 'not integration' tests (252 passed, 14 deselected)
  • repository-wide search confirms no scheduler symbol, 10pm schedule, or validation-enable environment toggle remains

@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/libkernelbot
  application_validation.py 26-32
  leaderboard_db.py
  utils.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant