Skip to content

Show per-submission application validation badges - #261

Merged
msaroufim merged 4 commits into
mainfrom
agent/cholesky-validation
Jul 26, 2026
Merged

Show per-submission application validation badges#261
msaroufim merged 4 commits into
mainfrom
agent/cholesky-validation

Conversation

@msaroufim

@msaroufim msaroufim commented Jul 26, 2026

Copy link
Copy Markdown
Member

Summary

  • join the latest validation result for the exact ranked submission and GPU
  • only show results matching the leaderboard task's current contract version
  • keep validation strictly additive: missing, partial, or failed validation
    never removes or reorders a leaderboard entry
  • render VALIDATED, X/Y VALIDATED, or VALIDATION ERROR beside the
    submitter name
  • show shape coverage, synchronized-wall speedup, contract version, and check
    time in the tooltip
  • fix the leaderboard's error-state precedence and initially-expanded row
    toggle behavior exposed by Leaderboard.test.tsx

Validation

  • Leaderboard.test.tsx — 14/14 passed
  • production frontend build — passed
  • frontend lint and Python Ruff — passed
  • API regression coverage proves ranked submission IDs are unchanged for both
    missing and failed validation
  • current-tip local Kernelbot + Kernelboard + Postgres + Redis stack rendered
    the attached Seraphim result as 5/8 VALIDATED at 2.82x geomean speedup
  • repository-wide Vitest run — 93 passed; 8 unrelated existing assertions
    remain stale in NavBar, News, Login, and NavUserProfile
  • production Kernelbot migration and problem sync completed before merge; the
    manual top-10-users B200 sweep completed 10/10 jobs, with coverage ranging
    from 1/8 to 8/8 and one fully validated result

Depends on merged and deployed gpu-mode/kernelbot#518. The Cholesky producer is
merged gpu-mode/reference-kernels#168.

@msaroufim
msaroufim marked this pull request as ready for review July 26, 2026 16:49
@msaroufim
msaroufim merged commit bd048b5 into main Jul 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant