Skip to content

Remove updateChallengeCompletionMetrics scheduled job#1246

Merged
jake-low merged 2 commits into
mainfrom
jlow/refactor-metrics
Jun 4, 2026
Merged

Remove updateChallengeCompletionMetrics scheduled job#1246
jake-low merged 2 commits into
mainfrom
jlow/refactor-metrics

Conversation

@jake-low

@jake-low jake-low commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

#1236 added a new completion_metrics JSONB column which is kept up to date by triggers, but it did not remove the scheduled job that maintains the old completion_percentage and tasks_remaining counts. Having both at once creates write contention on the challenges and projects tables and makes lots of API operations very slow while the job is running.

This PR does the following:

  • drops the completion_percentage and tasks_remaining columns (and computes those fields from the completion_metrics data when building JSON responses)
  • removes the updateCompletionMetricsOfActiveChallenges job which maintained those columns
  • changes when the update_challenge_completion_metrics and update_project_completion_metrics SQL functions are triggered, so updating a task (or challenge) will only recompute metrics for the parent challenge (or project) if a column was modified that could affect the results.

There are still problems with the present approach which we should fix in a future PR:

  • The metrics calculation SQL functions are triggered per row, not per statement. This means that a bulk operation (like inserting 10,000 new tasks) calls the function 10,000 times, when calling it once at the end would be sufficient.
  • The code paths for creating a challenge or refreshing tasks build task models one at a time and then INSERT or UPDATE them in individual queries. This is an antipattern on its own (refreshing a challenge with 50k tasks means 50k DB roundtrips), but it also means that even if we switch to per-statement triggers as described above, we still won't see any benefit until these code paths are refactored to insert/update tasks in batches.

@sonarqubecloud

sonarqubecloud Bot commented Jun 4, 2026

Copy link
Copy Markdown

@jake-low jake-low marked this pull request as ready for review June 4, 2026 21:55
@CollinBeczak

Copy link
Copy Markdown
Contributor

LGTM, i tested locally and things work well in both frontends. Will keep the problems you noted in mind.

@jake-low jake-low merged commit 0bdd80f into main Jun 4, 2026
9 checks passed
@jake-low jake-low deleted the jlow/refactor-metrics branch June 4, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants