feat(migrations): digest-batch columns on apify_scraper_runs (batch_id, completed_at, new_post_urls)

[sweetmantech]

Summary

Schema prerequisite for the one-digest-per-scrape design on recoupable/chat#1855 (PR #2 of 6): a roster scrape starts one Apify run per platform, each completing independently by webhook — to send one consolidated email instead of 4+, completions need to find their siblings (batch_id), know when the batch is done (completed_at), and know what each run found (new_post_urls, diffed against existing posts so only genuinely-new content digests).

Nullable columns, no FKs, no backfill — rows predating the migration never digest, matching the table's loose-ids stance from chat#1840. Partial index on batch_id for the sibling lookup. Additive and safe to apply independently of the api PRs (columns sit unused until the aggregation PR lands).

🤖 Generated with Claude Code

---

Summary by cubic

Add digest-batch columns on apify scraper runs to group per-scrape runs and send one consolidated “new posts” digest. Adds a partial index for fast sibling lookups.

• Migration
  • Added columns: batch_id (UUID), completed_at (timestamptz), new_post_urls (JSONB); all nullable.
  • Created partial index on batch_id where not null.
  • No FKs and no backfill; existing rows won’t digest. Additive and safe until aggregation logic ships.

^{Written for commit b09499a. Summary will update on new commits.}

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[coderabbitai]

Warning

Review limit reached

@sweetmantech, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 26 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e9702934-cbb6-426b-995d-4cef2f4894b1

📥 Commits

Reviewing files that changed from the base of the PR and between dcd0695 and b09499a.

📒 Files selected for processing (1)

• supabase/migrations/20260707090000_apify_runs_digest_batch_columns.sql

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/apify-runs-digest-batch-columns

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[supabase]

Updates to Preview Branch (feat/apify-runs-digest-batch-columns) ↗︎

Deployments	Status	Updated
Database	✅	Mon, 06 Jul 2026 21:57:14 UTC
Services	✅	Mon, 06 Jul 2026 21:57:14 UTC
APIs	✅	Mon, 06 Jul 2026 21:57:14 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks	Status	Updated
Configurations	✅	Mon, 06 Jul 2026 21:57:25 UTC
Migrations	❌	Mon, 06 Jul 2026 21:57:30 UTC
Seeding	⏸️	Mon, 06 Jul 2026 21:57:07 UTC
Edge Functions	⏸️	Mon, 06 Jul 2026 21:57:07 UTC

---

❌ Branch Error • Mon, 06 Jul 2026 21:57:31 UTC

ERROR: relation "public.apify_scraper_runs" does not exist (SQLSTATE 42P01)
At statement: 0
-- Digest-batch columns for apify_scraper_runs (recoupable/chat#1855, PR #2 of 6).
--
-- One artist-socials scrape (POST /api/artist/socials/scrape) starts one Apify
-- run per platform; each completes independently via webhook. To send ONE
-- consolidated "new posts" digest per scrape instead of one email per platform,
-- completions must find their sibling runs and know what each found:
--   batch_id       — shared id minted per scrape call; siblings share it.
--   completed_at   — set by the webhook when the run's results are processed;
--                    the batch's last writer assembles and sends the digest.
--   new_post_urls  — post URLs genuinely new to the platform found by this run
--                    (diffed against posts before insert); the digest's content.
--
-- Nullable, no FKs, no backfill: rows predating this migration simply never
-- digest (same loose-ids stance as the table's creation in chat#1840).

ALTER TABLE public.apify_scraper_runs
    ADD COLUMN IF NOT EXISTS batch_id      UUID,
    ADD COLUMN IF NOT EXISTS completed_at  TIMESTAMP WITH TIME ZONE,
    ADD COLUMN IF NOT EXISTS new_post_urls JSONB

---

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

[cubic-dev-ai]

No issues found across 1 file

_{Re-trigger cubic}

[sweetmantech]

Closing unmerged — this migration ALTERs public.apify_scraper_runs, but that table was never created: #39 (which would have created it for chat#1840) was closed unmerged on 2026-07-03 when run polling moved to the capability model. Applying this to prod fails with 42P01: relation "public.apify_scraper_runs" does not exist (verified live 2026-07-09). It also under-specifies the consumer contract — recoupable/api#760 reads/writes account_id, social_id, and platform in addition to the three columns here.

Superseded by #47, which creates the table matching api#760's contract, scoped to digest bookkeeping only (the chat#1840 capability-model decision stands).