perf: replace UUIDv4 with UUIDv7 for ordered primary keys#511
perf: replace UUIDv4 with UUIDv7 for ordered primary keys#511Mathew2k-hash wants to merge 1 commit into
Conversation
- Add src/utils/idGenerator.ts with UUIDv7 implementation - UUIDv7 is timestamp-ordered, reducing B-tree index fragmentation in PostgreSQL compared to random UUIDv4 - Includes monotonic counter for strict ordering within same millisecond - Replace all crypto.randomUUID() calls across controllers, services, and jobs
|
@Mathew2k-hash Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
📝 WalkthroughWalkthroughIntroduces a new ChangesUUIDv7 ID Generator Rollout
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/controllers/complianceController.ts`:
- Line 92: The tombstoneSuffix generation in the complianceController is using
the first 8 characters of generateId() which captures the time-derived prefix of
UUIDv7-style IDs, making the suffix predictable and prone to collisions during
burst deletes. Fix this by changing the substring call in the tombstoneSuffix
assignment to use the random tail bits of the ID instead of the leading
timestamp portion, such as using substring with indices that target the end of
the generated ID rather than the beginning.
In `@src/controllers/onrampController.ts`:
- Line 8: Fix the Prettier formatting violations in the onrampController.ts
file. On line 8, adjust the import statement for generateId to comply with
Prettier's formatting rules by checking line length, spacing, and proper
formatting. Additionally, on lines 127-129, fix the correlation fallback
formatting to meet ESLint/Prettier standards by reviewing the multi-line
construct formatting, indentation, and spacing. Run Prettier on the file to
automatically format these sections correctly, or manually adjust the formatting
to match your project's Prettier configuration (including proper line breaks,
indentation, and spacing conventions).
In `@src/controllers/webhookController.ts`:
- Line 17: The import statement for generateId on line 17 and the code sections
at lines 221-223 and 302-304 in webhookController.ts have Prettier formatting
violations. Run Prettier on the entire file to automatically resolve all
formatting issues across these sections. This will ensure consistent formatting
throughout the file in compliance with the project's Prettier configuration.
In `@src/jobs/withdrawalProcessingJob.ts`:
- Line 5: The import statement for generateId from '../utils/idGenerator' on
line 5 uses single quotes which does not comply with Prettier's quote style
configuration. Change the single quotes around the import path
'../utils/idGenerator' to double quotes to satisfy Prettier's formatting rules.
In `@src/jobs/xlmToAcbuJob.ts`:
- Line 12: The import statement for generateId from '../utils/idGenerator' on
line 12 violates Prettier formatting rules due to incorrect quote style.
Normalize the quotes around the module path in this import statement to match
your project's Prettier configuration, likely requiring double quotes instead of
single quotes around the module path string.
In `@src/services/auth/authService.ts`:
- Around line 892-893: The refresh-token secret generation in the code around
lines 892-893 is using generateId() which produces predictable tokens with
embedded timestamp/counter structure. Replace the
Buffer.from(generateId()).toString('base64') calls used for the refresh-token
secret with cryptographically random bytes generated from a CSPRNG (such as
crypto.randomBytes()). Keep using generateId() only for the tokenFamilyId if
present, but ensure the actual refresh-token secret portion uses properly random
cryptographic bytes instead.
In `@src/services/bills/billsService.ts`:
- Line 3: The import statement for generateId from the idGenerator utility is
using single quotes around the module path, which violates the configured
Prettier quote style rule. Change the single quotes to double quotes in the
import statement where generateId is imported from '../../utils/idGenerator' to
match the project's quote style configuration.
In `@src/services/bills/simulatedBillsPartner.ts`:
- Line 10: The import statement for generateId from utils/idGenerator on line 10
uses single quotes but does not match the project's Prettier configuration for
quote style. Change the quotes around the import path '../../utils/idGenerator'
to match the Prettier config (typically double quotes are the default). Update
the import statement to use the correct quote style consistently with the rest
of the codebase.
In `@src/utils/idGenerator.ts`:
- Around line 24-25: The timestamp fields in the UUIDv7 generation are
incorrectly ordered, breaking lexical ordering. In the timeLow and timeMid
variable assignments (lines 24-25 and also at lines 32-33), swap the bit
extraction logic so that timeLow receives the upper 32 bits of the millisecond
timestamp (ms right-shifted by 16) and timeMid receives the lower 16 bits (ms
masked with 0xffff). This ensures the first 48 bits of the UUID represent the
millisecond timestamp in proper big-endian order, which is required for UUIDv7
index-locality behavior.
- Around line 15-21: The monotonic ordering logic does not handle the case where
the current timestamp moves backward (ms < lastMs), which can occur due to time
skew or NTP adjustments. Currently, the else branch unconditionally resets
lastMs to the new (earlier) value, breaking monotonicity. Add a condition to
check if ms is less than lastMs in addition to checking if ms equals lastMs.
When the clock moves backward, either wait for the lastMs timestamp to elapse or
use lastMs as the basis for the ID and increment the sequence counter to ensure
uniqueness, rather than resetting the sequence. This ensures the ID generator
maintains strictly monotonic ordering regardless of system clock adjustments.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6a2de120-9a74-45da-8e6d-a5943a40bdfb
📒 Files selected for processing (13)
src/controllers/complianceController.tssrc/controllers/kycController.tssrc/controllers/onrampController.tssrc/controllers/userController.tssrc/controllers/webhookController.tssrc/jobs/withdrawalProcessingJob.tssrc/jobs/xlmToAcbuJob.tssrc/services/auth/authService.tssrc/services/bills/billsService.tssrc/services/bills/simulatedBillsPartner.tssrc/services/salary/salaryService.tssrc/services/transfer/transferService.tssrc/utils/idGenerator.ts
|
|
||
| // 2. Tombstone the User record | ||
| const tombstoneSuffix = crypto.randomUUID().substring(0, 8); | ||
| const tombstoneSuffix = generateId().substring(0, 8); |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Avoid timestamp-prefixed tombstone suffixes
Line 92 takes substring(0, 8) from generateId(). For UUIDv7-style IDs, the prefix is time-derived, so suffixes become predictable and can collide under burst deletes. Use random tail bits instead of the leading timestamp portion.
Suggested fix
- const tombstoneSuffix = generateId().substring(0, 8);
+ const tombstoneSuffix = generateId().replace(/-/g, "").slice(-8);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const tombstoneSuffix = generateId().substring(0, 8); | |
| const tombstoneSuffix = generateId().replace(/-/g, "").slice(-8); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/controllers/complianceController.ts` at line 92, The tombstoneSuffix
generation in the complianceController is using the first 8 characters of
generateId() which captures the time-derived prefix of UUIDv7-style IDs, making
the suffix predictable and prone to collisions during burst deletes. Fix this by
changing the substring call in the tombstoneSuffix assignment to use the random
tail bits of the ID instead of the leading timestamp portion, such as using
substring with indices that target the end of the generated ID rather than the
beginning.
| import { Response, NextFunction } from "express"; | ||
| import { z } from "zod"; | ||
| import crypto from "crypto"; | ||
| import { generateId } from '../utils/idGenerator'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix Prettier violations on the new import and correlation fallback formatting.
Line 8 and Lines 127-129 are currently failing ESLint/Prettier.
Suggested patch
-import { generateId } from '../utils/idGenerator';
+import { generateId } from "../utils/idGenerator";
@@
- const correlationId =
- (req.headers["x-request-id"] as string | undefined) ??
- generateId();
+ const correlationId =
+ (req.headers["x-request-id"] as string | undefined) ?? generateId();Also applies to: 127-129
🧰 Tools
🪛 ESLint
[error] 8-8: Replace '../utils/idGenerator' with "../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/controllers/onrampController.ts` at line 8, Fix the Prettier formatting
violations in the onrampController.ts file. On line 8, adjust the import
statement for generateId to comply with Prettier's formatting rules by checking
line length, spacing, and proper formatting. Additionally, on lines 127-129, fix
the correlation fallback formatting to meet ESLint/Prettier standards by
reviewing the multi-line construct formatting, indentation, and spacing. Run
Prettier on the file to automatically format these sections correctly, or
manually adjust the formatting to match your project's Prettier configuration
(including proper line breaks, indentation, and spacing conventions).
Source: Linters/SAST tools
| } | ||
| import { Request, Response, NextFunction } from "express"; | ||
| import crypto from "crypto"; | ||
| import { generateId } from '../utils/idGenerator'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Resolve ESLint/Prettier formatting errors in the touched webhook paths.
Line 17 and Lines 221-223/302-304 are flagged by Prettier in the current diff.
Suggested patch
-import { generateId } from '../utils/idGenerator';
+import { generateId } from "../utils/idGenerator";
@@
- const paystackCorrelationId =
- (req.headers["x-request-id"] as string | undefined) ??
- generateId();
+ const paystackCorrelationId =
+ (req.headers["x-request-id"] as string | undefined) ?? generateId();
@@
- const flwCorrelationId =
- (req.headers["x-request-id"] as string | undefined) ??
- generateId();
+ const flwCorrelationId =
+ (req.headers["x-request-id"] as string | undefined) ?? generateId();Also applies to: 221-223, 302-304
🧰 Tools
🪛 ESLint
[error] 17-17: Replace '../utils/idGenerator' with "../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/controllers/webhookController.ts` at line 17, The import statement for
generateId on line 17 and the code sections at lines 221-223 and 302-304 in
webhookController.ts have Prettier formatting violations. Run Prettier on the
entire file to automatically resolve all formatting issues across these
sections. This will ensure consistent formatting throughout the file in
compliance with the project's Prettier configuration.
Source: Linters/SAST tools
| * disburse via fintech, update transaction status, optionally publish user notification. | ||
| */ | ||
| import { randomUUID } from "crypto"; | ||
| import { generateId } from '../utils/idGenerator'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix import quote style to satisfy Prettier.
Line 5 is currently flagged by ESLint/Prettier.
Suggested patch
-import { generateId } from '../utils/idGenerator';
+import { generateId } from "../utils/idGenerator";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import { generateId } from '../utils/idGenerator'; | |
| import { generateId } from "../utils/idGenerator"; |
🧰 Tools
🪛 ESLint
[error] 5-5: Replace '../utils/idGenerator' with "../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/jobs/withdrawalProcessingJob.ts` at line 5, The import statement for
generateId from '../utils/idGenerator' on line 5 uses single quotes which does
not comply with Prettier's quote style configuration. Change the single quotes
around the import path '../utils/idGenerator' to double quotes to satisfy
Prettier's formatting rules.
Source: Linters/SAST tools
| import { mintFromUsdcInternal } from "../controllers/mintController"; | ||
| import { fetchXlmRateUsd } from "../services/oracle/cryptoClient"; | ||
| import { randomUUID } from "crypto"; | ||
| import { generateId } from '../utils/idGenerator'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Normalize import quotes to pass Prettier.
Line 12 is reported as a Prettier violation.
Suggested patch
-import { generateId } from '../utils/idGenerator';
+import { generateId } from "../utils/idGenerator";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import { generateId } from '../utils/idGenerator'; | |
| import { generateId } from "../utils/idGenerator"; |
🧰 Tools
🪛 ESLint
[error] 12-12: Replace '../utils/idGenerator' with "../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/jobs/xlmToAcbuJob.ts` at line 12, The import statement for generateId
from '../utils/idGenerator' on line 12 violates Prettier formatting rules due to
incorrect quote style. Normalize the quotes around the module path in this
import statement to match your project's Prettier configuration, likely
requiring double quotes instead of single quotes around the module path string.
Source: Linters/SAST tools
| const bytes = Buffer.from(generateId()).toString('base64'); | ||
| return bytes + Buffer.from(generateId()).toString('base64'); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Use CSPRNG bytes for refresh-token secrets
Line 892 and Line 893 now build bearer refresh tokens from generateId(), which embeds timestamp/counter structure. That makes tokens more predictable than necessary and leaks issuance timing. Keep generateId() for tokenFamilyId, but generate refresh-token secrets from cryptographically random bytes.
Suggested fix
function generateSecureRefreshToken(): string {
- const bytes = Buffer.from(generateId()).toString('base64');
- return bytes + Buffer.from(generateId()).toString('base64');
+ return randomBytes(48).toString("base64url");
}🧰 Tools
🪛 ESLint
[error] 892-892: Replace 'base64' with "base64"
(prettier/prettier)
[error] 893-893: Replace 'base64' with "base64"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/auth/authService.ts` around lines 892 - 893, The refresh-token
secret generation in the code around lines 892-893 is using generateId() which
produces predictable tokens with embedded timestamp/counter structure. Replace
the Buffer.from(generateId()).toString('base64') calls used for the
refresh-token secret with cryptographically random bytes generated from a CSPRNG
(such as crypto.randomBytes()). Keep using generateId() only for the
tokenFamilyId if present, but ensure the actual refresh-token secret portion
uses properly random cryptographic bytes instead.
| @@ -1,5 +1,6 @@ | |||
| import { Decimal } from "@prisma/client/runtime/library"; | |||
| import { prisma } from "../../config/database"; | |||
| import { generateId } from '../../utils/idGenerator'; | |||
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Fix import quote style flagged by ESLint/Prettier.
Line 3 is failing the configured Prettier rule.
Suggested patch
-import { generateId } from '../../utils/idGenerator';
+import { generateId } from "../../utils/idGenerator";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import { generateId } from '../../utils/idGenerator'; | |
| import { generateId } from "../../utils/idGenerator"; |
🧰 Tools
🪛 ESLint
[error] 3-3: Replace '../../utils/idGenerator' with "../../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/bills/billsService.ts` at line 3, The import statement for
generateId from the idGenerator utility is using single quotes around the module
path, which violates the configured Prettier quote style rule. Change the single
quotes to double quotes in the import statement where generateId is imported
from '../../utils/idGenerator' to match the project's quote style configuration.
Source: Linters/SAST tools
| PartnerBillRefundRequest, | ||
| PartnerBillRefundResponse, | ||
| } from "./types"; | ||
| import { generateId } from '../../utils/idGenerator'; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Adjust import quotes to match Prettier config.
Line 10 is flagged by ESLint/Prettier.
Suggested patch
-import { generateId } from '../../utils/idGenerator';
+import { generateId } from "../../utils/idGenerator";📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import { generateId } from '../../utils/idGenerator'; | |
| import { generateId } from "../../utils/idGenerator"; |
🧰 Tools
🪛 ESLint
[error] 10-10: Replace '../../utils/idGenerator' with "../../utils/idGenerator"
(prettier/prettier)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/bills/simulatedBillsPartner.ts` at line 10, The import statement
for generateId from utils/idGenerator on line 10 uses single quotes but does not
match the project's Prettier configuration for quote style. Change the quotes
around the import path '../../utils/idGenerator' to match the Prettier config
(typically double quotes are the default). Update the import statement to use
the correct quote style consistently with the rest of the codebase.
Source: Linters/SAST tools
| if (ms === lastMs) { | ||
| seq = (seq + 1) & 0xfff; | ||
| if (seq === 0) ms = ++lastMs; // counter overflow: advance clock | ||
| } else { | ||
| seq = 0; | ||
| lastMs = ms; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Monotonic ordering regresses when Date.now() moves backward.
The state machine only checks ms === lastMs. If ms < lastMs (NTP/time skew, or immediately after the overflow branch advanced lastMs), the else path resets lastMs backward and breaks monotonic ordering guarantees.
Suggested fix
export function generateId(): string {
let ms = BigInt(Date.now());
+ if (ms < lastMs) ms = lastMs;
+
if (ms === lastMs) {
seq = (seq + 1) & 0xfff;
- if (seq === 0) ms = ++lastMs; // counter overflow: advance clock
+ if (seq === 0) {
+ ms = lastMs + 1n; // counter overflow: advance logical clock
+ }
} else {
seq = 0;
- lastMs = ms;
}
+ lastMs = ms;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/utils/idGenerator.ts` around lines 15 - 21, The monotonic ordering logic
does not handle the case where the current timestamp moves backward (ms <
lastMs), which can occur due to time skew or NTP adjustments. Currently, the
else branch unconditionally resets lastMs to the new (earlier) value, breaking
monotonicity. Add a condition to check if ms is less than lastMs in addition to
checking if ms equals lastMs. When the clock moves backward, either wait for the
lastMs timestamp to elapse or use lastMs as the basis for the ID and increment
the sequence counter to ensure uniqueness, rather than resetting the sequence.
This ensures the ID generator maintains strictly monotonic ordering regardless
of system clock adjustments.
| const timeLow = Number(ms & BigInt(0xffffffff)); | ||
| const timeMid = Number((ms >> BigInt(32)) & BigInt(0xffff)); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win
Timestamp fields are swapped, so this is not UUIDv7-ordered.
Line 24/25 currently maps ms as low-32 then high-16. UUIDv7 requires the first 48 bits to be the millisecond timestamp in big-endian order, so the first 32 bits must be ms >> 16 and the next 16 bits ms & 0xffff. As written, lexical ordering will wrap every ~49.7 days and break the intended index-locality behavior.
Suggested fix
- const timeLow = Number(ms & BigInt(0xffffffff));
- const timeMid = Number((ms >> BigInt(32)) & BigInt(0xffff));
+ const timeLow = Number((ms >> BigInt(16)) & BigInt(0xffffffff));
+ const timeMid = Number(ms & BigInt(0xffff));Also applies to: 32-33
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/utils/idGenerator.ts` around lines 24 - 25, The timestamp fields in the
UUIDv7 generation are incorrectly ordered, breaking lexical ordering. In the
timeLow and timeMid variable assignments (lines 24-25 and also at lines 32-33),
swap the bit extraction logic so that timeLow receives the upper 32 bits of the
millisecond timestamp (ms right-shifted by 16) and timeMid receives the lower 16
bits (ms masked with 0xffff). This ensures the first 48 bits of the UUID
represent the millisecond timestamp in proper big-endian order, which is
required for UUIDv7 index-locality behavior.
|
@Mathew2k-hash fix conflict |
2 participants
perf: replace UUIDv4 with UUIDv7 for ordered primary keys
Summary
crypto.randomUUID() generates random (UUIDv4) identifiers, which scatter inserts randomly
across PostgreSQL B-tree index pages, causing fragmentation and degrading write performance
over time.
This PR introduces src/utils/idGenerator.ts — a zero-dependency UUIDv7 implementation using
Node.js built-in crypto. UUIDv7 embeds a 48-bit millisecond timestamp as the most-significant
bits, so new rows land on the tail of the index (sequential inserts), dramatically reducing
page splits and improving locality.
Changes
strict ordering within the same millisecond and safe clock-overflow handling
randomUUID() runtime calls replaced with generateId(); doc comments left unchanged
Testing
introduced by this change)
closes #439
Summary by CodeRabbit