SQLite PoC + provenance-tagged AST#80
Merged
Merged
Conversation
ryanrasti
commented
Jul 1, 2026
ryanrasti
commented
Owner
- Shared SqlValue base extracted from PG's Any. Dialect is now { name, root, bool }; DialectName is the string tag.
- SQLite inference tool (src/types/sqlite/generate.ts) — reads pragma_function_list, probes with a typed input matrix, emits functions.json + class stubs. LLM-over-docs plan in the file header for later cleanup.
- SqliteDriver — better-sqlite3, row-value normalization at the boundary.
- Sweep: instanceof Any → SqlValue across builders/util; runtime.ts utilities widened.
There was a problem hiding this comment.
Pull request overview
This PR introduces a proof-of-concept SQLite dialect alongside Postgres, extracting a shared SqlValue<N> base for typed SQL expressions and making compilation/deserialization dialect-aware.
Changes:
- Added SQLite dialect scaffolding (base type, deserialize registry, generated type stubs) plus end-to-end smoke tests using
better-sqlite3. - Refactored core type system to use shared
SqlValue+ a dialect-aware type registry and compile context (pgCtx/sqliteCtx). - Reorganized Postgres type generation into
src/types/postgres/*, updating generated/override imports and runtime helpers (funcCall/opCall/Srf).
Reviewed changes
Copilot reviewed 24 out of 124 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/util.ts | Updates runtime typed-expression checks from Any to SqlValue. |
| src/types/sqlite/table.test.ts | Adds SQLite mutation + Table/QueryBuilder smoke tests. |
| src/types/sqlite/smoke.test.ts | Adds SQLite end-to-end expression/compile/execute smoke tests. |
| src/types/sqlite/index.ts | SQLite dialect barrel export for generated + hand-written entries. |
| src/types/sqlite/generated/text.ts | Generated SQLite Text typed value methods. |
| src/types/sqlite/generated/bool.ts | Temporary SQLite Bool stub (Phase 1 placeholder). |
| src/types/sqlite/deserialize.ts | Registers SQLite typname→TS-type/deserializer mapping. |
| src/types/sqlite/base.ts | Defines SqliteValue root and dialect descriptor wiring. |
| src/types/runtime.ts | Generalizes nullability/type utilities to SqlValue and adds generic runtime call helpers. |
| src/types/postgres/overrides/record.ts | Fixes import paths after Postgres codegen rehome. |
| src/types/postgres/overrides/bool.ts | Refactors Bool ops to use shared bool helpers and updated runtime import paths. |
| src/types/postgres/overrides/anycompatiblearray.ts | Updates runtime/sql imports for new Postgres module layout. |
| src/types/postgres/overrides/anyarray.ts | Updates runtime/sql/exoeval imports for new Postgres module layout. |
| src/types/postgres/overrides/any.ts | Introduces PG root Any override carrying dialect metadata + pg-specific bool-returning methods. |
| src/types/postgres/introspect.ts | Adds Postgres introspection utilities for codegen. |
| src/types/postgres/index.ts | New Postgres barrel exporting generated + overrides. |
| src/types/postgres/index.test.ts | Updates tests to new compile context API and import paths. |
| src/types/postgres/generated/xml.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/xid8.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/xid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/varchar.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/varbit.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/uuid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/unknown.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/txid_snapshot.ts | Regenerated PG type bindings to use runtime.Srf and new runtime call helpers. |
| src/types/postgres/generated/tid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/regtype.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regrole.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regprocedure.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regproc.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regoperator.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regoper.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regnamespace.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regdictionary.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regconfig.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/regcollation.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/regclass.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/refcursor.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_snapshot.ts | Regenerated PG type bindings to use runtime.Srf and new runtime call helpers. |
| src/types/postgres/generated/pg_node_tree.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_ndistinct.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_mcv_list.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_dependencies.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_brin_minmax_multi_summary.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/pg_brin_bloom_summary.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/jsonpath.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/json.ts | Regenerated PG type bindings to use runtime.funcCall/opCall/Srf and new import paths. |
| src/types/postgres/generated/gtsvector.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/cidr.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/cid.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/char.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/bool.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anynonarray.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/anyenum.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anyelement.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/anycompatiblerange.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblenonarray.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblemultirange.ts | Regenerated PG type bindings import paths. |
| src/types/postgres/generated/anycompatiblearray.ts | Regenerated PG type bindings to use runtime.funcCall/opCall and new import paths. |
| src/types/postgres/generated/anycompatible.ts | Regenerated PG type bindings to use runtime.funcCall and new import paths. |
| src/types/postgres/generated/any.ts | Makes generated PG Any extend shared SqlValue and updates runtime call helpers. |
| src/types/postgres/generated/aclitem.ts | Regenerated PG type bindings to use runtime.opCall and new import paths. |
| src/types/postgres/generate.ts | Updates codegen to new Postgres module layout + new runtime helper names + dialect-aware typedef lookup. |
| src/types/postgres/deserialize.ts | Adds Postgres deserialize/type registry registration. |
| src/types/meta.ts | Introduces standalone meta symbol module to avoid load-order cycles. |
| src/types/match.test.ts | Updates tests to new compile context API. |
| src/types/index.ts | Transitional re-export to new Postgres barrel (./postgres). |
| src/types/deserialize.ts | Replaces single registry with dialect-aware registry + registration API. |
| src/types/bool.ts | Adds dialect-agnostic Bool marker interface + isBool helper. |
| src/types/bool-mixin.ts | Adds shared bool operator helpers for dialect-specific Bool classes. |
| src/types/any.ts | Extracts shared SqlValue base with dialect metadata and common typed-value behaviors. |
| src/tables/generate.ts | Updates introspection import path to new Postgres location. |
| src/table.ts | Switches column detection to dialect-agnostic isColumn from types/any. |
| src/driver.ts | Adds dialect to Driver, introduces SqliteDriver adapter, and updates compile to dialect context. |
| src/builder/values.ts | Replaces Any checks with SqlValue for VALUES builder row typing/coercion. |
| src/builder/update.ts | Makes UPDATE builder dialect-aware (ctx threading) and uses shared Bool validation. |
| src/builder/sql.ts | Introduces CompileContext, dialect-aware param emission, and a Cast node. |
| src/builder/sql.test.ts | Updates compile tests to use pgCtx/sqliteCtx. |
| src/builder/query.test.ts | Updates compile usage to CompileContext and adjusts validation expectations. |
| src/builder/insert.ts | Threads compile context through bind/debug and updates getColumn import. |
| src/builder/delete.ts | Threads compile context through bind/debug and uses shared Bool validation. |
| package.json | Moves codegen script to Postgres module path and adds better-sqlite3 deps/types. |
| eslint.config.js | Updates ignore path to new Postgres generated directory. |
| docs/ISSUES.md | Documents RPC security considerations for cross-dialect/tenant typed value provenance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+196
to
+211
| const normalizeValue = (v: unknown): string => { | ||
| if (v === null || v === undefined) { | ||
| // Preserved as null via cast; downstream checks for null-ness. | ||
| return v as unknown as string; | ||
| } | ||
| if (typeof v === "string") {return v;} | ||
| if (typeof v === "number" || typeof v === "bigint" || typeof v === "boolean") {return String(v);} | ||
| if (v instanceof Uint8Array) { | ||
| // Match PG bytea repr: \x-prefixed lowercase hex. | ||
| return "\\x" + Buffer.from(v).toString("hex"); | ||
| } | ||
| return String(v); | ||
| }; | ||
|
|
||
| const normalizeRow = (row: { [key: string]: unknown }): { [key: string]: string } => | ||
| Object.fromEntries(Object.entries(row).map(([k, v]) => [k, normalizeValue(v)])); |
Comment on lines
+22
to
+26
| // Blob deserialize is a placeholder — the driver returns Buffer | ||
| // directly, so this path is only used if a raw string sneaks in. | ||
| // Real repr TBD (Buffer? Uint8Array? hex? see plan.md A1). | ||
| blob: { tsType: "object", deserialize: identity }, | ||
| bool: { tsType: "boolean", deserialize: (raw) => raw === "1" }, |
Comment on lines
+42
to
+46
| const rows = await Users.from().execute(db); | ||
| expect(rows.length).toBe(2); | ||
| // Rows come out in insertion order (SQLite rowid ordering). | ||
| expect(rows[0]!.name).toBe("alice"); | ||
| expect(rows[1]!.name).toBe("bob"); |
Comment on lines
+57
to
+60
| const rows = await Users.from() | ||
| .select((u) => ({ upperName: u.users.name.upper() })) | ||
| .execute(db); | ||
| expect(rows.map((r) => r.upperName)).toEqual(["ALICE", "BOB", "CAROL"]); |
| @@ -9,6 +11,7 @@ export type QueryResult = { rows: { [key: string]: string }[] }; | |||
| export type ExecuteFn = (query: Sql) => Promise<QueryResult>; | |||
Comment on lines
11
to
14
| import { Database } from "../database"; | ||
| import { Any, getColumn } from "../types/overrides/any"; | ||
| import { SqlValue, getColumn } from "../types/any"; | ||
| import { fn, expose } from "../exoeval/tool"; | ||
| import z from "zod"; |
Comment on lines
+82
to
+83
| @expose(z.union([z.literal(true), fn.returns(z.custom<SharedBool<any>>(isBool))])) | ||
| where(fn: ((ns: Namespace<Name, T>) => SharedBool<any>) | true): UpdateBuilder<Name, T, R> { |
ryanrasti
force-pushed
the
ryan_sqlite
branch
3 times, most recently
from
e45482a to
695332d
Compare
- SQLite dialect: pragma_function_list inference, codegen'd Integer/Real/ Text/Blob/Bool with runtime.match dispatch, SqliteDriver via better-sqlite3, Table + QueryBuilder + INSERT/UPDATE/DELETE + RETURNING end-to-end. - Provenance: every schema-referencing Ident carries its Database; every Func/Op/UnaryOp/Cast/TypedParam/Srf carries its dialect. Cross-DB and cross-dialect misuse throw at compile-time; no untagged escape hatches in the public API. AND/OR/NOT thread the caller's dialect through the boolAnd/boolOr/boolNot helpers. - Database / Connection split: Database is an immutable metadata handle (dialect + scopedIdent factory); db.attach(driver) returns a runtime Connection. Multiple attach calls allowed. Table classes come from db.Table only — the free Table() factory is gone. - Deserialize registry replaced with per-class overrides (overrides/bool.ts, int2/int4/float4/float8/oid, sqlite integer/real/ bool). SqlValue exposes a static `primitiveTs` for serialize/match.
ryanrasti
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.