[SEA-NodeJS] (6/8) CloudFetch + Inline Arrow result fetching

[msrathore-db]

Stack

Linear stack of 8 PRs landing the M0 + M1 Phase 1 SEA NodeJS work. Merge in order from base ↑ to tip. The tip branch (msrathore/sea-auth-u2m, PR #383) is the single snapshot containing everything in flight — point your test or benchmark harness at it for an end-to-end check.

Pos	PR	Branch	Scope
1/8	#378	sea-abstraction	IBackend / ISessionBackend / IOperationBackend interfaces
2/8	#380	sea-napi-binding	TS loader + build script for the kernel-provided .node artifact
3/8	#377	sea-errors-logging	Kernel ErrorCode → JS error-class mapping (M0 minimum)
4/8	#379	sea-auth	PAT auth via useSEA: true
5/8	#382	sea-execution	executeStatement + openSession (sessionConfig, initialCatalog/Schema)
6/8	#381	sea-results	CloudFetch + Inline Arrow result fetching
7/8	#384	sea-operation	cancel / close / finished lifecycle + INTERVAL parity + napi-relocation acceptance (absorbed sea-integration content)
8/8	#383	sea-auth-u2m ← TIP	M1 Phase 1 OAuth M2M + U2M (5 review rounds, ZERO HIGH at close)

Companion kernel stack (databricks/databricks-sql-kernel): 8 PRs — root #26 (async-public-api) → #27 → #25 → #29 → #28 → #30 → #24 → #23 (tip).

Policy: new PRs always stack on the current tip. No sibling/parallel topology. No force-pushes on existing PRs unless absolutely necessary; if a PR's content is wrong, add a fix-up commit on top of the stack tip rather than rewriting history.

---

This PR is position 6/8.

Summary

SeaOperationBackend + SeaResultsProvider + SeaArrowIpc wire the kernel's CloudFetch and Inline Arrow paths to JS row-shape conversion.

Size note (964 LOC, marginally over cap)

CloudFetch and Inline Arrow share too much (ArrowResultConverter integration) to split cleanly. Reviewable in one pass.

Downstream fixes / reviewer note

• This PR is the initial result-fetching layer. Final INTERVAL result parity uses kernel Parameterized queries #96's intervalsAsString knob and is wired in feat(sea): Thrift-parity — params, intervals, getInfo, SQL-error class, input validation + queryTags #403; reviewers do not need to request that final interval behavior in this lower result-fetching PR.
• SQL-error class parity and true async lifecycle/cancel behavior are also fixed downstream in feat(sea): Thrift-parity — params, intervals, getInfo, SQL-error class, input validation + queryTags #403 and feat(sea): execute statements asynchronously (submit + poll) for Thrift runAsync parity #406 respectively.
• 2026-06-01 review-fix cascade: the IOperationBackend contract mismatch called out on this PR is fixed on this branch and carried forward through [SEA-NodeJS] (7/8) Operation lifecycle — cancel / close / finished + INTERVAL parity + napi relocation #384/[SEA-NodeJS] (8/8) M1 Phase 1 — OAuth M2M + OAuth U2M (5 review rounds, ZERO HIGH at close) #383/feat(sea): expose maxConnections + lock complex types as Arrow default #392/feat(sea): queryTags through executeStatement + fetchAll regression lock #393/feat(sea): expose TLS verification toggle + custom CA on ConnectionOptions #407/feat(sea): wire rowLimit + statementConf + TIMESTAMP_NTZ/LTZ params #408; SEA status/metadata now return the neutral DTO shape while the result converter still uses the Thrift metadata internally.
• 2026-06-01 review-fix cascade: the old per-statement useCloudFetch plumbing has been removed from the SEA execution path and unsupported useCloudFetch is rejected rather than silently ignored in the downstream stack.

Test plan

• ✅ M0 parity gate clean against pecotesting at 23.2 MB/s on a 100k×17-column run
• ✅ Default-on CloudFetch; per-statement useCloudFetch override forwarded via session_conf
• ✅ Inline Arrow result fetching same converter path as thrift

Draft until you give the go for review.

Tracking

• PECOBLR-2875 — CloudFetch link refresh on 401
• PECOBLR-2876 — Inline Large Results threshold control
• Parent epic: PECOBLR-2666

Co-authored-by: Isaac

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[github-actions]

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

[gopalldb]

🔴 P0 (Critical) — verified directly, must fix

status() and getResultMetadata() return Thrift-shaped objects instead of the neutral OperationStatus/ResultMetadata DTOs the contract requires — silently corrupts the public status()/getMetadata() output.
lib/sea/SeaOperationBackend.ts:169-175 and :177-206

I traced and executed the data flow to confirm this is real (not theoretical):

• The interface on this branch (IOperationBackend.ts:45,48) requires status(): Promise (which carries .state: OperationState — a string enum) and getResultMetadata(): Promise
  (carries .resultFormat: ResultFormat — a string enum).
• SeaOperationBackend.status() instead returns { status, operationState: TOperationState, hasResultSet } — no .state field.
• The facade (DBSQLOperation.ts:154) feeds it to synthesizeThriftStatus, which reads status.state (thriftWireSynthesis.ts:62). That field is undefined → falls through to UKNOWN_STATE. So operation.status()
  always reports UNKNOWN for SEA.
• getResultMetadata() returns resultFormat: TSparkRowSetType.ARROW_BASED_SET (the number 1). synthesizeThriftResultSetMetadata → resultFormatToThrift switches on the ResultFormat string enum
  (thriftWireSynthesis.ts:36-46); a number matches nothing → default → COLUMN_BASED_SET. So getMetadata() always reports the wrong result format.

▎ ⚠️ One correction to the agent's report: it called this a "compile break." It is not — I ran npx tsc --noEmit and it passes clean (TS method-return bivariance under implements lets the redeclared Thrift
▎ return types slip through). That makes this more dangerous, not less: a silent runtime bug with no compiler safety net.

Fix: return neutral DTOs —
status() → { state: this.cancelled ? OperationState.Cancelled : this.closed ? OperationState.Closed : OperationState.Succeeded, hasResultSet: true }
getResultMetadata() → { schema: thriftSchema, resultFormat: ResultFormat.ArrowBased, lz4Compressed: false, isStagingOperation: false }. getResultSlicer() keeps working since it only reads metadata.schema.

🟠 P1 (Important — should fix)

1. Fetch-time kernel errors are not mapped to typed driver errors — SeaResultsProvider.ts:95, SeaOperationBackend.ts:185. cancel()/close() wrap napi calls in rethrowKernelError, but fetchNextBatch() and
   schema() are unguarded. A mid-fetch failure (auth revoked, network, DataLoss) leaks as a raw napi Error with a sentinel-prefixed .reason instead of a typed HiveDriverError/AuthenticationError with SQLSTATE.
   → wrap both in try/catch → rethrowKernelError.
2. waitUntilReady never throws OperationStateError on terminal non-success — SeaOperationBackend.ts:157-167. The contract (IOperationBackend.ts:30-39) says it must throw on Canceled/Closed/Error so the
   facade mirrors state; the SEA impl always resolves. Lower live impact (the facade also sets its own flags in cancel()/close()), but diverges from contract and ThriftOperationBackend.
3. cancel() "set flag before await" comment describes behavior that doesn't exist — SeaOperationBackend.ts:212-213 (and class doc :90-95). No code in fetchChunk (:141-150) or SeaResultsProvider reads
   this.cancelled, and per the kernel's FIFO tokio::Mutex (native/sea/index.d.ts) cancel() queues behind an in-flight fetch anyway. Cancel cannot interrupt an active fetch — either implement the flag-check
   short-circuit or remove the misleading claim.

🟡 P2 (Minor)

• CloudFetch path is never exercised by a CI-runnable test — unit tests feed only small inline IPC batches; the e2e (results-e2e.test.ts:62) is a tiny SELECT 1 gated/skipped without DATABRICKS_PECOTESTING_*
  creds. The prime() empty-batch-skip recursion (SeaResultsProvider.ts:102-107) is untested. Add a unit test with interleaved zero-row batches.
• limit ignored by SeaResultsProvider.fetchNext (:75) — a full kernel batch is materialized before ResultSlicer trims it. Output is correct; memory is kernel-batch-bounded, not limit-bounded. Acceptable for
  M0; worth a comment.
• arrowTypeToTTypeId Float comment wrong (SeaArrowIpc.ts:177-178) — comment says "16=HALF, 32=SINGLE, 64=DOUBLE" but code correctly keys off the Precision enum (=== 2 → DOUBLE). Behavior right, comment
  misleading.

[msrathore-db]

Superseded — this layer is folded into the consolidated SEA foundation, now split into #409 → #410 → #411 (stacked on main).