Skip to content

Add client APIs to list instance IDs and read orchestration history#292

Open
bachuv wants to merge 6 commits into
mainfrom
vabachu/client-list-stream-history
Open

Add client APIs to list instance IDs and read orchestration history#292
bachuv wants to merge 6 commits into
mainfrom
vabachu/client-list-stream-history

Conversation

@bachuv

@bachuv bachuv commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Issue describing the changes in this PR

Adds client APIs to enumerate orchestration instances and read a completed instance's
history, plus a public history-event model.

Changes

  • DurableTaskClient.listInstanceIds(ListInstanceIdsQuery)ListInstanceIdsResult — page terminal/instance IDs by filter (completion window, statuses, continuation token).
  • DurableTaskClient.getOrchestrationHistory(String instanceId)List<HistoryEvent> — retrieve an instance's full ordered history.
  • New public model com.microsoft.durabletask.historyHistoryEvent plus per-type subclasses (ExecutionStarted/Completed, TaskScheduled/Completed/Failed, Timer*, SubOrchestration*, EntityOperation*/EntityLock*, EventSent/Raised, Generic, HistoryState, …).
  • Backed by the ListInstanceIds and StreamInstanceHistory gRPC operations (managed DTS; sidecar/emulator ≥ v0.4.22).
Notes
  • Foundation for a follow-up durable history-export feature (stacked PR).

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes are added to the CHANGELOG.md
  • I have added all required tests (Unit tests, E2E tests)

@bachuv
bachuv requested a review from a team as a code owner July 1, 2026 23:31
Copilot AI review requested due to automatic review settings July 1, 2026 23:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds new Durable Task client capabilities for bulk instance enumeration and orchestration history retrieval, along with a new public com.microsoft.durabletask.history domain model and conversion layer from protobuf history events.

Changes:

  • Adds DurableTaskClient.listInstanceIds(ListInstanceIdsQuery) and supporting query/result types for paging terminal instance IDs by completion-time window and status.
  • Adds DurableTaskClient.getOrchestrationHistory(String) backed by the sidecar StreamInstanceHistory gRPC operation, returning a typed public history-event model.
  • Introduces HistoryEventConverter plus unit/integration tests for history conversion and the new client APIs.

Reviewed changes

Copilot reviewed 40 out of 40 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
client/src/test/java/com/microsoft/durabletask/IntegrationTests.java Adds integration coverage for listing completed instance IDs and fetching typed history.
client/src/test/java/com/microsoft/durabletask/HistoryEventConverterTest.java Adds unit tests validating protobuf→domain history event conversion.
client/src/main/java/com/microsoft/durabletask/ListInstanceIdsResult.java New paging result model for instance ID enumeration.
client/src/main/java/com/microsoft/durabletask/ListInstanceIdsQuery.java New query model for completion-window instance ID listing.
client/src/main/java/com/microsoft/durabletask/HistoryEventConverter.java Implements protobuf→public domain model conversion for history events.
client/src/main/java/com/microsoft/durabletask/history/TraceContext.java Adds public model for W3C trace context on history events.
client/src/main/java/com/microsoft/durabletask/history/TimerFiredEvent.java Adds typed history event model for fired timers.
client/src/main/java/com/microsoft/durabletask/history/TimerCreatedEvent.java Adds typed history event model for created timers.
client/src/main/java/com/microsoft/durabletask/history/TaskScheduledEvent.java Adds typed history event model for scheduled activities.
client/src/main/java/com/microsoft/durabletask/history/TaskFailedEvent.java Adds typed history event model for failed activities.
client/src/main/java/com/microsoft/durabletask/history/TaskCompletedEvent.java Adds typed history event model for completed activities.
client/src/main/java/com/microsoft/durabletask/history/SubOrchestrationInstanceFailedEvent.java Adds typed history event model for failed sub-orchestrations.
client/src/main/java/com/microsoft/durabletask/history/SubOrchestrationInstanceCreatedEvent.java Adds typed history event model for created sub-orchestrations.
client/src/main/java/com/microsoft/durabletask/history/SubOrchestrationInstanceCompletedEvent.java Adds typed history event model for completed sub-orchestrations.
client/src/main/java/com/microsoft/durabletask/history/ParentInstanceInfo.java Adds model for parent-orchestration metadata referenced by history events.
client/src/main/java/com/microsoft/durabletask/history/OrchestratorStartedEvent.java Adds typed history event model for replay/episode start markers.
client/src/main/java/com/microsoft/durabletask/history/OrchestratorCompletedEvent.java Adds typed history event model for replay/episode end markers.
client/src/main/java/com/microsoft/durabletask/history/OrchestrationState.java Adds snapshot model carried by history state checkpoint events.
client/src/main/java/com/microsoft/durabletask/history/OrchestrationInstance.java Adds model identifying an orchestration instance (+ optional execution ID).
client/src/main/java/com/microsoft/durabletask/history/HistoryStateEvent.java Adds typed history event model for orchestration state snapshots.
client/src/main/java/com/microsoft/durabletask/history/HistoryEvent.java Adds base class for the public history event hierarchy.
client/src/main/java/com/microsoft/durabletask/history/GenericEvent.java Adds typed history event model for generic/free-form payload events.
client/src/main/java/com/microsoft/durabletask/history/ExecutionTerminatedEvent.java Adds typed history event model for termination.
client/src/main/java/com/microsoft/durabletask/history/ExecutionSuspendedEvent.java Adds typed history event model for suspension.
client/src/main/java/com/microsoft/durabletask/history/ExecutionStartedEvent.java Adds typed history event model for orchestration start.
client/src/main/java/com/microsoft/durabletask/history/ExecutionRewoundEvent.java Adds typed history event model for rewind operations.
client/src/main/java/com/microsoft/durabletask/history/ExecutionResumedEvent.java Adds typed history event model for resume.
client/src/main/java/com/microsoft/durabletask/history/ExecutionCompletedEvent.java Adds typed history event model for terminal completion.
client/src/main/java/com/microsoft/durabletask/history/EventSentEvent.java Adds typed history event model for sending external events.
client/src/main/java/com/microsoft/durabletask/history/EventRaisedEvent.java Adds typed history event model for receiving external events.
client/src/main/java/com/microsoft/durabletask/history/EntityUnlockSentEvent.java Adds typed history event model for entity unlock operations.
client/src/main/java/com/microsoft/durabletask/history/EntityOperationSignaledEvent.java Adds typed history event model for one-way entity signals.
client/src/main/java/com/microsoft/durabletask/history/EntityOperationFailedEvent.java Adds typed history event model for failed entity operations.
client/src/main/java/com/microsoft/durabletask/history/EntityOperationCompletedEvent.java Adds typed history event model for completed entity operations.
client/src/main/java/com/microsoft/durabletask/history/EntityOperationCalledEvent.java Adds typed history event model for two-way entity calls.
client/src/main/java/com/microsoft/durabletask/history/EntityLockRequestedEvent.java Adds typed history event model for entity lock requests.
client/src/main/java/com/microsoft/durabletask/history/EntityLockGrantedEvent.java Adds typed history event model for entity lock grants.
client/src/main/java/com/microsoft/durabletask/history/ContinueAsNewEvent.java Adds typed history event model for continue-as-new.
client/src/main/java/com/microsoft/durabletask/DurableTaskGrpcClient.java Implements new APIs over sidecar gRPC: listInstanceIds + stream history conversion.
client/src/main/java/com/microsoft/durabletask/DurableTaskClient.java Adds new abstract public APIs for instance ID listing and orchestration history retrieval.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread client/src/main/java/com/microsoft/durabletask/history/OrchestrationState.java Outdated
@bachuv bachuv mentioned this pull request Jul 1, 2026
4 tasks

@nytian nytian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments. Thanks for the PR!

Comment thread client/src/main/java/com/microsoft/durabletask/DurableTaskClient.java Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@YunchuWang YunchuWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Strong, well-structured PR. The domain model is clean and immutable, and the protobuf→model converter is exactly correct — I verified all 28 HistoryEvent oneof cases (fields 3–30) and sampled event fields against orchestrator_service.proto, and every mapping and null-guard lines up. Test coverage is good. A handful of things worth resolving before merge, none catastrophic.

Strengths

  • Converter correctness: all 28 event types handled; stringOrNull(hasX, getX) correctly guards the StringValue-wrapped optional fields. One unit test per event type (608 lines), including null-handling and the EVENTTYPE_NOT_SET → IllegalArgumentException branch.
  • Model quality: event classes are final, fully immutable, @Nullable-annotated, well-documented; tags is defensively copied (Collections.unmodifiableMap(new HashMap<>(tags)), null→emptyMap).
  • Back-compat: base-class defaults throw UnsupportedOperationException, so external DurableTaskClient subclasses still compile; the sole impl (DurableTaskGrpcClient) overrides both.
  • ListInstanceIdsQuery validates pageSize ≥ 1 and defensively copies the status list; ListInstanceIdsResult returns an unmodifiable list.

Findings (by severity)

1. getOrchestrationHistory — forward-compat + stream cleanup (Medium). The converter throws on any unrecognized event type. A future server adding a new event type makes the entire history read fail with UnsupportedOperationException. Worse, the blocking gRPC server-streaming iterator is drained in a while(hasNext) loop with no try/finally; if fromProto throws mid-stream the loop exits by exception and the streaming RPC is abandoned without cancellation — a known grpc-java leak pattern. Recommend (a) deciding forward-compat behavior (skip / GenericEvent fallback vs. fail-fast), and (b) cancelling the call on early exit.

2. Whole history materialized in memory (Medium / design). The API returns List<HistoryEvent>, buffering the full history even though the server streams it in chunks — this defeats streaming for very large histories. For the archival/export use case, consider an Iterable/streaming/callback shape, or explicitly document the in-memory materialization.

3. Pagination termination — null vs. empty token (Low–Medium). continuationToken becomes null only when the server omits lastInstanceKey. If the server instead returns a present-but-empty StringValue on the last page, the client hands back "" (non-null); a caller looping on != null re-requests with "", which restarts from the beginning → potential infinite paging loop. Recommend treating empty as end-of-results: hasLastInstanceKey() && !value.isEmpty() ? value : null. No multi-page pagination test exists to catch this.

4. Public constructors on ~30 model types (Low / API design). Because HistoryEventConverter sits in com.microsoft.durabletask while the model lives in ...history, every event constructor must be public — pushing long positional constructors (e.g. ExecutionStartedEvent has 11 params) into the permanent public API even though users only ever read these. Moving the converter into the history package (or a package-private factory there) would let the constructors be package-private, keeping the public surface read-only.

5. Naming (Low). listInstanceIds doesn't convey "terminal instances filtered by completion time." The Javadoc clarifies, but the method name is the real contract and it's still unreleased — consider something like listCompletedInstanceIds.

6. CHANGELOG not updated (Low). New public API but no Unreleased entry; the PR checklist item is unchecked.

7. Unrelated scope (Nit). <h3><h2> Javadoc edits in EntityQueryPageable, TypedEntityMetadata, TypedEntityQueryPageable (doclint heading-order fixes) are harmless but unrelated to this feature.

Questions for the author

  • Intended behavior when a client hits an event type it doesn't recognize (older client / newer server) — fail the whole read, or skip?
  • Is List the intended long-term shape, or will a follow-up expose the underlying stream for large histories?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants