Skip to content

feat: Add opt-in scenario-level parallelism (ParallelizationScope.Scenario)#1093

Open
sourrinn wants to merge 1 commit into
reqnroll:mainfrom
sourrinn:feature/scenario-level-parallelism
Open

feat: Add opt-in scenario-level parallelism (ParallelizationScope.Scenario)#1093
sourrinn wants to merge 1 commit into
reqnroll:mainfrom
sourrinn:feature/scenario-level-parallelism

Conversation

@sourrinn

@sourrinn sourrinn commented Jul 4, 2026

Copy link
Copy Markdown

Summary

Adds scenario-level parallel execution as a non-breaking, opt-in configuration. When "parallelizationScope": "Scenario" is set in reqnroll.json, scenarios within the same feature run concurrently on independent threads.

Motivation

Closes #114. Feature-level parallelism leaves threads idle when one feature has many more scenarios than others. Scenario-level parallelism achieves linear speedup proportional to total scenario count, not feature count.

Design

Runtime

  • Configuration: New ParallelizationScope enum (Feature default, Scenario opt-in) with full JSON schema support
  • FeatureLifecycleManager: Reference-counted BeforeFeature/AfterFeature hooks — fire exactly once per feature using Interlocked.CompareExchange + SemaphoreSlim for async-safe mutual exclusion
  • ReadOnlyFeatureContextProxy: Copy-on-write isolation — scenario writes go to a thread-local ConcurrentDictionary, shared context remains read-only after BeforeFeature completes
  • TestExecutionEngine: Branches on ParallelizationScope in OnFeatureStartAsync/OnFeatureEndAsync; delegates lifecycle to FeatureLifecycleManager in scenario mode
  • TestRunnerManager: Work-stealing pool (no feature affinity) in scenario mode; EndFeatureForAvailableTestRunnersAsync is a no-op since lifecycle is per-scenario

Generator

  • In scenario mode, class-level fixture setup/teardown bodies are empty
  • Each test's cleanup calls OnFeatureEndAsync before releasing the runner
  • ParallelizationScope stored in CustomData for providers to inspect

Framework Plugins

Framework Scenario Mode Behavior
NUnit Adds [Parallelizable(ParallelScope.Self)], skips [OneTimeSetUp]/[OneTimeTearDown]
xUnit2 Skips IClassFixture/FixtureData pattern
xUnit3 Skips IClassFixture/IAsyncLifetime fixture
MSTest Skips [ClassInitialize]/[ClassCleanup]
TUnit Skips [Before(Class)]/[After(Class)] (already supports method-level parallelism)

Configuration

{
  "runtime": {
    "parallelizationScope": "Scenario"
  }
}

Default remains Feature — zero behavior change for existing users.

Thread-Safety Guarantees

  1. BeforeFeature fires exactly onceInterlocked.CompareExchange CAS
  2. AfterFeature fires only when all scenarios complete — atomic ref-count reaching zero
  3. No FeatureContext corruptionReadOnlyFeatureContextProxy with ConcurrentDictionary overlay
  4. No deadlocksSemaphoreSlim (async-compatible), no nested lock acquisition
  5. Error propagationExceptionDispatchInfo relays BeforeFeature failures to all concurrent scenarios
  6. Static accessors protected — existing DisableSingletonInstance() throws on FeatureContext.Current in parallel mode

Testing

  • 2458 existing tests pass (RuntimeTests: 2136, GeneratorTests: 214, PluginTests: 108) — 0 failures
  • 9 new stress tests covering:
    • 10-way concurrent initialization (exactly-once validation)
    • Error propagation from failed BeforeFeature to all callers
    • Ref-count finalization (AfterFeature only at zero)
    • Idempotent finalization under concurrent calls
    • Sequential acquire/release lifecycle
    • 20 concurrent scenarios on same feature
    • 10 features x 10 scenarios (100 total) in parallel
    • 50-thread rapid acquire/release
    • Multiple independent feature lifecycles
  • Repeated 10x with zero flaky failures

Breaking Changes

None. The default ParallelizationScope.Feature preserves all existing behavior. The constructor parameter is optional with a default value.

@sourrinn sourrinn marked this pull request as ready for review July 4, 2026 18:15
@sourrinn sourrinn force-pushed the feature/scenario-level-parallelism branch 2 times, most recently from a3072c0 to a219982 Compare July 4, 2026 19:26
…nario)

Introduces scenario-level parallel execution as a non-breaking, opt-in feature.
When enabled via reqnroll.json, scenarios within the same feature can run
concurrently across threads.

Key changes:
- New ParallelizationScope enum (Feature/Scenario) with JSON config support
- FeatureLifecycleManager: ref-counted BeforeFeature/AfterFeature hook execution
- ReadOnlyFeatureContextProxy: copy-on-write context isolation per scenario
- TestExecutionEngine: branches on parallelization scope
- TestRunnerManager: work-stealing pool without feature affinity in scenario mode
- Generator: skips class-level fixture setup/teardown in scenario mode
- All 5 framework providers adapted (NUnit, xUnit2, xUnit3, MSTest, TUnit)
- 9 stress tests validating thread-safety under concurrent load

Closes reqnroll#114
@sourrinn sourrinn force-pushed the feature/scenario-level-parallelism branch from a219982 to 4acde9a Compare July 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallel scenario tests fail

1 participant