Skip to content

Fix #120: Async multi-threaded benchmark returns correct aggregated metrics#148

Open
mcurrier2 wants to merge 2 commits into
mainfrom
fix/120-async-multi-threaded-aggregation
Open

Fix #120: Async multi-threaded benchmark returns correct aggregated metrics#148
mcurrier2 wants to merge 2 commits into
mainfrom
fix/120-async-multi-threaded-aggregation

Conversation

@mcurrier2

Copy link
Copy Markdown
Collaborator

Summary

  • Fixes incorrect metrics when using --concurrency > 1 in async mode
  • Multi-threaded one-way and round-trip functions now return the aggregated PerformanceMetrics directly from aggregate_worker_metrics(), instead of attempting to repopulate a shared MetricsCollector

Problem

When --concurrency > 1, the async benchmark runner had three compounding bugs:

  1. Latency discarded: The repopulation loop called record_message(size, None) — the None latency meant all percentiles/min/max/mean from the correct aggregation were thrown away
  2. Throughput 2x inflated: Two separate loops (one for latency samples, one for throughput messages) both called record_message, doubling the throughput counter
  3. messages_per_worker unused: Computed and logged but never applied — each worker ran the full msg_count

Fix

Changed run_multi_threaded_one_way and run_multi_threaded_round_trip to:

  • Return Result<PerformanceMetrics> instead of Result<()>
  • Return the output of aggregate_worker_metrics() directly
  • Remove the broken record_message repopulation loops
  • Remove the unused metrics_collector parameter

The parent dispatch functions (run_one_way_test, run_round_trip_test) now use the returned aggregated metrics directly for the concurrency > 1 path.

Scope

This bug only affects async mode with --concurrency > 1 (not the default). Single-threaded paths and all blocking mode paths are unaffected.

Test plan

  • cargo fmt — clean
  • cargo clippy --all-targets -- -D warnings — zero warnings
  • cargo test — all tests pass
  • cargo build --release — clean build
  • Pre-commit hooks pass

Closes #120

Made with Cursor

…etrics

The run_multi_threaded_one_way and run_multi_threaded_round_trip
functions had three compounding bugs when --concurrency > 1:

1. Aggregated latency data was discarded: the code attempted to
   "repopulate" the main MetricsCollector by calling record_message
   with None for latency, which threw away all percentiles, min, max,
   and mean from the correct aggregation.

2. Throughput was double-counted: two separate loops (one for
   "latency samples" and one for "throughput messages") both called
   record_message, inflating the throughput counter by ~2x.

3. messages_per_worker was computed but never used: each worker ran
   the full msg_count rather than a divided portion. This is now
   documented as intentional behavior (total = N * msg_count).

Fix: Change both multi-threaded functions to return the aggregated
PerformanceMetrics directly from aggregate_worker_metrics(), bypassing
the broken repopulation logic entirely. The parent dispatch functions
now return the aggregated result directly for the concurrency > 1
path, while single-threaded paths continue using the MetricsCollector
as before.

- Remove unused metrics_collector parameter from multi-threaded fns
- Remove broken record_message repopulation loops
- Remove unused messages_per_worker variable
- Clean up stale doc comments from old implementation
- All tests pass, clippy clean, cargo fmt applied

AI-assisted-by: Claude Opus 4 (claude-sonnet-4-20250514)
Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@mcurrier2, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 35 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 2830c920-f030-45fe-a593-07d084cb1412

📥 Commits

Reviewing files that changed from the base of the PR and between d586320 and 7307659.

📒 Files selected for processing (2)
  • src/benchmark.rs
  • src/ipc/shared_memory_blocking.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

- Add #[cfg(unix)] to test_backpressure_with_small_buffer,
  test_payload_integrity_under_backpressure,
  test_ring_buffer_wrap_around_under_backpressure, and
  test_shutdown_detected_during_blocked_write
- These tests rely on pthread_mutex/condvar-based blocking
  synchronization which only works on Unix platforms
- Fixes Windows CI failure where the tight buffer causes a race
  condition due to lack of proper inter-process sync primitives

AI-assisted-by: Claude Opus 4.6
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Async multi-threaded benchmark aggregation produces incorrect metrics

1 participant