Cross-Language Serializer Benchmark

Serialization 101 & Benchmark Reports — theory, methodology, and results for senior engineers and data scientists.

Scientific, multi-language benchmark suite for comparing serialization libraries fairly: identical conceptual payloads, dual I/O modes (bytes/stream), nanosecond timing (C# ticks normalized in analysis), and publication-oriented statistics (bootstrap CIs, effect sizes, non-parametric A/B tests).

Who it is for

Audience	Use case
Researchers	Reproducible methods, CIs, effect sizes, configurable outlier/warmup policy
Serializer authors	Compare old vs new version: --compare-a old.csv --compare-b new.csv
System integrators	Custom payloads (schemas/test_data_config.json) and environments; same CSV + analysis pipeline
Engineers	Pick a serializer for a language/runtime with apples-to-apples metrics

Supported languages

Language	Harness	Serializers (target)	Logs
C# (.NET)	c-sharp/	38	logs/csharp/
Python	python/	10	logs/python/
Rust	rust/	12	logs/rust/
C	c/	12	logs/c/
JavaScript (Node)	javascript/	11–12	logs/javascript/

Add more languages via docs/architecture/ADDING_A_LANGUAGE.md.

Configuration

Most parameters live in config/benchmark_config.yaml (modes, statistics, languages, CSV schema, paths). Test-data shape/seed: schemas/test_data_config.json.

Quick start

# One language (smoke)
./rust/scripts/run-benchmarks.sh smoke
./c/scripts/run-benchmarks.sh smoke
./javascript/scripts/run-benchmarks.sh smoke
./python/scripts/run-benchmarks.sh smoke   # may use Docker
./c-sharp/scripts/run-benchmarks.sh smoke

# All languages
./scripts/run-all-benchmarks.sh --mode all-single

# One language only
./scripts/run-all-benchmarks.sh --mode full --lang rust

# Analysis (install analysis package first)
cd analysis && pip install -e .   # or: uv pip install -e .
analyze-benchmarks --generate-summary --generate-plots --output-dir ../reports

# Serializer version A vs B
analyze-benchmarks --compare-a logs/rust/v1.csv --compare-b logs/rust/v2.csv --output-dir reports

Modes: smoke (2 reps) · all-single (10) · full (100) · research (500) — see config.

Shared test data

• Person, Integer, Telemetry, SimpleObject, StringArray, EDI_835, ObjectGraph (cycles; only graph-capable serializers)
• Config: Test data configuration

Statistics (analysis)

See Analysis methodology and statistics: in the master config:

1. Exclude warmup (rep 0)
2. IQR outlier filter
3. Mean / median / std / MAD / CV / percentiles
4. Bootstrap 95% CI on the mean
5. Cliff's δ + Hedges' g vs fastest in group
6. Optional Mann–Whitney U + Holm for A/B versions

Architecture & extensibility

• Architecture
• Adding a language
• Harness contract: emit logs/<lang>/benchmark-log.csv with Language=<id>, times in nanoseconds (except legacy C# ticks)

Documentation site

Language-specific serializer lists:

• C# · Python
• Rust · C · JavaScript

License

MIT