Initial drop of openssl.

README.md

@@ -0,0 +1,272 @@
+# OpenSSL Benchmark Wrapper
+
+## Description
+
+This wrapper facilitates the automated execution of the OpenSSL speed benchmark. OpenSSL speed is a built-in benchmarking tool that measures the throughput and operations-per-second of various cryptographic algorithms, including symmetric ciphers, hash functions, and asymmetric key operations.
+
+The wrapper provides:
+- Automated OpenSSL speed benchmark execution across multiple cryptographic algorithms.
+- Configurable thread counts with automatic CPU-aware interval generation.
+- Configurable run duration per algorithm.
+- Multi-iteration testing with averaged results.
+- Result collection, processing, and verification.
+- CSV and JSON output formats.
+- System configuration metadata capture.
+- Integration with test_tools framework.
+- Optional Performance Co-Pilot (PCP) integration.
+
+## Command-Line Options
+
+```
+OpenSSL Options:
+  --run_time <value>: Duration in seconds for each algorithm test. Defaults to 60.
+  --threads <value>: Comma-separated list of thread counts to test with.
+      If not set, automatically generates intervals based on CPU count (up to 8 intervals).
+  --tests <value>: Space-separated list of OpenSSL algorithms to benchmark.
+      Defaults to: chacha20-poly1305 sha256 sha512 chacha20 rsa4096 aes-128-gcm aes-256-gcm
+
+General test_tools options:
+  --home_parent <value>: Parent home directory. If not set, defaults to current working directory.
+  --host_config <value>: Host configuration name, defaults to current hostname.
+  --iterations <value>: Number of times to run the test, defaults to 1.
+  --run_user: User that is actually running the test on the test system. Defaults to current user.
+  --sys_type: Type of system working with (aws, azure, hostname). Defaults to hostname.
+  --sysname: Name of the system running, used in determining config files. Defaults to hostname.
+  --tuned_setting: Used in naming the results directory. For RHEL, defaults to current active tuned profile.
+      For non-RHEL systems, defaults to 'none'.
+  --use_pcp: Enable Performance Co-Pilot monitoring during test execution.
+  --tools_git <value>: Git repo to retrieve the required tools from.
+      Default: https://github.com/redhat-performance/test_tools-wrappers
+  --usage: Display this usage message.
+```
+
+## What the Script Does
+
+The `openssl.sh` script performs the following workflow:
+
+1. **Environment Setup**:
+   - Clones the test_tools-wrappers repository if not present (default: ~/test_tools).
+   - Sources error codes and general setup utilities.
+   - Gathers system hardware information.
+
+2. **Package Installation**:
+   - Installs required dependencies via package_tool.
+   - Dependencies are defined in openssl.json for different OS variants.
+
+3. **Thread Configuration**:
+   - If `--threads` is not specified, detects CPU count via `nproc`.
+   - Generates thread intervals automatically using the test_tools `generate_intervals` utility.
+   - Caps at 8 intervals for systems with 8 or more CPUs.
+
+4. **Test Execution**:
+   - Iterates through each configured thread count.
+   - For each thread count, runs `openssl speed` against every algorithm in the test list.
+   - Uses `-evp` flag for all algorithms except rsa4096 (which uses direct mode).
+   - Runs each algorithm for the configured duration (default: 60 seconds per algorithm).
+   - Records start and end timestamps for each algorithm run.
+   - Skips thread/iteration combinations that already have output files (resume support).
+   - Executes for specified number of iterations (`--iterations`).
+
+5. **Data Collection**:
+   - Captures raw output from each iteration in `results_<iter>_iter_<threads>_threads.data` files.
+   - Extracts performance metrics:
+     - For symmetric ciphers and hashes: throughput in operations per second.
+     - For rsa4096: sign, verify, encrypt, and decrypt operations per second.
+   - Optionally records PCP performance data during execution.
+
+6. **Result Processing**:
+   - Averages results across all iterations for each algorithm/thread combination.
+   - Generates CSV file with configuration metadata and performance data.
+   - Creates JSON output for verification.
+   - Validates results against Pydantic schema (results_schema.py).
+
+7. **Verification**:
+   - Validates results against Pydantic schema ensuring:
+     - All test names match expected enum values.
+     - Thread count is a positive integer.
+     - Operations per second is a positive integer.
+     - Timestamps are valid datetime objects.
+   - Uses csv_to_json and verify_results from test_tools.
+
+8. **Output**:
+   - Saves all raw output files, processed CSV/JSON, and system metadata.
+   - Optionally saves PCP performance data.
+   - Archives results to configured storage location via save_results.
+
+## Dependencies
+
+**General packages required**: openssl
+
+To run:
+```bash
+git clone <repo-url>
+cd openssl-wrapper/openssl
+./openssl.sh
+```
+
+The script will automatically detect your CPU configuration and run all default algorithms.
+
+## The OpenSSL Speed Benchmark
+
+OpenSSL speed measures the cryptographic throughput of the system's OpenSSL installation. It tests how quickly the CPU can perform various cryptographic operations, which is critical for TLS/SSL performance, disk encryption, and secure communications.
+
+### Algorithms Tested
+
+1. **chacha20-poly1305**: Authenticated encryption cipher combining ChaCha20 stream cipher with Poly1305 MAC. Widely used in TLS 1.3, especially on platforms without AES hardware acceleration.
+
+2. **sha256**: SHA-2 family hash function producing 256-bit digests. Used extensively in TLS certificates, digital signatures, and data integrity verification.
+
+3. **sha512**: SHA-2 family hash function producing 512-bit digests. Often faster than SHA-256 on 64-bit platforms due to native 64-bit operations.
+
+4. **chacha20**: Stream cipher designed as a high-speed alternative to AES. Performs well on platforms without dedicated AES instructions.
+
+5. **rsa4096**: RSA asymmetric key operations with 4096-bit keys. Tests four distinct operations:
+   - **sign**: Private key signing operations per second.
+   - **verify**: Public key verification operations per second.
+   - **encrypt**: Public key encryption operations per second.
+   - **decrypt**: Private key decryption operations per second.
+
+6. **aes-128-gcm**: AES in Galois/Counter Mode with 128-bit keys. Authenticated encryption widely used in TLS, benefits heavily from AES-NI hardware instructions.
+
+7. **aes-256-gcm**: AES in Galois/Counter Mode with 256-bit keys. Higher security level than AES-128, with slightly lower throughput.
+
+### Performance Metrics
+
+Each algorithm reports:
+
+1. **ops_per_sec**: Operations (or bytes) processed per second. For symmetric ciphers and hashes, this is throughput in bytes/sec. For RSA, this is operations/sec.
+
+2. **threads**: The number of parallel threads used for the test.
+
+Results are reported for each algorithm at each thread count, allowing analysis of both single-threaded performance and multi-threaded scaling.
+
+## Output Files
+
+The results directory contains:
+
+- **results_openssl.csv**: CSV file with all algorithm performance metrics, including columns: Test, threads, ops_per_sec, Start_Date, End_Date
+- **results_openssl.json**: JSON file with validated results data
+- **results_\<iter\>_iter_\<threads\>_threads.data**: Raw output files from each test iteration/thread combination
+- **test_results_report**: Detailed test execution report
+- **meta_data\*.yml**: System metadata (CPU info, memory, kernel version)
+- **PCP data** (if --use_pcp option used): Performance Co-Pilot monitoring data
+
+## Examples
+
+### Basic run with defaults
+```bash
+./openssl.sh
+```
+This runs with:
+- All default algorithms (chacha20-poly1305, sha256, sha512, chacha20, rsa4096, aes-128-gcm, aes-256-gcm)
+- 60-second run time per algorithm
+- Automatically determined thread counts based on CPU count
+- 1 iteration
+
+### Run with custom thread counts
+```bash
+./openssl.sh --threads 1,2,4,8
+```
+Tests each algorithm at 1, 2, 4, and 8 threads specifically.
+
+### Run specific algorithms only
+```bash
+./openssl.sh --tests "aes-128-gcm aes-256-gcm"
+```
+Benchmarks only AES-GCM algorithms, skipping all others.
+
+### Run with longer duration per algorithm
+```bash
+./openssl.sh --run_time 120
+```
+Runs each algorithm for 120 seconds instead of the default 60 seconds, producing more stable results.
+
+### Run multiple iterations
+```bash
+./openssl.sh --iterations 3
+```
+Runs the complete test suite 3 times and averages the results for consistency validation.
+
+### Run with PCP monitoring
+```bash
+./openssl.sh --use_pcp
+```
+Collects Performance Co-Pilot data during the run for detailed performance analysis.
+
+### Combination example
+```bash
+./openssl.sh --iterations 3 --run_time 120 --threads 1,4,8,16 --tests "sha256 aes-256-gcm" --use_pcp
+```
+Runs SHA-256 and AES-256-GCM for 120 seconds each at 1, 4, 8, and 16 threads, repeats 3 times, and collects PCP data.
+
+## How Thread Scaling Works
+
+When `--threads` is not specified, the wrapper automatically generates thread counts:
+
+1. Detects the number of CPUs via `nproc`.
+2. Determines the number of intervals (capped at 8).
+3. Uses `generate_intervals` to create evenly spaced thread counts from 1 up to the total CPU count.
+
+For example, on a 32-core system, the generated thread counts might be: 1 4 8 12 16 20 24 28 32.
+
+This allows measuring how each algorithm scales with increasing parallelism on the specific hardware.
+
+## How Result Averaging Works
+
+When running multiple iterations (--iterations > 1):
+
+1. Each iteration produces a complete set of results for all algorithms at all thread counts.
+2. Raw results are saved in separate `results_<iter>_iter_<threads>_threads.data` files.
+3. The wrapper extracts metrics from each iteration and sums them.
+4. Final results are the arithmetic mean across all iterations.
+
+This approach:
+- Reduces impact of transient system effects.
+- Provides more reliable performance measurements.
+- Helps identify result variance across runs.
+
+## Return Codes
+
+The script uses standardized error codes from test_tools error_codes:
+- **0 (E_SUCCESS)**: Success
+- **101**: Git clone failure (test_tools)
+- **E_GENERAL**: General execution errors
+- **E_USAGE**: Invalid usage/arguments
+
+## Notes
+
+### Supported Platforms
+- **Linux**: x86_64 and aarch64 architectures
+- **OS Support**: RHEL (dependencies defined in openssl.json)
+
+### EVP vs Direct Mode
+- Most algorithms are tested using the `-evp` flag, which routes through the OpenSSL EVP (Envelope) API. This is the recommended interface and engages hardware acceleration (e.g., AES-NI).
+- RSA4096 is tested in direct mode (without `-evp`) as it uses the asymmetric key API, which reports sign/verify/encrypt/decrypt metrics separately.
+
+### Performance Considerations
+- **AES-NI**: Systems with AES-NI hardware instructions will show significantly higher AES-GCM throughput. Check with `grep aes /proc/cpuinfo`.
+- **ChaCha20**: Performs well on platforms without AES hardware acceleration, making it a good comparison point.
+- **RSA4096**: Asymmetric operations are orders of magnitude slower than symmetric operations. Sign and decrypt (private key operations) are much slower than verify and encrypt (public key operations).
+- Run multiple iterations (`--iterations 3` or higher) for reliable results.
+- Ensure system is idle during testing for best consistency.
+
+### Performance Tips
+- Run multiple iterations (`--iterations 3+`) to verify consistency.
+- Ensure system is idle (no other workloads) for best results.
+- Disable CPU frequency scaling (use performance governor) for reproducible results.
+- Consider the active tuned profile on RHEL systems.
+- Use longer run times (`--run_time 120`) for more stable per-algorithm measurements.
+- PCP monitoring (`--use_pcp`) adds minimal overhead but provides detailed metrics.
+
+### Troubleshooting
+- If openssl is not found, ensure the openssl package is installed.
+- If results seem inconsistent, run more iterations and check system load during testing.
+- Use `--use_pcp` to collect detailed performance counters for analysis.
+- Check `results_*_iter_*_threads.data` files for detailed error messages from `openssl speed`.
+- Verify hardware acceleration support: `openssl engine` and `grep -E 'aes|avx' /proc/cpuinfo`.
+
+## References
+
+- OpenSSL Documentation: https://www.openssl.org/docs/
+- OpenSSL Speed Manual: https://www.openssl.org/docs/man3.0/man1/openssl-speed.html
+- test_tools Framework: https://github.com/redhat-performance/test_tools-wrappers

openssl.json

@@ -0,0 +1,7 @@
+{
+  "dependencies": {
+    "rhel": [
+      "openssl"
+    ]
+  }
+}

openssl/openmetrics_openssl_reset.txt

@@ -0,0 +1,18 @@
+iteration 0
+running 0
+numthreads 0
+runtime 0
+throughput 0
+latency 0
+runlog 0
+IterationsPerSec 0
+aes_128_gcm NaN
+aes_256_gcm NaN
+chacha20 NaN
+chacha20_poly1305 NaN
+rsa4096_decr NaN
+rsa4096_encr NaN
+rsa4096_sign NaN
+rsa4096_verify NaN
+sha256 NaN
+sha512 NaN