Scheduler scales poorly

[mkroening]

Prepare

Apply the following patch to hermit-rs:

From 2359c5885a057ea40620409e70a7e23ba897a35f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Martin=20Kr=C3=B6ning?=
 <martin.kroening@eonerc.rwth-aachen.de>
Date: Tue, 16 Jun 2026 08:18:13 +0000
Subject: [PATCH] feat: focus on laplace

---
 examples/demo/src/laplace.rs |  4 +---
 examples/demo/src/main.rs    | 16 +++++++---------
 2 files changed, 8 insertions(+), 12 deletions(-)

diff --git a/examples/demo/src/laplace.rs b/examples/demo/src/laplace.rs
index 7112320..3e48db4 100644
--- a/examples/demo/src/laplace.rs
+++ b/examples/demo/src/laplace.rs
@@ -7,7 +7,7 @@ use std::{mem, vec};
 
 use rayon::prelude::*;
 
-const SIZE: usize = if cfg!(debug_assertions) { 16 } else { 64 };
+const SIZE: usize = 0x1000;
 const ITERATIONS: usize = 1000;
 
 pub fn laplace() {
@@ -20,8 +20,6 @@ pub fn laplace() {
 	let residual = compute(&mut matrix, SIZE, SIZE, ITERATIONS);
 	let elapsed = now.elapsed();
 	eprintln!("{ITERATIONS} iterations: {elapsed:?} (residual: {residual})");
-
-	assert!(residual < 0.001);
 }
 
 fn matrix_setup(size_x: usize, size_y: usize) -> vec::Vec<f64> {
diff --git a/examples/demo/src/main.rs b/examples/demo/src/main.rs
index 4239096..51e290e 100644
--- a/examples/demo/src/main.rs
+++ b/examples/demo/src/main.rs
@@ -11,16 +11,14 @@ mod pi;
 mod thread;
 
 fn main() -> io::Result<()> {
-	hello();
-	print_env();
-	arithmetic();
-	thread::sleep();
-	thread::spawn()?;
-	fs::fs()?;
-	pi::pi();
-	matmul::matmul();
+	let rayon_num_threads = env::var("RAYON_NUM_THREADS").unwrap();
+	println!("RAYON_NUM_THREADS = {rayon_num_threads}");
+
+	laplace::laplace();
+	laplace::laplace();
+	laplace::laplace();
+	laplace::laplace();
 	laplace::laplace();
-	mandelbrot::mandelbrot();
 	Ok(())
 }
 
-- 
2.53.0

Run on Linux (Host)

RAYON_NUM_THREADS=16 cargo run --package rusty_demo --release

Run on Hermit

cargo build --package rusty_demo --release --target x86_64-unknown-hermit -Zbuild-std=std,panic_abort

sudo qemu-system-x86_64 \
    -accel kvm \
    -cpu host \
    -display none \
    -serial stdio \
    -kernel kernel/hermit-loader-x86_64 \
    -initrd target/x86_64-unknown-hermit/release/rusty_demo \
    -smp 16 \
    -m 1G \
    -append "env=RAYON_NUM_THREADS=16"

Results

Run on an AMD EPYC 9354P 32-Core Processor:

OS	Available Parallelism	RAYON_NUM_THREADS	Time
Linux (Host)	32	1	24s
Linux (Host)	32	2	13.5s
Linux (Host)	32	4	9.0s
Linux (Host)	32	8	7.6s
Linux (Host)	32	16	6.2s
Hermit (VM)	16	1	11.5s
Hermit (VM)	16	2	5.8s
Hermit (VM)	16	4	5.2s
Hermit (VM)	16	8	6.9s
Hermit (VM)	16	16	12.5s

Issue

The results are surprising, to say the least. Hermit being faster with a single thread is nice and suspicious. Still, Hermit should not get slower when running on more cores. Not sure if this is a futex issue, scheduler bug, or a symptom of the scheduler not doing work stealing yet.

[zyuiop]

When working on #2468, I added prints in futex related functions. At a glance, it looked like every thread was basically waking up, waking up another thread, and then going back to sleep. This is also why the race condition could bring Laplace to a full halt: any missed wakeup stopped this cycle. But this also indicates that there is a situation where progress is stuck on a single thread.