runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390
runtime,internal/poll,loader: wasip2 pollable poll-integration + TCP I/O#5390achille-roussel wants to merge 1 commit into
Conversation
3baa42c to
39b8954
Compare
|
@achille-roussel can you please rebase this PR against the latest Also, can you please remove Claude as your co-author? Thank you! |
Mirrors PR tinygo-org#5386's wasip1 work for wasip2. The cooperative scheduler's idle path now calls wasi:io/poll.Poll over a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a single monotonic-clock subscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines. Plumbing components: - runtime/netpoll_wasip2.go: pollable-keyed pollDesc registry; pollIO builds one combined wasi:io/poll.Poll call (clock pollable + active pollables). Linkname-exposed runtime_netpoll_addpollable_wasip2 / done / pdfired / wake for internal/poll and future net. - runtime/scheduler_idle_wasip2.go + scheduler_idle_wasip2_none.go: cooperative-variant sleepTicks / waitForEvents that route through pollIO; non-coop fallback uses monotonicclock.Block. Mirrors the wasip1 structure introduced in 7000e7b. - runtime/runtime_wasip2.go: sleepTicks moved out to the scheduler_idle_wasip2*.go files. - runtime/wait_other.go: build tag tightened to exclude wasip2. internal/poll surface: - internal/poll/fd_wasip2.go: WasipNFD wraps a (TcpSocket, InputStream, OutputStream) triple. DialTCPWasip2, ListenTCPWasip2, Accept, Read, Write, Close, SetDeadline*. Each blocking op tries the wasi call, on would-block subscribes, parks, retries — same pattern as the wasip1 internal/poll.FD but pollable-keyed. Linkname-friendly Wasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline} wrappers for test / future net callers. - internal/poll/errors_wasip.go: ErrFileClosing / ErrNetClosing / ErrDeadlineExceeded / ErrNoDeadline extracted from fd_wasip1.go to a wasip1||wasip2 shared file. Loader change: - loader/goroot.go: listGorootMergeLinks now filters TinyGo files by //go:build constraints (via go/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides in directories like src/net/ for future net.wasip2 work without disturbing wasip1. End-to-end verification: $ wasmtime run -Sinherit-network -Stcp ./tcpecho_wasip2.wasm & listening on 127.0.0.1:9999 tick 1 tick 2 tick 3 $ echo hello | nc 127.0.0.1 9999 hello # echoed by the wasm $ # two concurrent clients echo cleanly while ticker keeps ticking The test program (not shipped) uses //go:linkname to drive the internal/poll TCP helpers directly, since TinyGo doesn't yet have a net.Listen / net.Dial path on wasip2 (upstream Go's net doesn't build for wasip2 due to cgo_linux.go reaching for Linux headers). The src/net/ wasip2 wrappers are out of scope for this PR and tracked as follow-up — once they land, callers will use net.Listen / Dial directly and the linkname wrappers can drop. Wasip1 regression sweep: tcpecho.wasm still passes; time.Sleep / parkfile / parksynth unchanged.
39b8954 to
d9e0859
Compare
|
Done. I'm exploring how to add a few tests to verify the new behavior as well. |
b14e4c3 to
d9e0859
Compare
| // buildContext is used to evaluate //go:build constraints on TinyGo | ||
| // source files. A TinyGo file that doesn't match the current target | ||
| // (e.g. a *_wasip2.go file in a wasip1 build) must not be treated as | ||
| // "TinyGo owns this directory" — otherwise wasip1 builds would lose | ||
| // upstream Go files at the same level. | ||
| bctx := build.Default | ||
| bctx.GOOS = config.GOOS() | ||
| bctx.GOARCH = config.GOARCH() | ||
| bctx.BuildTags = config.BuildTags() | ||
| bctx.Compiler = "gc" |
There was a problem hiding this comment.
Why is this needed? (And why do normal build tags not work?)
There was a problem hiding this comment.
The loader builds a merged GOROOT before the compiler runs: symlinks files from upstream Go and TinyGo into a temp dir. At that point Go's normal build-tag filtering hasn't happened yet.
Without the change: TinyGo's old rule was "if any TinyGo file exists in this dir, drop all upstream files at the same level." So a foo_wasip2.go sitting in src/runtime/ would make the wasip1 build also drop upstream files from that dir, even though foo_wasip2.go is invisible to a wasip1 compile.
The MatchFile call filters TinyGo files by //go:build constraints first, so only files actually compiled for the current target count as "TinyGo owns this dir."
Let me know if I overlooked something, happy to change the approach as needed.
Stacked on #5386. Mirrors the wasip1 work for wasip2. The cooperative scheduler's idle path now calls
wasi:io/poll.Pollover a combined list of (clock pollable, registered pollables) instead of blocking the wasm module on a singlemonotonic-clocksubscription, so goroutines doing TCP I/O can park while the scheduler runs other goroutines.End to end:
Two concurrent
ncclients both echo while the ticker keeps progressing — the cooperative scheduler keeps running goroutines parked onsock_recvvia the new pollable registry.How it works
Wasip2's polling primitive is
wasi:io/poll.Poll(list<own<pollable>>) -> list<u32>taking a list of pollable resource handles. Each blocking wasi operation has asubscribe()that yields a pollable.The wasip2 path mirrors the wasip1 PR's structure file-for-file:
runtime/netpoll_wasip1.go(FD-keyedpoll_oneoffregistry)runtime/netpoll_wasip2.go(pollable-keyedwasi:io/poll.Pollregistry)runtime/scheduler_idle_wasip1.go(cooperativesleepTicksroutes viapollIO)runtime/scheduler_idle_wasip2.go(cooperativesleepTicksroutes viapollIO)runtime/scheduler_idle_wasip1_none.go(non-coop fallback)runtime/scheduler_idle_wasip2_none.go(non-coop fallback)internal/poll/fd_wasip1.go(FD type)internal/poll/fd_wasip2.go(WasipNFD over (TcpSocket, InputStream, OutputStream))syscall/syscall_libc_wasip1.gopark-on-EAGAINwould-blockdirectly intointernal/poll)What's added
src/runtime/netpoll_wasip2.go— pollable-keyedpollDescregistry,pollIO(timeoutNs)building one combinedwasi:io/poll.Pollcall (clock + active pollables), linkname-exposed wake helpers (runtime_netpoll_addpollable_wasip2/done/pdfired/wake). Three timing cases handled like wasip1'spollIO.src/runtime/scheduler_idle_wasip2.go+scheduler_idle_wasip2_none.go— cooperative-variantsleepTicks/waitForEventsthat route throughpollIOwhen pollables are registered; non-coop fallback usesmonotonicclock.Block. Matches the wasip1 file split.src/runtime/runtime_wasip2.go—sleepTicksmoved out (now per-config inscheduler_idle_wasip2*.go).src/runtime/wait_other.go— build tag tightened to exclude wasip2.src/internal/poll/fd_wasip2.go—WasipNFDwraps(TcpSocket, InputStream, OutputStream).DialTCPWasip2,ListenTCPWasip2,Accept,Read,Write,Close,SetDeadline*. Each blocking op tries the wasi call, onwould-blocksubscribes, parks viaruntime_netpoll_addpollable_wasip2 + task.Pause, on resume drops pollable + retries. Deadline-aware variants useparkUntil + time.AfterFunc + runtime_netpoll_wake_wasip2. Linkname-friendlyWasip2TCP{Listen,Dial,Accept,Read,Write,Close,SetDeadline}wrappers for test / future net callers.src/internal/poll/errors_wasip.go— error sentinels (ErrFileClosing,ErrNetClosing,ErrDeadlineExceeded,ErrNoDeadline) extracted fromfd_wasip1.goto awasip1 || wasip2shared file.src/os/poll_link_wasip2.go— pullsinternal/pollinto the wasip2 build (no-op blank import with justifying comment for the lint check).loader/goroot.go—listGorootMergeLinksnow filters TinyGo files by//go:buildconstraints (viago/build.Context.MatchFile) before deciding "TinyGo owns this directory". Files that don't match the current target no longer cause upstream Go files at the same level to be dropped. Unblocks per-target overrides for follow-up work without disturbing wasip1.Non-goals (deferred)
net.Listen("tcp", ...)/net.Dial("tcp", ...)on wasip2 via the standardnetpackage. Upstream Go'snetdoesn't currently build for wasip2 because itscgo_linux.goreaches fornetdb.heven though TinyGo doesn't enable cgo. Bringing up upstream net on wasip2 (either by filtering itscgofiles from the merge or providing a TinyGo-native net override) is its own piece of work — the linkname-friendly TCP helpers in this PR are the foundation. The synthetic test program calls them directly.PacketConnand DNS resolution. Same shape will work (wasi:sockets/udp,wasi:sockets/ip-name-lookup) but not in scope here.Wasip1 regression sweep:
tcpecho.wasmfrom #5386 still passes (echo hi | nc 127.0.0.1 9998round-trip + concurrent clients);time.Sleep/parkfile/parksynthunchanged.