Add riscv64 cross-compile support and update workflows #285
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,6 +22,7 @@ | |
| ['wasmtime-{}-x86_64-macos-c-api.tar.xz', 'macos-x86_64'], | ||
| ['wasmtime-{}-aarch64-linux-c-api.tar.xz', 'linux-aarch64'], | ||
| ['wasmtime-{}-aarch64-macos-c-api.tar.xz', 'macos-aarch64'], | ||
| ['wasmtime-{}-riscv64gc-linux-c-api.tar.xz', 'linux-riscv64'], | ||
| ] | ||
|
|
||
| try: | ||
|
|
@@ -49,7 +50,10 @@ | |
| z.extractall() | ||
| else: | ||
| t = tarfile.open(fileobj=io.BytesIO(contents)) | ||
| if hasattr(tarfile, 'data_filter'): | ||
| t.extractall(filter='data') | ||
| else: | ||
| t.extractall() | ||
|
Comment on lines
+53
to
+56
Author
There was a problem hiding this comment. The Python |
||
|
|
||
| src = filename.replace('.zip', '').replace('.tar.xz', '') | ||
| include_src = src + '/min/include' if args.min else src + '/include' | ||
|
|
||
|
Member
There was a problem hiding this comment. This file unfortuantely hasn't been maintained in awhile. Nowadays can this be updated to use CMake to build the C API? That'll handle headers and such. Also, can all the risc-v specific logic be removed? I don't think any of it should be necessary here (e.g. Zig shouldn't be needed)
Author
There was a problem hiding this comment. Updated to use CMake for building the C API as requested |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,35 +1,53 @@ | ||
| #!/bin/bash | ||
| set -euo pipefail | ||
|
|
||
| wasmtime=$1 | ||
| if [ "$wasmtime" = "" ]; then | ||
| echo "usage: $0 <path-to-wasmtime>" | ||
| exit 1 | ||
| fi | ||
|
|
||
| rm -rf build | ||
|
|
||
| # Build the C API with CMake (invokes cargo under the hood) | ||
| cmake -S "$wasmtime/crates/c-api" -B "$wasmtime/build-cmake" -DCMAKE_BUILD_TYPE=Release | ||
| cmake --build "$wasmtime/build-cmake" | ||
| rm -rf "$wasmtime/build-cmake" | ||
|
|
||
| # Create the expected directory structure with empty.go files | ||
| for d in "include" "include/wasmtime" "include/wasmtime/component" "include/wasmtime/component/types" "include/wasmtime/types" "linux-x86_64" "macos-x86_64" "windows-x86_64" "linux-aarch64" "macos-aarch64" "linux-riscv64"; do | ||
| mkdir -p "build/$d" | ||
| name=$(basename "$d" | tr - _) | ||
| echo "package $name" > "build/$d/empty.go" | ||
| done | ||
|
|
||
| # Determine host platform | ||
| host_os=$(uname -s) | ||
| host_arch=$(uname -m) | ||
| case "$host_os-$host_arch" in | ||
| Linux-x86_64) platform=linux-x86_64; rust_target=x86_64-unknown-linux-gnu ;; | ||
| Linux-aarch64) platform=linux-aarch64; rust_target=aarch64-unknown-linux-gnu ;; | ||
| Linux-riscv64) platform=linux-riscv64; rust_target=riscv64gc-unknown-linux-gnu ;; | ||
| Darwin-x86_64) platform=macos-x86_64; rust_target=x86_64-apple-darwin ;; | ||
| Darwin-arm64) platform=macos-aarch64; rust_target=aarch64-apple-darwin ;; | ||
| *) echo "Unsupported platform: $host_os-$host_arch"; exit 1 ;; | ||
| esac | ||
|
|
||
| # The CMake/cargo build puts artifacts in the wasmtime target directory. | ||
| # Native builds go to target/release/, cross builds to target/$rust_target/release/. | ||
| wasmtime_target_dir="$wasmtime/target/$rust_target/release" | ||
| if [ ! -d "$wasmtime_target_dir" ]; then | ||
| wasmtime_target_dir="$wasmtime/target/release" | ||
| fi | ||
|
Comment on lines
+24
to
41
Member
There was a problem hiding this comment. By using the
Author
There was a problem hiding this comment. CMake’s |
||
|
|
||
| # Find the built library (may be libwasmtime.a or wasmtime.lib) | ||
| built_lib=$(find "$wasmtime_target_dir" -maxdepth 1 -name "libwasmtime*.a" -o -name "wasmtime.lib" | head -1) | ||
| if [ -z "$built_lib" ]; then | ||
| echo "Failed to find built wasmtime library in $wasmtime_target_dir" | ||
| exit 1 | ||
| fi | ||
| ln -s "$built_lib" "build/$platform/libwasmtime.a" | ||
|
|
||
| # Copy headers | ||
| cp "$wasmtime/crates/c-api/include/"*.h build/include/ | ||
| cp -r "$wasmtime/crates/c-api/include/wasmtime" build/include/ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,35 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" | ||
| cd "$SCRIPT_DIR" | ||
|
|
||
| WASMTIME_GO="$(cd "$SCRIPT_DIR/../.." && pwd)" | ||
| DOWNLOAD_SCRIPT="$WASMTIME_GO/ci/download-wasmtime.py" | ||
|
|
||
| # Step 1: Ensure the local wasmtime-go build directory has the v45 full libraries. | ||
| # The local build may contain libraries built from a different wasmtime version | ||
| # (e.g., a newer version built from local Rust source), which causes serialization format mismatches. | ||
| # We temporarily replace them with the official v45 release binaries. | ||
| trap 'rm -rf vendor module.cwasm' EXIT | ||
| ( | ||
| cd "$WASMTIME_GO" | ||
| python3 "$DOWNLOAD_SCRIPT" | ||
| ) | ||
|
|
||
| # Step 2: Create a pre-compiled module using the full Wasmtime library. | ||
| go run create_cwasm.go | ||
|
|
||
| # Step 3: Vendor the module, then adjust the vendored copy so it compiles | ||
| # against the minimal Wasmtime library: | ||
| # a) Remove Go source files that call C functions absent from the min binary. | ||
| # b) Download the min static libraries over the full ones so CGO links the right lib. | ||
| go mod vendor | ||
| ( | ||
| cd vendor/github.com/bytecodealliance/wasmtime-go/v45 | ||
| rm -f wat2wasm.go wasi.go *_feat_*.go *_feats_*.go | ||
| python3 "$DOWNLOAD_SCRIPT" --min | ||
| ) | ||
|
|
||
| # Step 4: Test that the minimal Wasmtime binary can deserialize and run a module. | ||
| go test -count=1 . |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can the test suite be built for another architecture without running it? Or can Go run tests with QEMU emulation? I'd prefer to avoid adding such weighty new CI configuration here if possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, Go can cross-compile the test suite without running it by using
GOOSandGOARCHwithout executing the tests. However, go test with CGO dependencies cannot run under QEMU emulation - you’d need a native riscv64 environment or use user-mode QEMU with proper cross-compilation setup, which would be more complex than the current approach.The current CI builds a binary to verify cross-compilation works, but we could simplify this by just running
go buildon the test files instead of the full test suite. Would that address your concern about avoiding weighty CI configuration?