Add procedure HTTP request API for WASM modules and the Rust module bindings library#3684
Merged
coolreader18 merged 24 commits intomasterfrom Nov 20, 2025
Merged
Add procedure HTTP request API for WASM modules and the Rust module bindings library#3684coolreader18 merged 24 commits intomasterfrom
coolreader18 merged 24 commits intomasterfrom
Conversation
With this PR, procedures (at least, those defined in Rust modules) can perform HTTP requests! This is performed through a new field on the `ProcedureContext`, `http: HttpClient`, which has a method `send` for sending an `http::Request`, as well as a convenience wrapper `get`. Internally, these methods hit the `procedure_http_request` ABI call / host function, which uses reqwest to perform an HTTP request. The request is run with a user-configurable timeout which defaults and is clamped to 500 ms. Rather than exposing the HTTP stream to modules, we download the entire response body immediately, within the same timeout. I've added an example usage of `get` to `module-test` which performs a request against `localhost:3000` to read its own schema/moduledef. Not included in this commit, but intended to be within this PR, are: - [ ] A test using the `sdk-test` framework. - [ ] Expanded documentation. Left as TODOs are: - Metrics for recording request and response size. - Improving performance by stashing a long-lived `reqwest::Client` someplace. Currently we build a new `Client` for each request. - Improving performance (possibly) by passing the request-future to the global tokio executor rather than running it on the single-threaded database executor.
coolreader18
requested changes
Nov 18, 2025
Despite it appearing locally
Centril
reviewed
Nov 19, 2025
Collaborator
|
Phoebe and I were discussing Furthermore, if we did want to change HttpRequest in the future, it'd probably just be best to introduce a new abi function altogther, since that's our existing mechanism for versioning. Under the enum scheme, if we added a new variant and someone built a new module and ran it against an old spacetimedb server, it would be a runtime error only when the http request is actually made. Also, no other types in |
Error-handling changes to follow
This appears to be our only use of `NodesError::DecodeValue`, so I just made `err_to_errno` treat that variant as `BSATN_DECODE_ERROR`.
Centril
approved these changes
Nov 20, 2025
coolreader18
approved these changes
Nov 20, 2025
Contributor
|
Ran the Unreal tests locally it appears to be the flaky nature of Unreal in CI. |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 21, 2025
# Description of Changes Follow up to #3684. Moves `Error` and `Timeout` out of lib, so that we don't have to implement `SpacetimeType` for them, and then removes the http dependency altogether, so that `lib` can be leaner. I also got rid of the separate `HttpValue` type, since it only really exists to mirror the `http` crate and typescript won't make use of it. # Expected complexity level and risk 1 # Testing n/a - just code movement.
3 tasks
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jan 6, 2026
…re format to match `spacetimedb_lib::http` (#3944) # Description of Changes Rust added procedure-scoped HTTP via the `procedure_http_request` ABI (see Rust PR #3684) to C# host bindings. What changed: 1) Fix for BSATN wire format * Updated C# BSATN wire types for HTTP (`BSATN.Runtime/HttpWireTypes.cs`) to match exactly the Rust stable types: * `spacetimedb_lib::http::Request` fields and order: `method`, `headers`, `timeout`, `uri`, `version` * `spacetimedb_lib::http::Response` fields and order: `headers`, `version`, `code` * Header pairs are (`name: string, value: bytes`); no additional metadata is encoded. * Added explicit “do not reorder/extend” note in the C# wire type file since the host BSATN-decodes these bytes directly and may trap on mismatch. 2) C# runtime HTTP client implementation * Implemented `ProcedureContext.Http` support (`Runtime/Http.cs`) that: * BSATN-serializes `HttpRequestWire` + sends body bytes via `procedure_http_request` * On success, BSATN-decodes `HttpResponseWire` and returns `(response, body)` as a typed `HttpResponse` * On `HTTP_ERROR`, decodes the error payload as a BSATN `string` * On `WOULD_BLOCK_TRANSACTION`, returns a stable error message (host rejects blocking HTTP while a mut tx is open) * Clamps timeouts to 500ms max (matching host behavior documented on the Rust side) 3) ABI wiring / import plumbing * Added the `procedure_http_request` import to the C# wasm shim (`Runtime/bindings.c`) under the `spacetime_10.3` import module. * Verified the generated C# P/Invoke signature matches the host ABI (request ptr/len, body ptr/len, out `[BytesSource; 2]`). 4) Procedure entrypoint contract (trap vs errno) * Updated/clarified the module procedure trampoline behavior (`Runtime/Internal/Module.call_procedure`): it must either return `Errno.OK` or trap (log + rethrow). This mirrors the host’s expectations and avoids “unexpected errno values from guest entrypoints” behavior. Behavioral notes vs Rust * Rust bindings may `expect(...)`/panic on “should never happen” serialization failures; C# runtime explicitly avoids throwing across the procedure boundary for the HTTP client path and instead returns `Result.Err` for unexpected failures. This prevents whole-module traps from user-space HTTP usage while still surfacing rich errors. * The host remains authoritative on timeout enforcement; C# now mirrors the effective host clamp (500ms) to keep behavior aligned. # API and ABI breaking changes No new host ABI introduced, and does not change the syscall signature. * This is not an ABI “break” in the host, but it is a guest-side wire compatibility correction: older C# guests built with the incorrect wire types could cause the host to trap during BSATN decode. After this PR, C# guests are compatible with the host’s expected layout. **Adds API**: `ProcedureContextBase.Http` is available for procedures No existing public types/method signatures are removed * A notable difference from Rust's panic behavior is that Rust code sometimes `expect(...)`s and may panic/trap on internal “should not happen” cases. On the other hand, C#'s `HttpClient.Send` explicitly converts unexpected failures into `Result.Err` to avoid trapping the module. This is a behavioral difference, but not an API break. # Expected complexity level and risk 2 - Mostly just creating equivalents to the Rust version. # Testing C# regression tests updated to cover: - [X] Successful request path (`ReadMySchemaViaHttp`) - [X] Invalid request path (`InvalidHttpRequest`) returning error without trapping Testing performed: - [X] Full regression suite run against local node (see chat log); no wasm traps, all suites reported `Success`. --------- Signed-off-by: Ryan <r.ekhoff@clockworklabs.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of Changes
Closes #3517 .
With this PR, procedures (at least, those defined in Rust modules) can perform HTTP requests! This is performed through a new field on the
ProcedureContext,http: HttpClient, which has a methodsendfor sending anhttp::Request, as well as a convenience wrapperget.Internally, these methods hit the
procedure_http_requestABI call / host function, which uses reqwest to perform an HTTP request. The request is run with a user-configurable timeout which defaults and is clamped to 500 ms.Rather than exposing the HTTP stream to modules, we download the entire response body immediately, within the same timeout.
I've added an example usage of
gettomodule-testwhich performs a request againstlocalhost:3000to read its own schema/moduledef.This PR also makes all procedure-related definitions in the Rust module bindings library
#[cfg(feature = "unstable")], as per #3644 . The rename of the/v1/database/:name/procedure/:nameroute is not included in this PR, so this does not close #3644 .Left as TODOs are:
reqwest::Clientsomeplace.Currently we build a new
Clientfor each request.rather than running it on the single-threaded database executor.
API and ABI breaking changes
Adds new APIs, which are marked as unstable. Adds a new ABI, which is not unstable in any meaningful way (we can't really do that). Marks unreleased APIs as unstable. Does not affect any pre-existing already-released APIs or ABIs.
Expected complexity level and risk
3 or so: networking is scary, and even though we impose a timeout which prevents these connections from being truly long-lived, they're still potentially long-lived on the scale of Tokio futures. It's possible that running them on the database core is problematic in some way, and so what I've left as a performance TODO could actually be a concurrency-correctness issue.
Testing
sdk-testsuite,procedure::http_okandprocedure::http_err, which make successful and failing requests respectively, then return its result. A client then makes some assertions about the result.