feat: bundle local (non-pip) modules for serverless endpoints#352
feat: bundle local (non-pip) modules for serverless endpoints#352deanq wants to merge 13 commits into
Conversation
Add a shared materialized_modules context manager in runpod_flash.runtime
that writes a {relative_path: source} map to a temp dir, prepends it to
sys.path, and cleans up on exit (including on exception). Wire it into
the /execute LB handler so function_code exec has access to local
modules shipped in the request body's "modules" field.
The LB /execute endpoint parses the request as a plain JSON dict (not a
FunctionRequest model), so modules are read via body.get("modules", {})
rather than an attribute on the FastAPI Request object.
|
Capy auto-review is paused for this organization because the usage-cycle auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
|
Promptless prepared a documentation update related to this change. Triggered by PR #352: feat: bundle local (non-pip) modules for serverless endpoints Documents Flash's new first-class support for bundling local (non-pip) Python modules that endpoints import: a new "Import local modules" section in the Flash endpoints guide (supported import forms, transitive resolution, the 8 MiB live-execution cap, and the parent-directory absolute-import constraint), a force-include note in the build CLI reference, and two troubleshooting entries for the new resolution and payload-size errors. Review: Document Flash local (non-pip) module bundling for endpoints |
1 participant
Summary
Flash could not reliably include local (non-pip) Python modules an endpoint imports — a sibling
utils.py, or ahelpers/package next to the endpoint, imported viaimport utils/from helpers import x. This adds first-class support across both code paths that ship user code to a worker.Resolves SLS-360.
@Endpoint(...).run(...),flash devLB dispatch): the endpoint's transitive local-import closure is now shipped inline alongside the function source.flash build/flash deploy): local modules an endpoint imports are force-included in the artifact even when the ignore filter would drop them, and the build fails loudly (clean error, not a traceback) if an endpoint's local import can't be resolved.What changed
FunctionRequest.modules: dict[str, str]— additive protocol field (POSIX relative path → source text), default empty. Backward compatible: old workers ignore it, old SDKs send nothing. Shared contract with the worker repo — see the companion worker PR: feat: import shipped local modules before executing user code runpod-workers/flash#100.stubs/local_modules.py— a shared, pureresolve_local_modules(...)that walks the transitive local-import closure (absolute, relative, andimportlib.import_module("literal")imports, at module level and inside function bodies), classifies local vs stdlib/installed, pulls in package__init__.py, terminates on cycles, warns on non-literal dynamic imports, and raisesLocalModuleResolutionErroron unresolved relative imports / out-of-root files.stubs/live_serverless.pyandstubs/load_balancer_sls.pypopulatemodulesfrom the resolved closure, with an 8 MiB inline cap raisingLocalModulePayloadTooLargeError(points users toflash deployfor larger local dependencies).runtime/module_loader.py::materialized_moduleswrites shipped modules to a temp dir onsys.pathand cleans up; wired intoruntime/lb_handler.py's/executeso the temp dir stays live through the function call (not justexec).augment_with_local_modulesincli/commands/build.pyforce-includes resolved local files past the ignore filter. Strictness is scoped to endpoint files (@Endpoint/@remote): an endpoint with an unresolved local import fails the build loudly; an incidental non-endpoint file that fails resolution is warned and skipped (no regression on unrelated files).Test plan
sys.pathon exception, no-op when empty), both live senders populatemodules, LB/executedef-now/call-later import regression, build force-include + endpoint-only-strict A/B.make quality-checkgreen (86%+ coverage).flash build: an ignoredtest_*.pysibling imported by an endpoint is force-included intoartifact.tar.gz; an endpoint with an unresolvable relative import fails the build with a clean error (exit 1, no traceback).Coordinated release
The worker-side materializer requires the companion worker PR (runpod-workers/flash#100) to be released for the live inline-shipping path to work end-to-end. The deploy path is independent (bundled files sit on the worker filesystem regardless of image version). Live-path E2E against real endpoints is a gating step once the worker image is rebuilt/published.
Follow-ups (non-blocking)
flash deploy.sys.pathmaterialization assumes one invocation at a time per worker; isolate per-invocation if concurrent execution is enabled.ep = Endpoint(...)assignment form is not yet detected (fails safe toward warn).