Skip to content

feat: add isolated Python module runtime - #3478

Open
TomCC7 wants to merge 1 commit into
cc/feat/openyam-macos-supportfrom
cc/feat/external-python-native-module
Open

feat: add isolated Python module runtime#3478
TomCC7 wants to merge 1 commit into
cc/feat/openyam-macos-supportfrom
cc/feat/external-python-native-module

Conversation

@TomCC7

@TomCC7 TomCC7 commented Aug 15, 2026

Copy link
Copy Markdown
Member

Contribution path

Problem

dimOS needs to run Python modules with dependency sets that conflict with the host environment while preserving normal typed streams, RPCs, skills, module references, lifecycle management, and restart behavior.

Solution

  • Add IsolatedPythonModule, a host-side contract backed by a concrete subclass in a sibling Python project.
  • Keep RPC definitions on the contract and forward calls through a unique internal runtime endpoint.
  • Forward streams and injected module references to the isolated runtime.
  • Support locked uv projects and Pixi-provided uv without installing runtime dependencies into the host environment.
  • Add a runnable external module example and lifecycle, validation, RPC, restart, and shutdown coverage.

How to Test

uv run pytest dimos/core/test_isolated_python_module.py dimos/core/test_isolated_python_bootstrap.py dimos/core/test_isolated_python_end_to_end.py -q
uv run python examples/external_python_module/run.py

Validated on the #3465 base: 21 focused tests passed, targeted mypy passed, pre-commit passed, and the real subprocess example exited cleanly.

Checklist

  • I have read and approved the CLA.

@mintlify

mintlify Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
dimensional 🟢 Ready View Preview Aug 15, 2026, 3:49 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.39216% with 90 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/core/isolated_python_bootstrap.py 48.68% 35 Missing and 4 partials ⚠️
dimos/core/isolated_python_module.py 79.89% 22 Missing and 15 partials ⚠️
dimos/core/test_isolated_python_bootstrap.py 87.71% 7 Missing ⚠️
dimos/core/native_module.py 25.00% 6 Missing ⚠️
dimos/core/test_isolated_python_module.py 98.66% 1 Missing ⚠️
@@            Coverage Diff             @@
##             main    #3478      +/-   ##
==========================================
+ Coverage   74.05%   76.05%   +2.00%     
==========================================
  Files        1283     1229      -54     
  Lines      124704   119197    -5507     
  Branches    11141    11117      -24     
==========================================
- Hits        92349    90661    -1688     
+ Misses      29493    25440    -4053     
- Partials     2862     3096     +234     
Flag Coverage Δ
OS-ubuntu-24.04-arm 70.47% <80.39%> (+0.03%) ⬆️
OS-ubuntu-latest 72.25% <80.39%> (+0.03%) ⬆️
Py-3.10 72.25% <80.39%> (+0.03%) ⬆️
Py-3.11 72.25% <80.39%> (+0.03%) ⬆️
Py-3.12 72.25% <80.39%> (+0.03%) ⬆️
Py-3.13 72.25% <80.39%> (+0.03%) ⬆️
Py-3.14 72.25% <80.39%> (+0.03%) ⬆️
Py-3.14t 72.25% <80.39%> (+0.03%) ⬆️
SelfHosted-Large 29.76% <30.06%> (+0.05%) ⬆️
SelfHosted-Linux 35.81% <30.06%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/core/rpc_client.py 86.66% <100.00%> (+0.83%) ⬆️
dimos/core/test_isolated_python_end_to_end.py 100.00% <100.00%> (ø)
dimos/core/test_isolated_python_module.py 98.66% <98.66%> (ø)
dimos/core/native_module.py 74.70% <25.00%> (-1.39%) ⬇️
dimos/core/test_isolated_python_bootstrap.py 87.71% <87.71%> (ø)
dimos/core/isolated_python_module.py 79.89% <79.89%> (ø)
dimos/core/isolated_python_bootstrap.py 48.68% <48.68%> (ø)

... and 85 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread dimos/core/isolated_python_bootstrap.py
@TomCC7
TomCC7 marked this pull request as ready for review August 16, 2026 07:02
@github-actions github-actions Bot added ready-to-merge Required CI checks have passed on this PR and removed ready-to-merge Required CI checks have passed on this PR labels Aug 16, 2026
Comment thread docs/usage/native_modules.md Outdated
Comment thread docs/usage/native_modules.md Outdated
Comment thread dimos/core/python_native_module.py Outdated
@TomCC7
TomCC7 force-pushed the cc/feat/external-python-native-module branch from 7b1680e to e9c8f9b Compare August 18, 2026 00:07
@greptile-apps

greptile-apps Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Too many files changed for review (139 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

@TomCC7
TomCC7 changed the base branch from main to cc/feat/openyam-macos-support August 18, 2026 00:08
@TomCC7 TomCC7 changed the title feat: add Python native module runtime feat: add isolated Python module runtime Aug 18, 2026
@TomCC7 TomCC7 mentioned this pull request Aug 18, 2026
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants