feat(core): add external Python modules

[TomCC7]

Problem

DimOS Python modules cannot use their own local dependency environment while preserving normal Blueprint, RPC, stream, module-reference, and restart behavior.

Closes DIM-1222

Solution

Add ExternalPythonModule declarations backed by a sibling uv runtime project and a private external-python worker manager. The declaration owns the RPC contract; the runtime imports that contract directly from the packaged example, without PYTHONPATH or a separate contract distribution.

The implementation adds lifecycle diagnostics, process-group cleanup, restart-safe Blueprint overrides, a runnable example external package, and real end-to-end coverage.

How to Test

uv run python examples/external_python_module/deployment.py
uv run pytest dimos/core/test_external_python_end_to_end.py -q

Contributor License Agreement

• I have read and approved the CLA.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
dimensional	🟢 Ready	View Preview	Jul 15, 2026, 8:14 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
dimensional	🟡 Building	–	Jul 15, 2026, 8:13 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[codecov]

❌ 4 Tests Failed:

Tests completed	Failed	Passed	Skipped
4281	4	4277	71

View the top 3 failed test(s) by shortest run time

dimos.e2e_tests.test_dimsim_path_replaning::test_path_replanning

Stack Traces | 114s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466222ab0b0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x746621b8b060>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74662240bd40>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x746741fca8d0>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x7466218fc680>

    @pytest.mark.self_hosted_large
    def test_path_replanning(
        lcm_spy, start_blueprint, dim_sim, direct_cmd_vel_explorer, spawn_wall_on_pose
    ) -> None:
        start_blueprint(
            "--dimsim-scene=empty",
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        # robot spawns at (3, 2)
    
        # side wall
        dim_sim.add_wall(2, -2.5, 12, -2.5)
        # other side wall
        dim_sim.add_wall(2, 3.5, 12, 3.5)
        # back wall (behind robot)
        dim_sim.add_wall(2, -2.5, 2, 3.5)
        # forward wall (far end)
        dim_sim.add_wall(12, -2.5, 12, 3.5)
        # dividing wall at x=7 with doors at y=[-1.5,-0.5] and y=[1.5,2.5]
        dim_sim.add_wall(7, -2.5, 7, -1.5)
        dim_sim.add_wall(7, -0.5, 7, 1.5)
        dim_sim.add_wall(7, 2.5, 7, 3.5)
    
        direct_cmd_vel_explorer.linear_speed = 0.8
>       direct_cmd_vel_explorer.follow_points([(10, 2), (2.5, 2), (3, 2)])

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74662240bd40>
direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x746741fca8d0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466222ab0b0>
spawn_wall_on_pose = <function spawn_wall_on_pose.<locals>.spawn at 0x7466218fc680>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x746621b8b060>

dimos/e2e_tests/test_dimsim_path_replaning.py:47: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../simulation/mujoco/direct_cmd_vel_explorer.py:105: in follow_points
    self._wait_for_pose()
        self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x746741fca8d0>
        waypoints  = [(10, 2), (2.5, 2), (3, 2)]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x746741fca8d0>

    def _wait_for_pose(self) -> PoseStamped:
        self._new_pose.clear()
        self._new_pose.wait(timeout=5.0)
>       assert self._pose is not None, "No odom received"
E       AssertionError: No odom received

self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x746741fca8d0>

.../simulation/mujoco/direct_cmd_vel_explorer.py:65: AssertionError

dimos.e2e_tests.test_manipulation_planning_groups::test_single_arm_plans_and_executes_through_control_coordinator

Stack Traces | 139s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466988844d0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fe520>

    def test_single_arm_plans_and_executes_through_control_coordinator(
        lcm_spy: LcmSpy,
        start_blueprint: Callable[..., DimosCliCall],
    ) -> None:
        """Plan with one arm and execute through its trajectory task."""
        _start_openarm_mock_planner(start_blueprint, lcm_spy)
    
        client = RPCClient(None, ManipulationModule)
        coordinator_client = RPCClient(None, ControlCoordinator)
        try:
>           [left] = _wait_for_groups(client, 1)

client     = <dimos.core.rpc_client.RPCClient object at 0x746621f146b0>
coordinator_client = <dimos.core.rpc_client.RPCClient object at 0x74662349c8c0>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466988844d0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fe520>

dimos/e2e_tests/test_manipulation_planning_groups.py:158: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

client = <dimos.core.rpc_client.RPCClient object at 0x746621f146b0>, count = 1

    def _wait_for_groups(
        client: RPCClient,
        count: int,
        *,
        timeout: float = 120.0,
    ) -> tuple[PlanningGroupInfo, ...]:
        deadline = time.time() + timeout
        last_error: BaseException | None = None
        while time.time() < deadline:
            try:
                groups = client.list_planning_groups()
                if len(groups) == count:
                    return groups
            except Exception as exc:
                last_error = exc
            time.sleep(0.5)
>       raise TimeoutError(f"Timed out waiting for {count} planning groups") from last_error
E       TimeoutError: Timed out waiting for 1 planning groups

client     = <dimos.core.rpc_client.RPCClient object at 0x746621f146b0>
count      = 1
deadline   = 1786751531.8756964
groups     = (PlanningGroupInfo('left_arm/manipulator', joints=('left_arm/openarm_left_joint1', 'left_arm/openarm_left_joint2', 'le...right_joint6', 'right_arm/openarm_right_joint7'), base='openarm_body_link0', tip='openarm_right_link7', gripper=False))
last_error = None
timeout    = 120.0

dimos/e2e_tests/test_manipulation_planning_groups.py:64: TimeoutError

dimos.e2e_tests.test_dimsim_walk_forward::test_walk_forward

Stack Traces | 200s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466234941a0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fd4e0>
human_input = <function human_input.<locals>.send_human_input at 0x7466218fd580>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74669afd82f0>

    @pytest.mark.self_hosted_large
    def test_walk_forward(lcm_spy, start_blueprint, human_input, dim_sim) -> None:
        start_blueprint(
            "run",
            "--disable",
            "spatial-memory",
            "--disable",
            "security-module",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
        origin_x, origin_y = 1, 2
        dim_sim.set_agent_position(origin_x, origin_y)
    
        human_input("move forward 3 meter")
    
>       lcm_spy.wait_until_odom_position(origin_x + 3, origin_y, threshold=0.4, timeout=120)

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74669afd82f0>
human_input = <function human_input.<locals>.send_human_input at 0x7466218fd580>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466234941a0>
origin_x   = 1
origin_y   = 2
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fd4e0>

dimos/e2e_tests/test_dimsim_walk_forward.py:37: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/lcm_spy.py:167: in wait_until_odom_position
    self.wait_for_message_result(
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7466218fe2a0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466234941a0>
        threshold  = 0.4
        timeout    = 120
        x          = 4
        y          = 2
dimos/e2e_tests/lcm_spy.py:153: in wait_for_message_result
    wait_until(
        event      = <threading.Event at 0x74669af7e000: unset>
        fail_message = 'Failed to get to position x=4, y=2'
        listener   = <function LcmSpy.wait_for_message_result.<locals>.listener at 0x7466218fe660>
        predicate  = <function LcmSpy.wait_until_odom_position.<locals>.predicate at 0x7466218fe2a0>
        self       = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7466234941a0>
        timeout    = 120
        topic      = '/odom#geometry_msgs.PoseStamped'
        type       = <class 'dimos.msgs.geometry_msgs.PoseStamped.PoseStamped'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

predicate = <bound method Event.is_set of <threading.Event at 0x74669af7e000: unset>>

    def wait_until(
        predicate: Callable[[], bool],
        *,
        timeout: float,
        interval: float = 0.1,
        message: str | None = None,
    ) -> None:
        """Poll ``predicate`` until it returns truthy or ``timeout`` elapses."""
        deadline = time.monotonic() + timeout
        while time.monotonic() < deadline:
            if predicate():
                return
            time.sleep(interval)
>       raise TimeoutError(message or f"Timed out after {timeout}s waiting for condition")
E       TimeoutError: Failed to get to position x=4, y=2

deadline   = 3220476.58054831
interval   = 0.1
message    = 'Failed to get to position x=4, y=2'
predicate  = <bound method Event.is_set of <threading.Event at 0x74669af7e000: unset>>
timeout    = 120

.../utils/testing/waiting.py:35: TimeoutError

View the full list of 1 ❄️ flaky test(s)

dimos.e2e_tests.test_dimsim_spatial_memory::test_go_to_the_bed

Flake rate in main: 16.11% (Passed 125 times, Failed 24 times)

Stack Traces | 105s run time

lcm_spy = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7467423464b0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fd080>
human_input = <function human_input.<locals>.send_human_input at 0x7466218fd260>
dim_sim = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74662349e4e0>
explore_house = <function explore_house.<locals>.explore at 0x7466218fd940>

    @pytest.mark.self_hosted_large
    def test_go_to_the_bed(lcm_spy, start_blueprint, human_input, dim_sim, explore_house) -> None:
        start_blueprint(
            "run",
            "unitree-go2-agentic",
            simulator="dimsim",
        )
        lcm_spy.save_topic(".../McpClient/on_system_modules/res")
        lcm_spy.wait_for_saved_topic(".../McpClient/on_system_modules/res", timeout=1200.0)
    
>       explore_house()

dim_sim    = <dimos.e2e_tests.dim_sim_client.DimSimClient object at 0x74662349e4e0>
explore_house = <function explore_house.<locals>.explore at 0x7466218fd940>
human_input = <function human_input.<locals>.send_human_input at 0x7466218fd260>
lcm_spy    = <dimos.e2e_tests.lcm_spy.LcmSpy object at 0x7467423464b0>
start_blueprint = <function start_blueprint.<locals>.set_name_and_start at 0x7466218fd080>

dimos/e2e_tests/test_dimsim_spatial_memory.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
dimos/e2e_tests/conftest.py:249: in explore
    direct_cmd_vel_explorer.follow_points(points)
        direct_cmd_vel_explorer = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7466234b16d0>
        points     = [(3.881, 4.803), (4.16, 1.615), (1.596, 1.505), (1.649, 0.137), (-3.644, -0.064), (-3.759, -2.661), ...]
.../simulation/mujoco/direct_cmd_vel_explorer.py:105: in follow_points
    self._wait_for_pose()
        self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7466234b16d0>
        waypoints  = [(3.881, 4.803), (4.16, 1.615), (1.596, 1.505), (1.649, 0.137), (-3.644, -0.064), (-3.759, -2.661), ...]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7466234b16d0>

    def _wait_for_pose(self) -> PoseStamped:
        self._new_pose.clear()
        self._new_pose.wait(timeout=5.0)
>       assert self._pose is not None, "No odom received"
E       AssertionError: No odom received

self       = <dimos.simulation.mujoco.direct_cmd_vel_explorer.DirectCmdVelExplorer object at 0x7466234b16d0>

.../simulation/mujoco/direct_cmd_vel_explorer.py:65: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[greptile-apps]

Greptile Summary

This PR adds external Python modules that run in a sibling Python environment. The main changes are:

• A dedicated external Python worker manager and bootstrap process.
• Coordinator support for deploying, restarting, and unloading external modules.
• A runnable external-module example and end-to-end tests.
• Documentation for the sibling-project layout and configuration flow.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
dimos/core/coordination/external_python_worker.py	Adds runtime preparation, child-process startup, handshake diagnostics, output capture, and process-group cleanup.
dimos/core/coordination/worker_manager_external_python.py	Adds deployment, cleanup, health checks, and fresh-worker restart handling for external modules.
dimos/core/coordination/module_coordinator.py	Registers the external manager and generalizes unload and restart behavior across deployment types.
dimos/core/external_python_bootstrap.py	Adds child-process contract loading, runtime validation, RPC startup, error reporting, and signal shutdown.
examples/external_python_module/python/pyproject.toml	Defines the sibling runtime project's dependencies and local DimOS source dependency.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant C as ModuleCoordinator
    participant W as External Worker Manager
    participant U as Sibling uv Project
    participant B as Bootstrap Process
    participant R as Runtime Module

    C->>W: Deploy declaration and configuration
    W->>U: Sync runtime dependencies
    W->>B: Launch with declaration and implementation references
    B->>R: Import, validate, and instantiate
    R-->>B: Start RPC server
    B-->>W: READY handshake
    W-->>C: Return RPC proxy
    C->>W: Restart or unload
    W->>B: Stop process group

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant C as ModuleCoordinator
    participant W as External Worker Manager
    participant U as Sibling uv Project
    participant B as Bootstrap Process
    participant R as Runtime Module

    C->>W: Deploy declaration and configuration
    W->>U: Sync runtime dependencies
    W->>B: Launch with declaration and implementation references
    B->>R: Import, validate, and instantiate
    R-->>B: Start RPC server
    B-->>W: READY handshake
    W-->>C: Return RPC proxy
    C->>W: Restart or unload
    W->>B: Stop process group

Loading

_{Reviews (4): Last reviewed commit: "chore: trim hosted teleoperation whitesp..." | Re-trigger Greptile}

[paul-nechifor]

Why ignore lock files?

[TomCC7]

should we keep them? I feel like there's no need to keep locks for example purpose

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

[TomCC7]

close in lieu of #3478