diff --git a/docs/deployment/configuration.md b/docs/deployment/configuration.md index b9724131..07d3ca3b 100644 --- a/docs/deployment/configuration.md +++ b/docs/deployment/configuration.md @@ -185,6 +185,45 @@ swim: --- +## Web Dashboard (`viz`) + +The runtime serves its own web dashboard: the admin ChiMod starts one HTTP +server per node, and every ChiMod that ships a `viz/` directory of +HTML/CSS/JS is mounted at `/viz/` with the routes its container +registered. Nothing here is collective, so each node serves the same UI +independently. See [Monitoring → Runtime Dashboard](./monitoring#runtime-dashboard) +for the pages and the REST API. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| `enabled` | *(see note)* | Serve the dashboard on this node. Setting this key is an explicit choice that overrides the CLI default. | +| `port` | `8080` | TCP port. `0` binds an ephemeral port (the bound port is logged). | +| `bind` | `"127.0.0.1"` | Bind address. Loopback by default: the dashboard exposes runtime internals and has no authentication, so binding `0.0.0.0` publishes them to the network — prefer an SSH tunnel. | +| `max_threads` | `16` | HTTP request threads. Thread-per-connection: a browser parks ~6 keep-alive sockets, so keep this comfortably above that. | + +```yaml +viz: + enabled: true + port: 8080 + bind: "127.0.0.1" + max_threads: 16 +``` + +:::note Who gets the dashboard by default +`clio_run start` / `clio_run restart` serve the dashboard by default; an +**embedded** runtime — a unit test, an adapter, or a library user's +`CLIO_INIT` — does not, so a library user never gets an unexpected listening +socket. Setting `viz.enabled` here (or `CLIO_VIZ_ENABLE`) decides it for +both. On the command line, `--viz` / `--no-viz`, `--viz-port N`, and +`--viz-bind ADDR` do the same; naming a port or bind address implies `--viz`. +::: + +A port that is already taken logs a warning and disables the dashboard; it +never fails the runtime. The dashboard is only compiled in when `Poco::Net` +is found at configure time. + +--- + ## Compose Section The `compose` section declaratively creates module pools at runtime startup. Each entry defines one pool. @@ -868,6 +907,18 @@ engine's ranks stagger and retry their client init so they do not stampede it. | `CLIO_INIT_ATTEMPTS` | `60` | Client-init retry attempts. | | `CLIO_INIT_SLEEP_MS` | `3000` | **Mean** backoff between attempts; the actual sleep is uniform over `[0.5×, 1.5×]` with a per-rank seed, so same-node ranks do not all retry on the same second. Default budget: 60 × ~3 s ≈ 3 minutes. | +### Web dashboard + +These override the [`viz`](#web-dashboard-viz) section. + +| Variable | Default | Description | +|----------|---------|-------------| +| `CLIO_VIZ_ENABLE` | *(daemon: on, embedded: off)* | `1`/`0`. Counts as an explicit choice, so `clio_run start` will not override it. | +| `CLIO_VIZ_PORT` | `8080` | Dashboard TCP port. `0` = pick a free one. | +| `CLIO_VIZ_BIND` | `127.0.0.1` | Dashboard bind address. | +| `CLIO_VIZ_MAX_THREADS` | `16` | HTTP thread-pool size. | +| `CLIO_VIZ_PATH` | *(unset)* | `:`-separated list of viz roots (each holding one subdirectory per ChiMod) to serve pages from instead of the ones next to the loaded ChiMod libraries — edit pages against a running daemon without rebuilding. | + ### Task scheduling | Variable | Default | Description | diff --git a/docs/deployment/monitoring.md b/docs/deployment/monitoring.md index 21b0e3f4..1e8aea94 100644 --- a/docs/deployment/monitoring.md +++ b/docs/deployment/monitoring.md @@ -1,14 +1,14 @@ --- sidebar_position: 4 title: Monitoring -description: Monitoring and debugging Clio deployments — logging, the runtime dashboard, and Darshan I/O analysis. +description: Monitoring and debugging Clio deployments — logging, the runtime's built-in web dashboard, and Darshan I/O analysis. --- # Monitoring & Debugging This page covers everything you need to observe a running CLIO Runtime -cluster: structured logging, the real-time runtime dashboard -(`context_visualizer`), and external I/O analysis via Darshan. +cluster: structured logging, the runtime's built-in web dashboard, and +external I/O analysis via Darshan. :::info In active development Additional capabilities being added: @@ -42,218 +42,356 @@ docker exec iowarp-runtime clio_run monitor ## Runtime Dashboard -The `context_visualizer` package provides a lightweight Flask web -application that lets you inspect and manage a live CLIO Runtime cluster -from your browser. It connects to the runtime using the same client API -used by application code and surfaces cluster topology, per-node worker -statistics, system resource utilization, block device stats, pool -configuration, and the active YAML config. - -### Prerequisites - -- IOWarp installed with Python support (`CLIO_CORE_ENABLE_PYTHON=ON`) -- A running CLIO Runtime (`clio_run start`) -- Python dependencies: `flask`, `pyyaml`, `msgpack` +The runtime serves its own web dashboard. When you start a daemon with +`clio_run start`, the admin ChiMod brings up an HTTP server on that node +(default `http://127.0.0.1:8080`) that shows cluster membership, per-node +worker and utilization stats, every pool composed on the node, the settings +the daemon actually came up with, and a management page for every ChiMod that +ships one (bdev, safe-bdev, CTE core). Pools can be created and destroyed +from it. + +Nothing about the dashboard is collective: each node answers from its own +state, or forwards an explicitly-addressed query to a peer, so an N-node +cluster serves the same UI from every node without coordinating. + +:::note The Python `context_visualizer` is retired +Earlier releases shipped a separate Flask process (`python -m +context_visualizer`, `pip install iowarp-core[visualizer]`) on port 5000. +It has been removed: the package, its console script, and the `flask` +dependency are gone from the pip, conda, spack, and cpack installers, and +`CLIO_CORE_ENABLE_VISUALIZER` is no longer a CMake option. Everything it did +now lives in the runtime, with no Python required. See +[Deprecation Notes](../deprecation-notes#context-visualizer). +::: -Install the Python dependencies with any of: +### Starting it ```bash -pip install flask pyyaml msgpack -# or -pip install iowarp-core[visualizer] -# or (conda) -conda install flask pyyaml python-msgpack +clio_run start # dashboard on http://127.0.0.1:8080 +clio_run start --viz-port 9000 # a different port +clio_run start --viz-bind 0.0.0.0 # reachable off-box (see the warning below) +clio_run start --no-viz # don't serve it ``` -### Starting the dashboard +When the server comes up the daemon logs the address in green: -```bash -python -m context_visualizer ``` - -Then open [http://127.0.0.1:5000](http://127.0.0.1:5000) in your browser. - -#### CLI options - -| Flag | Default | Description | -|------|---------|-------------| -| `--host` | `127.0.0.1` | Bind address. Use `0.0.0.0` to expose on all interfaces. | -| `--port` | `5000` | Listen port. | -| `--debug` | *(off)* | Enable Flask debug mode (auto-reload, verbose errors). | - -```bash -# Expose on all interfaces, non-default port -python -m context_visualizer --host 0.0.0.0 --port 8080 - -# Debug mode (development only) -python -m context_visualizer --debug +Viz: dashboard listening at http://127.0.0.1:8080 ``` -### Pages - -#### Topology (`/`) {#topology} +The same knobs are available in `~/.clio/clio.yaml` (which overrides the +CLI's default) and in the environment (which overrides the file): -The landing page shows a live grid of all nodes in the cluster. Each node card displays: - -- **Hostname** and **IP address** -- **Status badge** (alive) -- **CPU**, **RAM**, and **GPU** utilization bars (GPU shown only when GPUs are present) -- **Restart** and **Shutdown** action buttons - -The search bar supports filtering by node ID (single `3`, range `1-20`, comma-separated `1,3,5`) or by hostname/IP substring. +```yaml +viz: + enabled: true # serve the dashboard on this node + port: 8080 # 0 = bind an ephemeral port + bind: "127.0.0.1" + max_threads: 16 # HTTP request threads +``` -Clicking a node card navigates to the per-node detail page. +| Variable | Default | Description | +|----------|---------|-------------| +| `CLIO_VIZ_ENABLE` | *(see below)* | `1`/`0`. Counts as an explicit choice, so `clio_run start` will not override it. | +| `CLIO_VIZ_PORT` | `8080` | TCP port. `0` picks a free one (the bound port is logged). | +| `CLIO_VIZ_BIND` | `127.0.0.1` | Bind address. | +| `CLIO_VIZ_MAX_THREADS` | `16` | HTTP thread-pool size. Thread-per-connection; a browser parks ~6 keep-alive sockets, so keep this comfortably above that. | +| `CLIO_VIZ_PATH` | *(unset)* | `:`-separated list of viz roots to serve pages from instead of the ones next to the loaded ChiMod libraries — edit pages against a running daemon without rebuilding. | + +`clio_run start` / `clio_run restart` also accept `--viz` / `--no-viz`, +`--viz-port `, and `--viz-bind `. Naming a port or bind address +implies `--viz`. See the [Configuration Reference](./configuration#web-dashboard-viz) +for the full table. + +Two defaults are deliberate: + +- **Loopback only.** The dashboard exposes worker queues, pool layout, and + device inventory, and it has **no authentication**. Binding `0.0.0.0` + publishes all of that to the network; prefer an SSH tunnel + (`ssh -L 8080:127.0.0.1:8080 node1`). +- **Off for embedded runtimes.** A unit test, an adapter, or any process that + calls `CLIO_INIT` itself gets no listening socket unless it asks. Only the + daemon CLI turns the dashboard on by default, so a daemon has it and a + library user does not. Setting `viz.enabled` or `CLIO_VIZ_ENABLE` + decides it for both. + +A port that is already taken logs a warning and disables the dashboard; it +never fails the runtime. -#### Node detail (`/node/`) {#node-detail} +### Pages -A per-node drilldown page showing: +The admin ChiMod's shell has three tabs — **Cluster**, **Pools**, **Config** — +plus per-node and per-pool drilldowns. Every page polls the JSON API below +every couple of seconds. + +#### Cluster (`/`, `/viz/clio_admin/index.html`) {#topology} + +Cluster membership as this node sees it (from its SWIM host table): one card +per node with its IP, alive/dead state, and leader / "this node" badges. +Membership comes from local state, so a dead peer costs nothing to display. +Below the grid, **This node** shows the local CPU and memory meters (with a +sparkline) and a workers summary — queued, blocked, and processed task +counts. Clicking a node card opens its detail page. + +#### Node detail (`/viz/clio_admin/node.html?node=`) {#node-detail} + +Utilization (CPU sparkline, memory, hostname/IP/leader) and the per-worker +table: active, queued, blocked, periodic, retry, processed, load, and suspend +period. `?node=local` (the default) is this node; a node id forwards each +panel's query to that node. + +#### Pools (`/viz/clio_admin/pools.html`) + +Every pool composed on this node, grouped into one section per ChiMod. Each +pool is a card: + +- **Click** the card to open the pool's website — its module's own page with + `?pool=` preselected, or the generic pool page + (`/viz/clio_admin/pool.html`) for modules that ship none. +- The card's corner **×** shuts the pool down after a confirmation + (`POST /api/pools/{pool}/destroy`). Destroying the admin pool is refused, + since that is the runtime itself. +- Cards summarize the pool's own `Monitor("stats")` where the module answers + one, and the tab paints the pool list immediately and streams stats in; + a partial failure shows a visible error rather than a blank tab. + +**Add Pool** lists every loaded ChiMod (with a search box). Modules that +register a create form (bdev, safe-bdev, CTE core) get a typed form with a +**Validate** button that checks every field without creating anything; +any other module gets the generic **compose editor** — identity fields +(`mod_name`, `pool_name`, `pool_id`, `pool_query`) plus a raw-YAML box whose +contents are the compose entry's module parameters, driven through the same +path as `clio_run compose`. Pool ids are prefilled with a free suggestion +that you may edit. + +The **Monitor explorer** at the bottom forwards any query string to a pool's +own `Monitor()` handler (`local` or `broadcast` routing) and shows the raw +JSON — the same endpoint a module's own page uses. + +Every pool page shows the pool's **task predictions**: the learned per-method +CPU/wall coefficients the scheduler routes with, and their MAPE (average +prediction error). + +:::caution Known issue +Destroying a pool whose module runs periodic tasks (safe-bdev, CTE) currently +leaves those periodics retrying against the destroyed pool +([#1000](https://github.com/iowarp/core/issues/1000)). +::: -- **Worker statistics** — per-worker queue depth, blocked tasks, processed count, and more -- **System stats** — time-series CPU, RAM, GPU, and HBM utilization -- **Block device stats** — per-bdev pool throughput and capacity +#### Config (`/viz/clio_admin/config.html`) -#### Pools (`/pools`) +The runtime settings this daemon **actually came up with** — not what a file +says — plus the full route table: every endpoint and asset mount, and which +ChiMod registered it. This is how you can tell which modules shipped a UI. -Lists all pools defined in the `compose` section of the active configuration file: +#### Module websites -| Column | Description | -|--------|-------------| -| **Module** | Module shared-library name (`mod_name`) | -| **Pool Name** | User-defined pool name | -| **Pool ID** | Unique pool identifier | -| **Query** | Routing policy (`local`, `dynamic`, `broadcast`) | +Any ChiMod may ship a `viz/` directory of HTML/CSS/JS, mounted at +`/viz//`. Three do today: -#### Config (`/config`) +| Page | What it shows / does | +|------|----------------------| +| `/viz/clio_bdev/` | One block-device pool at a time: a capacity meter plus the full statistics table (bandwidth, latency, ops, …) from the pool's `Monitor("stats")`. | +| `/viz/clio_safe_bdev/` | Pick an array, watch recovery progress and the member roster live. **Add** a member (an existing bdev pool by name, or name + capacity to create one on the spot; `as_parity=1` raises the parity level), **replace** a failed member, or **remove** one. | +| `/viz/clio_cte_core/` | The pool's storage-target roster (score, free space, capacity, latency, bandwidth, bytes read/written) with **register** (a fresh bdev by type + capacity, or `attach_pool_id` to attach an existing pool such as a safe-bdev array) and **unregister** buttons. | -Displays the full contents of the active YAML configuration file as formatted JSON, for quick inspection without opening a terminal. +See [Adding a dashboard page to your ChiMod](../sdk/context-runtime/2.module_dev_guide.md#web-dashboard-integration-registerviz) +to ship your own. ### REST API -All pages are backed by a JSON API. You can query these endpoints directly for scripting or integration with other monitoring tools. +Every page is backed by a JSON API on the same port, which you can hit +directly with `curl` or wire into other monitoring tools. All responses are +`application/json`; errors carry an HTTP status (`400` for a bad node/pool +id, `503` when the runtime or a peer did not answer, `404` for an unknown +route) and an `{"error": "..."}` body. -#### Cluster-wide +#### Node-local | Endpoint | Method | Description | |----------|--------|-------------| -| `/api/topology` | GET | List all nodes with hostname, IP, CPU/RAM/GPU utilization | -| `/api/system` | GET | High-level system overview (connected, worker/queue/blocked/processed counts) | -| `/api/workers` | GET | Per-worker stats plus a fleet summary (local node) | -| `/api/pools` | GET | Pool list from the `compose` section of the config | -| `/api/config` | GET | Full active configuration as JSON | +| `/api/health` | GET | Liveness plus this node's identity. | +| `/api/topology` | GET | Cluster membership and SWIM state, from the local host table. | +| `/api/pools` | GET | Pools composed on this node, from the pool manager. | +| `/api/config` | GET | The settings this daemon came up with. | +| `/api/routes` | GET | Every route and asset mount, per ChiMod. | +| `/api/chimods` | GET | Loaded modules, and which registered a create form or shipped pages. | #### Per-node +`{node}` is a node id from `/api/topology`, or `local`. Addressing this +node by its own id routes locally, so the single-node case never touches the +network. Each of these is one `Monitor()` task to the admin pool on that +node; the admin's forwards use a 5 s timeout so an unreachable peer cannot +pin an HTTP thread. + | Endpoint | Method | Description | |----------|--------|-------------| -| `/api/node//workers` | GET | Worker stats for a specific node | -| `/api/node//system_stats` | GET | System resource utilization entries for a specific node | -| `/api/node//bdev_stats` | GET | Block device stats for a specific node | +| `/api/nodes/{node}/workers` | GET | Per-worker queue depth, load, and task counts. | +| `/api/nodes/{node}/system_stats` | GET | Sampled CPU / RAM / GPU utilization ring, newer than `?min_event_id`. | +| `/api/nodes/{node}/containers` | GET | Per-pool containers and their learned task-cost models. | +| `/api/nodes/{node}/bdevs` | GET | Every block device on the node, with capacity and throughput. | -#### Node management +#### Per-pool | Endpoint | Method | Description | |----------|--------|-------------| -| `/api/topology/node//shutdown` | POST | Gracefully shut down a node via SSH | -| `/api/topology/node//restart` | POST | Restart a node via SSH | - -Shutdown and restart are performed by SSHing from the dashboard host to the target node and running `clio_run stop` or `clio_run restart`. This avoids the problem of a node killing itself mid-RPC. The SSH connection uses `StrictHostKeyChecking=no` and `ConnectTimeout=5`. - -**Shutdown response:** -```json -{ - "success": true, - "returncode": 0, - "stdout": "", - "stderr": "" -} -``` +| `/api/pools/{pool}/monitor` | GET | Forward `?query=` to the pool's own `Monitor()` (`?routing=local\|broadcast\|…`). | +| `/api/pools/{pool}/destroy` | POST | Shut a pool down (refused for the admin pool). | +| `/api/pools/compose` | POST | Create a pool of any ChiMod via the compose path — fields `mod_name`, `pool_name`, `pool_id`, `pool_query`, `config` (raw YAML), optional `action=validate`. | -Exit codes `0` and `134` (SIGABRT from `std::abort()` in `InitiateShutdown`) are both treated as success. +#### Per-module -**Restart** uses `nohup` so the SSH session returns immediately while the node restarts in the background. +Modules register their own endpoints under `/api/mod//…`; the +convention is `GET …/pools` (this node's pools of that module), +`GET …/create` (a form spec) and `POST …/create` (validate or create), plus +whatever management actions the module offers: -All endpoints return `Content-Type: application/json`. On error they return an appropriate HTTP status code (e.g., `503` if the runtime is unreachable, `404` if a node is not found) with an `"error"` field in the response body. +| Endpoint | Method | Description | +|----------|--------|-------------| +| `/api/mod/clio_bdev/pools` | GET | Block-device pools on this node. | +| `/api/mod/clio_safe_bdev/{pool}/add_member` | POST | Add a data or parity member (`member_name[, capacity, bdev_type, node_id, as_parity]`). | +| `/api/mod/clio_safe_bdev/{pool}/replace_member` | POST | Replace a failed member with a fresh bdev and recover onto it (`failed_pool_id, member_name, capacity[, bdev_type, node_id]`). | +| `/api/mod/clio_safe_bdev/{pool}/remove_member` | POST | Take a member out of service (`member_pool_id[, was_faulty]`). | +| `/api/mod/clio_cte_core/{pool}/targets` | GET | The pool's registered storage targets, with score and capacity. | +| `/api/mod/clio_cte_core/{pool}/register_target` | POST | Register a bdev as a storage target (`name[, bdev_type, capacity \| attach_pool_id]`). | +| `/api/mod/clio_cte_core/{pool}/unregister_target` | POST | Remove a target from placement (`name`). | +| `/api/mod//create` | GET / POST | The module's create-form spec, and the validate/create action. | + +`GET /api/routes` on a running daemon is the authoritative list, including +any module you added yourself. + +POST bodies of type `application/x-www-form-urlencoded` are parsed into the +same parameter map as the query string (the query string wins on a +collision), so `curl -d` and `?key=value` are interchangeable. Create +endpoints answer `{"ok":true,"pool_id":…}` on success and +`{"ok":false,"errors":{field: message}}` at HTTP 400 on validation failure, +all fields at once. + +The GETs are all read-only. The POSTs create pools and manage module +resources — the same operations any local RPC client can already perform — +and stay node-local; nothing here shuts a node down or deletes data. (The +retired Python dashboard's SSH-driven node shutdown/restart buttons have no +equivalent; use `clio_run stop` / `clio_run restart` on the node.) #### Examples ```bash -# Get cluster topology -curl http://127.0.0.1:5000/api/topology +# Is the dashboard up, and which node am I talking to? +curl http://127.0.0.1:8080/api/health + +# Cluster membership +curl http://127.0.0.1:8080/api/topology + +# Worker stats on this node, then on node 2 +curl http://127.0.0.1:8080/api/nodes/local/workers +curl http://127.0.0.1:8080/api/nodes/2/workers -# Get system overview -curl http://127.0.0.1:5000/api/system +# Ask the CTE core pool (512.0) for its stats, and list its targets +curl "http://127.0.0.1:8080/api/pools/512.0/monitor?query=stats" +curl http://127.0.0.1:8080/api/mod/clio_cte_core/512.0/targets -# Get worker stats for node 2 -curl http://127.0.0.1:5000/api/node/2/workers +# Validate, then create, a 1 GB RAM bdev through the module's own form +curl -d "action=validate&pool_name=ram::scratch&pool_id=305.0&bdev_type=ram&capacity=1GB" \ + http://127.0.0.1:8080/api/mod/clio_bdev/create +curl -d "pool_name=ram::scratch&pool_id=305.0&bdev_type=ram&capacity=1GB" \ + http://127.0.0.1:8080/api/mod/clio_bdev/create -# Shut down node 3 -curl -X POST http://127.0.0.1:5000/api/topology/node/3/shutdown +# Shut that pool down again +curl -X POST http://127.0.0.1:8080/api/pools/305.0/destroy -# Restart node 3 -curl -X POST http://127.0.0.1:5000/api/topology/node/3/restart +# What did this daemon register? +curl http://127.0.0.1:8080/api/routes ``` -### Configuration file discovery +### Where the pages come from -The dashboard reads the same config file as the runtime, using the same search order: +A ChiMod's assets are found **relative to the library the runtime actually +loaded**, so a built-but-not-installed tree and an installed tree both work +with no configuration: -| Source | Priority | -|--------|----------| -| `CLIO_SERVER_CONF` environment variable | **1st** | -| `~/.clio/clio.yaml` | **2nd** | +| Layout | Libraries | Assets | +|--------|-----------|--------| +| built, not installed | `/bin` | `/bin/viz/` | +| installed (make install, wheel, deb/rpm) | `/lib` | `/share/clio/viz/` | -Legacy paths (`~/.chimaera/…`) and the legacy `CHI_SERVER_CONF` env var are **no longer** accepted. See [Deprecation Notes](../deprecation-notes), and [Configuration](./configuration) for the file format. +`$CLIO_VIZ_PATH` is checked first. If the daemon logs +`Viz: no viz/ assets found for ChiMod `, the module's `viz/` directory +was not staged next to its library — rebuild, or point `CLIO_VIZ_PATH` at +the source tree. -### Connection lifecycle +### Build requirements -The dashboard connects to the runtime lazily — on the first request that needs live data. If the runtime is not yet running when the dashboard starts, it will show a disconnected state and retry on subsequent requests. Shutdown is handled automatically via `atexit` so the client is finalized cleanly when the server process exits. +The HTTP server is `Poco::Net`, linked privately into the runtime library +and picked up by `find_package(Poco COMPONENTS Net …)` at configure time. +The `iowarp/deps-cpu` devcontainer image (`libpoco-dev`) and the conda +recipe (`poco`) provide it, so source builds in the devcontainer and conda +installs have the dashboard. Without Poco the runtime still builds and +starts — CMake reports `Web dashboard (viz): DISABLED (Poco::Net not found)`, +`clio_run start` logs a warning, and the dashboard is simply absent. Install +`libpoco-dev` (apt), `poco-devel` (dnf), or `poco` (conda-forge / brew) and +reconfigure to enable it. `ctest -R cr_viz_tests` exercises the router and +drives the real server over TCP. ### Docker / remote access -When running the runtime inside Docker or on a remote host, bind the dashboard to all interfaces and forward the port: - -```bash -# On the host running the runtime -python -m context_visualizer --host 0.0.0.0 --port 5000 -``` +The daemon serves the dashboard itself, so exposing it from a container is +just a bind address and a port mapping: ```yaml # docker-compose.yml — expose the dashboard port alongside the runtime services: iowarp: image: iowarp/deploy-cpu:latest + environment: + - CLIO_VIZ_BIND=0.0.0.0 + - CLIO_VIZ_PORT=8080 ports: - "9413:9413" # CLIO Runtime RPC - - "5000:5000" # Dashboard - command: > - bash -c "clio_run start & - python -m context_visualizer --host 0.0.0.0" + - "8080:8080" # Dashboard + command: ["clio_run", "start"] +``` + +For a remote host, leave the bind address on loopback and tunnel instead: + +```bash +ssh -L 8080:127.0.0.1:8080 user@node1 +# then open http://127.0.0.1:8080 locally ``` :::warning -The dashboard has no authentication. Do not expose it on a public network without a reverse proxy that enforces access control. +The dashboard has no authentication, and it can create and destroy pools. +Do not bind it to a public interface without a reverse proxy that enforces +access control. ::: ### Try it: interactive Docker cluster {#interactive-cluster} -An interactive test environment is provided that spins up a **4-node CLIO Runtime cluster** with the dashboard so you can explore all features from your browser. +An interactive test environment spins up an **8-node CLIO Runtime cluster** +with the dashboard served by node 1 so you can explore every page from your +browser. #### Location ``` context-runtime/test/integration/interactive/ -├── docker-compose.yml # 4-node runtime cluster -├── hostfile # Node IP addresses (172.28.0.10-13) +├── docker-compose.yml # 8-node runtime cluster +├── hostfile # Node IP addresses ├── clio_conf.yaml # Runtime configuration └── run.sh # Launcher script ``` #### How it works -- **4 Docker containers** (`iowarp-interactive-node1` through `node4`) run the CLIO Runtime on a private `172.28.0.0/16` network, each with `sshd` for SSH-based shutdown/restart -- **Node 1** also runs the dashboard alongside its runtime -- The script connects the devcontainer to the Docker network and starts a local port-forward so that `localhost:5000` reaches the dashboard inside Docker — VS Code then auto-forwards this to your host browser -- SSH keys are distributed via a shared Docker volume so the dashboard can authenticate to all nodes +- **8 Docker containers** (`iowarp-interactive-node1` through `node8`) run + the CLIO Runtime on a private Docker network +- **Node 1** starts its runtime with `CLIO_VIZ_ENABLE=1`, `CLIO_VIZ_PORT=5000`, + and `CLIO_VIZ_BIND=0.0.0.0`, so its built-in dashboard is reachable from + outside the container; it also runs a `bdev_io` throughput benchmark for a + few minutes so the worker and device pages have something to show +- The script connects the devcontainer to the Docker network and starts a + local port-forward so that `localhost:5000` reaches node 1's dashboard — + VS Code then auto-forwards this to your host browser #### Running @@ -269,13 +407,17 @@ bash run.sh start # Follow runtime container logs bash run.sh logs -# Stop everything (cluster + dashboard) +# Stop everything (cluster + port forward) bash run.sh stop ``` -Once the cluster is up (~15 seconds), open [http://localhost:5000](http://localhost:5000) to browse the topology, click into individual nodes, and use the Restart/Shutdown buttons. +Once the cluster is up (~15 seconds), open [http://localhost:5000](http://localhost:5000) +to browse the cluster, click into individual nodes, open the Pools tab, and +try **Add Pool** against a real multi-node runtime. `DASHBOARD_PORT` changes +the forwarded port. -If running from a devcontainer or a host where the workspace is at a different path, set `HOST_WORKSPACE`: +If running from a devcontainer or a host where the workspace is at a +different path, set `HOST_WORKSPACE`: ```bash HOST_WORKSPACE=/host/path/to/workspace bash run.sh diff --git a/docs/deprecation-notes.md b/docs/deprecation-notes.md index 432a2234..4401959f 100644 --- a/docs/deprecation-notes.md +++ b/docs/deprecation-notes.md @@ -146,6 +146,27 @@ dynamic-loader stability. They have since been renamed too: Downstream CMake must `find_package` / link the `clio_*` target names. +### The Python `context-visualizer` {#context-visualizer} + +The standalone Flask dashboard has been removed; the runtime now serves the +dashboard itself (see [Monitoring → Runtime Dashboard](./deployment/monitoring#runtime-dashboard)). + +| Then | Now | +|------|-----| +| `python -m context_visualizer [--host H] [--port P]`, `context-visualizer` console script | `clio_run start` serves it; `--viz-bind H --viz-port P` (or `viz:` in `clio.yaml`, or `CLIO_VIZ_*`) | +| `http://127.0.0.1:5000` | `http://127.0.0.1:8080` | +| `pip install iowarp-core[visualizer]`, the `flask` dependency | nothing to install — no Python involved | +| `-DCLIO_CORE_ENABLE_VISUALIZER=ON` | option removed; the dashboard builds whenever `Poco::Net` is found | +| `GET /api/node//workers`, `/api/node//system_stats`, `/api/node//bdev_stats` | `GET /api/nodes/{node}/workers`, `…/system_stats`, `…/bdevs` (`{node}` may be `local`) | +| `GET /api/system`, `/api/workers` | `GET /api/nodes/local/workers` (`/api/health` for liveness) | +| `POST /api/topology/node//shutdown` / `restart` (via SSH) | no equivalent — run `clio_run stop` / `clio_run restart` on the node | +| Pools page listing the config's `compose` section | Pools page listing the pools actually composed, with **Add Pool** and per-pool destroy | + +The `context_visualizer` package, `installers/*` entries, and the pip +`[visualizer]` extra are gone; a script that still runs +`python -m context_visualizer` will fail with `No module named +context_visualizer`. + --- ## Migrating a downstream project diff --git a/docs/getting-started/dashboard.mdx b/docs/getting-started/dashboard.mdx new file mode 100644 index 00000000..064fe91a --- /dev/null +++ b/docs/getting-started/dashboard.mdx @@ -0,0 +1,176 @@ +--- +sidebar_position: 3 +title: Dashboard +description: Open the runtime's built-in web dashboard, take a tour of its pages, and create your first pool from the browser. +--- + +# Dashboard + +Every CLIO Runtime daemon serves a web dashboard. Start the runtime and open +[http://127.0.0.1:8080](http://127.0.0.1:8080) — there is nothing to install, +no separate process to run, and no Python involved. + +This page is a tour. The full reference — configuration, the REST API, and +how to add pages for your own ChiMod — is under +[Deployment → Monitoring](../deployment/monitoring#runtime-dashboard). + +## 1. Open it + +Start the runtime as in the [Quick Start](./quick-start): + +```bash +clio_run start & +``` + +The daemon logs where the dashboard is listening: + +``` +Viz: dashboard listening at http://127.0.0.1:8080 +``` + +Open that address in a browser. If `:8080` is taken on your machine, pick +another port; if you are on a remote host, keep the dashboard on loopback +and tunnel to it: + +```bash +# Different port +clio_run start --viz-port 9000 & + +# Remote host: tunnel from your laptop, then open http://127.0.0.1:8080 locally +ssh -L 8080:127.0.0.1:8080 user@node1 +``` + +Running in Docker? Bind the dashboard to all interfaces inside the +container and publish the port: + +```yaml +services: + iowarp: + image: iowarp/deploy-cpu:latest + environment: + - CLIO_VIZ_BIND=0.0.0.0 + ports: + - "9413:9413" # runtime RPC + - "8080:8080" # dashboard + command: ["clio_run", "start"] +``` + +:::warning No authentication +The dashboard shows runtime internals and can create and destroy pools. It +listens on loopback by default for a reason — do not bind it to a public +interface without a reverse proxy that enforces access control. +::: + +:::note If nothing is listening on 8080 +The dashboard is compiled into the runtime only when the build found +`Poco::Net`. If `clio_run start` logs a `Viz:` warning instead of the +green *listening* line, your build lacks it — see +[Build requirements](../deployment/monitoring#build-requirements). +::: + +## 2. The tour + +The navigation bar has three tabs — **Cluster**, **Pools**, **Config** — and +a connection indicator. Every page refreshes itself every couple of seconds. + +### Cluster + +The landing page. One card per node in the cluster with its IP, alive/dead +state, and *leader* / *this node* badges — on a laptop that is a single +card. Below it, **This node** shows live CPU and memory meters and a +workers summary: how many tasks are queued, blocked, and processed. Click a +node card (or the Workers card) to open the node page: utilization plus a +per-worker table of queue depth, blocked and periodic tasks, load, and +suspend period. + +### Pools + +Everything composed on this node, grouped by ChiMod. With the default +configuration you will see the DRAM block device (`clio_bdev`), the CTE +core (`clio_cte_core`) and its interposition chain (cache, indexer, +replication), the CAE, and the filesystem pool (`clio_cte_filesystem`) +that the FUSE mount drives. + +Each pool is a card. Click it to open the pool's own website: + +| Module | What its page shows | +|--------|---------------------| +| `clio_bdev` | One block device at a time: a capacity meter and the device's full statistics (bandwidth, latency, ops, …). | +| `clio_cte_core` | The storage-target roster — score, free space, capacity, latency, bandwidth, bytes read/written — with **register** / **unregister** buttons. | +| `clio_safe_bdev` | Array members and recovery progress, with add / replace / remove. | +| everything else | A generic pool page: identity, the scheduler's learned **task predictions** for the pool, and a box to run any of the pool's `Monitor()` queries. | + +The corner **×** on a card destroys that pool (after a confirmation). The +admin pool cannot be destroyed — that is the runtime itself. + +### Config + +The settings the daemon **actually came up with** — after the config file, +environment variables, and CLI flags were applied — plus the route table: +every REST endpoint and static-page mount, and which ChiMod registered it. +Handy when a setting does not seem to take effect. + +## 3. Create a pool from the browser + +Try the **Add Pool** button on the Pools tab: + +1. Pick `clio_bdev` from the module list. +2. Fill in the form — a RAM device is the quickest: + - **pool_name**: `ram::scratch` + - **pool_id**: leave the suggested free id + - **bdev_type**: `ram` + - **capacity**: `256MB` +3. Click **Validate**. Every field is checked and nothing is created; errors + come back per field. +4. Click **Create**. A new card appears under `clio_bdev`; click it to watch + the device's stats. + +Modules that ship a form (bdev, safe-bdev, CTE core) get typed fields like +these. Any other module gets a **compose editor** instead — the same identity +fields plus a raw-YAML box for module parameters, so anything you can put in +`clio.yaml`'s `compose` section you can also create here. + +To retire the pool, click the **×** on its card. + +## 4. Watch real work + +With the [FUSE mount from the Quick Start](./quick-start#3-mount-the-filesystem) +up, copy a few files into it and keep the dashboard open: + +- **Cluster → This node**: the *processed* count climbs and the worker + meters move. +- **Pools → `clio_cte_core`**: the target roster's *bytes written* and + *writes* columns show the pages landing on the DRAM tier. +- **Pools → `clio_bdev`**: the block device's capacity meter and statistics + update as pages are written. + +## 5. Turn it off or move it + +```bash +clio_run start --no-viz # don't serve it on this node +clio_run start --viz-bind 0.0.0.0 # reachable off-box (see the warning above) +``` + +Or set it once in `~/.clio/clio.yaml`: + +```yaml +viz: + enabled: true + port: 8080 + bind: "127.0.0.1" +``` + +Only a daemon started with `clio_run start` serves the dashboard by default. +A program that embeds the runtime (a test, an adapter, your own +`CLIO_INIT`) does not open a port unless you set `viz.enabled` or +`CLIO_VIZ_ENABLE=1`. + +## Next steps + +- [Monitoring → Runtime Dashboard](../deployment/monitoring#runtime-dashboard) — + the REST API behind every page, config knobs, and the multi-node + interactive cluster +- [Web Dashboard Integration](../sdk/context-runtime/2.module_dev_guide.md#web-dashboard-integration-registerviz) — + give your own ChiMod a page and endpoints +- [Configuration → Web Dashboard](../deployment/configuration#web-dashboard-viz) — + the `viz` section and `CLIO_VIZ_*` variables diff --git a/docs/getting-started/installation.mdx b/docs/getting-started/installation.mdx index cbf1357a..207b808e 100644 --- a/docs/getting-started/installation.mdx +++ b/docs/getting-started/installation.mdx @@ -64,7 +64,7 @@ need: - **HDF5** scientific data ingestion - **ADIOS2** adapter for streaming analytics - **Compression** backends (LibPressio, Blosc, etc.) -- **FUSE** adapter +- **FUSE** adapter on **macOS** (the Linux and Windows wheels already ship `clio_cte_fuse`) - **Custom ChiMods** you intend to compile against the C++ headers - **Sanitizer / debug builds** for development diff --git a/docs/getting-started/quick-start.mdx b/docs/getting-started/quick-start.mdx index c06745cf..256590d2 100644 --- a/docs/getting-started/quick-start.mdx +++ b/docs/getting-started/quick-start.mdx @@ -1,15 +1,15 @@ --- sidebar_position: 2 title: Quick Start -description: Start the CLIO Runtime and run your first CEE example. +description: Start the CLIO Runtime, mount the IOWarp filesystem, write files to it, and watch it in the dashboard. --- # Quick Start This guide assumes you have already [installed IOWarp](./installation). -It walks you through activating your environment, the default -configuration, starting the CLIO Runtime, and running a Context -Exploration Engine (CEE) example. +In a few minutes you will start the CLIO Runtime, mount the IOWarp +filesystem on your machine, write files to it with ordinary tools, and +watch what happens in the runtime's dashboard. ## 1. Activate Your Environment @@ -55,55 +55,192 @@ Verify the environment is active: clio_run --help ``` -## 2. Default Configuration +## 2. Start the Runtime -During installation a default `~/.clio/clio.yaml` is seeded for you. -You can edit it directly or point the runtime at a custom file with -`CLIO_SERVER_CONF=/path/to/config.yaml`. Existing files are never -overwritten on reinstall. +```bash +clio_run start & +``` -The default configuration starts **4 worker threads** on **port 9413** and -composes these modules automatically: +That is the whole deployment. The default `~/.clio/clio.yaml` seeded at +install time brings up everything the filesystem needs on this node: -| Module | Pool | Purpose | -|--------|------|---------| -| `clio_bdev` | `301.0` | DRAM block device (`0g` = 80% of total system DRAM) | -| `clio_cae_core` | `400.0` | Context Assimilation Engine — forwards its data path to CTE | -| `clio_cte_core` | `512.0` | Context Transfer Engine: a DRAM tier plus a 10 GB persistent disk tier at `~/.clio/cte_disk_tier.dat`, with metadata logging to `~/.clio/cte_metadata_log` | -| `clio_cte_replication` | `561.0` | One persistent replica of every blob on the disk tier | -| `clio_cte_indexer` | `564.0` | BM25 index that serves semantic search | -| `clio_cte_cache` | `563.0` | Node-local raw copies (the zero-IPC read fast path) | -| `clio_cte_filesystem` | `560.0` | POSIX-style filesystem the FUSE / POSIX adapters drive | +- a **DRAM block device** (80% of system memory by default) and a **10 GB + persistent disk tier** at `~/.clio/cte_disk_tier.dat` +- the **Context Transfer Engine** core with its interposition chain + (cache → indexer → replication → core), plus the Context Assimilation + Engine +- the **filesystem pool** (`clio_cte_filesystem`) that the FUSE mount in the + next step drives — paths become CTE tags, file contents become 1 MiB + page blobs, and every page rides the chain +- the **web dashboard** on [http://127.0.0.1:8080](http://127.0.0.1:8080) -The last four form the **interposition chain** — each speaks the CTE core's -own task interface and stacks over the next via `next_pool_id`: +Confirm it is up: +```bash +clio_run monitor --once ``` -cache(563.0) → indexer(564.0) → replication(561.0) → core(512.0) + +or open the dashboard in a browser. Nothing here needs to be composed by +hand; every layer is optional and can be trimmed later — see the +[shipped default](../deployment/configuration#the-shipped-default-clioclioyaml) +in the Configuration Reference. + +## 3. Mount the Filesystem + +The FUSE adapter (`clio_cte_fuse`) presents the runtime's filesystem pool as +an ordinary directory. It attaches as a client to the runtime you just +started, so set `CLIO_WITH_RUNTIME=0` — without it the adapter would spin up +a private runtime of its own and nothing you write would be visible to +anyone else. + + + + +Needs the distro's FUSE 3 runtime (`sudo apt install fuse3` / +`sudo dnf install fuse3`). In a second terminal: + +```bash +mkdir -p ~/cte-mnt +CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/cte-mnt -f ``` -Because they share the core's vocabulary, none of this changes how you call -CTE. Every layer is optional: delete its `compose` entry and re-point the -entry above it. See -[Cache / Replication / Indexing ChiMods](../sdk/context-transfer-engine/chimod-chain). + + -## 3. Start the Runtime +Needs [macFUSE](https://macfuse.github.io/) (`brew install --cask macfuse`, +then approve the kernel extension in System Settings → Privacy & Security +and reboot). The macOS wheel does not include the adapter, so build it with +the `release-mac-fuse` preset first — see +[FUSE Adapter → Installation](../deployment/adapter#installation). In a +second terminal: ```bash -# Start in the background -clio_run start & +mkdir -p ~/cte-mnt +CLIO_WITH_RUNTIME=0 clio_cte_fuse ~/cte-mnt -f +``` -# Verify it is running -clio_run monitor --once + + + +Needs [WinFsp](https://winfsp.dev). Pick a free drive letter; in a second +PowerShell window: + +```powershell +$env:CLIO_WITH_RUNTIME = "0" +clio_cte_fuse Z: -f +``` + +Give it a moment, then confirm with `Test-Path Z:\`. + + + + +`-f` keeps the adapter in the foreground so you can see what it is doing; +omit it to run it as a daemon. Everything about the mount — other +mountpoints, the FSKit backend on macOS, mounting at a directory on Windows, +libfuse options — is in the [FUSE Adapter](../deployment/adapter) guide. + +## 4. Use It + +Any program can now read and write files on the mount with standard tools; +nothing is IOWarp-specific. + + + + +```bash +echo "Hello, IOWarp!" > ~/cte-mnt/greeting.txt +mkdir -p ~/cte-mnt/data +cp dataset.csv ~/cte-mnt/data/ + +cat ~/cte-mnt/greeting.txt +ls -l ~/cte-mnt/ +``` + + + + +```bash +echo "Hello, IOWarp!" > ~/cte-mnt/greeting.txt +mkdir -p ~/cte-mnt/data +cp dataset.csv ~/cte-mnt/data/ + +cat ~/cte-mnt/greeting.txt +ls -l ~/cte-mnt/ +``` + + + + +```powershell +Set-Content -Path Z:\greeting.txt -Value "Hello, IOWarp!" +New-Item -ItemType Directory -Path Z:\data -Force | Out-Null +Copy-Item dataset.csv Z:\data\ + +Get-Content Z:\greeting.txt +Get-ChildItem Z:\ +``` + +Explorer, `cmd.exe`, and anything else that takes a path work against `Z:` +too. + + + + +## 5. Watch It in the Dashboard + +Open [http://127.0.0.1:8080](http://127.0.0.1:8080). While files land on the +mount: + +- **Cluster** shows this node's CPU and memory and the workers' processed + task count climbing. +- **Pools** lists every pool the runtime composed. Click `clio_cte_core` to + see the storage targets your writes are landing on, or the `clio_bdev` + card to watch the DRAM device's capacity and statistics. +- **Config** shows the settings the daemon actually came up with. + +Take the [Dashboard tour](./dashboard) for the rest, including creating pools +from the browser. + +## 6. Stop + +Unmount, then stop the runtime: -# When done + + + +```bash +fusermount3 -u ~/cte-mnt # or Ctrl-C in the clio_cte_fuse terminal clio_run stop ``` -## 4. Context Exploration Engine Example + + + +```bash +umount ~/cte-mnt # or Ctrl-C in the clio_cte_fuse terminal +clio_run stop +``` + + + + +```powershell +Stop-Process -Name clio_cte_fuse -Force # or Ctrl-C in the clio_cte_fuse window +clio_run stop +``` + + + + +Files on the DRAM tier are gone when the runtime stops; the persistent disk +tier and its metadata log under `~/.clio/` survive a restart. + +## 7. Optional: Assimilate Data from Python -The Context Exploration Engine (CEE) lets you assimilate data into IOWarp, -query for it by name or regex, retrieve it, and clean up -- all from Python. +The filesystem is one door into IOWarp. The Context Exploration Engine +(CEE) is another: it lets you assimilate data into IOWarp, query for it by +name or regex, retrieve it, and clean up -- all from Python. Save the following as **`cee_quickstart.py`**: @@ -156,7 +293,7 @@ os.unlink(tmp.name) print("Done!") ``` -Run it (make sure the runtime is still running in the background): +Run it against a running runtime (`clio_run start &` again if you stopped it above): ```bash python3 cee_quickstart.py @@ -173,7 +310,7 @@ Destroyed tag 'quickstart_demo' Done! ``` -## 5. Key Environment Variables +## 8. Key Environment Variables | Variable | Description | |----------|-------------| @@ -183,6 +320,8 @@ Done! | `CLIO_SERVER_ADDR` | Address clients dial to reach the runtime (default: `127.0.0.1`). | | `CLIO_BIND_ADDR` | Address the runtime's listeners bind to (default: `0.0.0.0`). | | `CLIO_EPHEMERAL` | `1` starts the runtime bare — the `compose` section is skipped. | +| `CLIO_WITH_RUNTIME` | `0` makes a client (such as `clio_cte_fuse`) attach to the running daemon instead of embedding its own runtime. | +| `CLIO_VIZ_PORT` / `CLIO_VIZ_BIND` / `CLIO_VIZ_ENABLE` | Dashboard port (default `8080`), bind address (default `127.0.0.1`), and on/off. | | `CLIO_CTE_POOL` | Bind the CTE client to an interposing pool, e.g. `563.0` for the chain top. | | `CTP_LOG_LEVEL` | Logging verbosity: `debug`, `info`, `success`, `warning`, `error`, `fatal` | | `CTP_LOG_OUT` | Also write log output to this file. | @@ -192,6 +331,10 @@ for the full list. ## Next Steps +- [Dashboard](./dashboard) -- tour the runtime's web dashboard and create + pools from the browser +- [FUSE Adapter](../deployment/adapter) -- every mount option, platform + notes, and troubleshooting - Edit `~/.clio/clio.yaml` to tune thread counts, storage tiers, or add file-backed block devices - [Configuration Reference](../deployment/configuration) -- full parameter diff --git a/docs/intro.md b/docs/intro.md index 6c74a724..b858a6c8 100644 --- a/docs/intro.md +++ b/docs/intro.md @@ -13,7 +13,7 @@ description: IOWarp is a context orchestration platform for agentic AI in scient | Audience | Start here | |----------|-----------| -| **Researchers** | [Installation Guide](./getting-started/installation) → [Quick Start](./getting-started/quick-start) | +| **Researchers** | [Installation Guide](./getting-started/installation) → [Quick Start](./getting-started/quick-start) → [Dashboard](./getting-started/dashboard) | | **HPC Practitioners** | [Deployment Guide](./deployment/hpc-cluster) → [Configuration](./deployment/configuration) | | **Developers** | [SDK Reference](./sdk/) → [Python API](./api/python) | | **AI Researchers** | [CLIO Kit MCP Servers](./clio-kit/mcp-servers) → [Platform Overview](https://iowarp.ai/platform/) | diff --git a/docs/sdk/context-runtime/2.module_dev_guide.md b/docs/sdk/context-runtime/2.module_dev_guide.md index 865a3969..ffb266ae 100644 --- a/docs/sdk/context-runtime/2.module_dev_guide.md +++ b/docs/sdk/context-runtime/2.module_dev_guide.md @@ -9,6 +9,7 @@ - [Client Implementation](#client-implementation-mod_name_clienthcc) - [Runtime Container](#runtime-container-mod_name_runtimehcc) - [Dynamic Scheduling (ScheduleTask)](#dynamic-scheduling-scheduletask) + - [Web Dashboard Integration (RegisterViz)](#web-dashboard-integration-registerviz) 5. [Configuration and Code Generation](#configuration-and-code-generation) 6. [Task Development](#task-development) 7. [Synchronization Primitives](#synchronization-primitives) @@ -754,6 +755,158 @@ The worker calls `ScheduleTask()` to obtain the route, sends the task to the resolved location, and then runs it to completion. Overriding `ScheduleTask()` is the only step a module needs; the worker handles the rest. +### Web Dashboard Integration (RegisterViz) + +The runtime serves a web dashboard from every daemon (see +[Monitoring → Runtime Dashboard](../../deployment/monitoring#runtime-dashboard)). +A ChiMod can give itself a **management website** in two independent steps: +ship a `viz/` directory of static pages, and register HTTP routes from its +Container. Neither requires the other, and neither pulls any HTTP dependency +into your module. + +#### Shipping pages + +Drop HTML/CSS/JS in `viz/` next to your module's sources and add one line to +the module's `CMakeLists.txt`: + +```cmake +clio_add_viz_assets(MOD_NAME my_module) # defaults to ./viz +``` + +The runtime mounts the directory at `/viz/my_module/` as soon as the module +is loaded. The assets are staged relative to the library the runtime loads +(`/bin/viz/my_module` in a build tree, `/share/clio/viz/my_module` +when installed), so both layouts work with no configuration; `CLIO_VIZ_PATH` +overrides both for editing pages against a live daemon. + +The admin shell's stylesheet and helpers are served from the same origin, so a +module page can reuse them: + +```html + + +``` + +`api.js` provides `API.get(path)` / `API.post(path, fields)`, `poolParam()` +(the `?pool=` the Pools tab hands your page), `table()`, `meter()`, +`sparkline()`, `poll(fn, ms)`, and `renderPredictions(elId, poolId)` for the +per-pool task-cost table every pool page shows. + +If the data your page needs is already a `Monitor()` query your module +answers, you need no C++ at all — fetch it through the generic forward: + +```js +const data = await API.get(`/api/pools/${poolId}/monitor?query=stats`); +``` + +#### Registering routes + +For anything else, override `Container::RegisterViz`: + +```cpp +#include +#include + +void Runtime::RegisterViz(clio::run::viz::VizServer &viz, + const std::string &mod_name) { + viz.AddRoute({"GET", "/api/mod/" + mod_name + "/{pool}/summary", mod_name, + "One-line summary of a pool of this module", + [](const clio::run::viz::Request &req, + clio::run::viz::Response &resp) { + clio::run::PoolId pool_id; + try { + pool_id = clio::run::PoolId::FromString(req.Var("pool")); + } catch (const std::exception &e) { + resp.Error(400, "bad pool id: " + req.Var("pool")); + return; + } + Client client(pool_id); // construct locally, never capture this + auto fut = client.AsyncGetStats(clio::run::PoolQuery::Local()); + if (!fut.Wait(5.0f)) { // always pass a timeout (seconds) + resp.Error(503, "pool did not answer"); + return; + } + clio::run::viz::JsonWriter w; + w.BeginObject(); + w.Field("pool_id", pool_id.ToString()); + w.Field("verbose", req.Param("verbose", "0") == "1"); + // ... fields from fut->... + w.EndObject(); + resp.Json(w.Str()); + }}); +} +``` + +- **`Route`** is `{method, path, mod_name, doc, handler}`. Path segments of + the form `{name}` match one segment and bind into `Request::path_vars` + (`req.Var("pool")`); query-string and `application/x-www-form-urlencoded` + POST fields both land in `Request::params` (`req.Param(key, default)`). +- **`Response`** defaults to an empty `200` JSON body; use `resp.Json(str)` or + `resp.Error(code, message)` (which emits `{"error": "..."}`). +- **`JsonWriter`** (`clio_runtime/viz/viz_json.h`) is a dependency-free + streaming JSON emitter (`BeginObject/EndObject`, `BeginArray/EndArray`, + `Key`, `Value`, `Field`, `RawField`, `Str()`), so a module answers routes + without inheriting the dashboard's HTTP dependencies. +- **`GET /api/routes`** lists every registered route with its `doc`, which is + how the dashboard's Config page shows what your module added. + +**When `RegisterViz` is called — and what that forbids.** It runs *twice +over*: once at module-**load** time on a throwaway, default-constructed +prototype instance (so your pages and create form exist before any pool of +the module does), and again per container from +`PoolManager::RegisterContainer`. Registration is idempotent — the first +`(method, path)` wins — so the second call is a no-op. Because of the +prototype call, **neither the method body nor any handler may read container +state or capture `this`**: capture by value and reach state through the +manager singletons or a locally-constructed client (`Client(pool_id)` +resolved from the request), as above. A module that captures `this` anyway +must call `viz.RemoveModule(mod_name)` before the container dies, or a later +request runs the handler over freed memory. + +**What a handler may do.** Handlers run on the dashboard's own HTTP thread +pool, **not** on a runtime worker: + +- **Blocking is allowed.** Reading manager state (`CLIO_POOL_MANAGER`, + `CLIO_IPC`, `CLIO_WORK_ORCHESTRATOR`, `CLIO_CONFIG_MANAGER`) is fine, and so + is submitting a task and waiting on its `Future` — the same thing a FUSE + thread or a client process does. **Always pass a timeout**; the admin's + own forwards use 5 s so an unreachable peer cannot pin an HTTP thread. +- **A handler is not a coroutine.** There is no `co_await` here. +- **Handlers must be reentrant.** Several may run at once. + +The server is stopped early in `RuntimeManager::ServerFinalize`, before the +workers stop, so no handler is left waiting on a task that will never run. + +#### The "Add Pool" convention + +The dashboard's Pools tab can create pools of your module. If you register +nothing, users get the generic compose editor (identity fields plus raw +YAML, driven through the same path as `clio_run compose`). To give them a +typed form instead, register two routes: + +- `GET /api/mod//create` answers a form spec: + `{"title", "note", "fields":[{name,label,type,required,default,options,placeholder,help}]}` + with field types `text`, `select`, `textarea`. +- `POST /api/mod//create` accepts those fields urlencoded. + `action=validate` checks everything and creates nothing (the form's + **Validate** button); otherwise create the pool with your own typed client + and answer `{"ok":true,"pool_id":...}`. Report validation failures as + `{"ok":false,"errors":{field: message}}` at HTTP 400, all fields at once. + +Two rules make the forms work: + +- **Explicit pool ids.** The runtime rejects null pool ids, so prefill + `pool_id` from `viz::SuggestFreePoolMajor(base)` — a node-local suggestion + the user may edit, not a reservation. +- **Never `ConfigParse::ParseSize` on form input.** It `exit(1)`s on garbage. + Use `viz::ParseSizeField(text, &out)`, which rejects instead of dying. + +`bdev`, `safe-bdev`, and the CTE core (`context-runtime/modules/bdev/src/bdev_runtime.cc`, +`context-runtime/modules/safe-bdev/src/safe_bdev_viz.cc`, +`context-transfer-engine/core/src/core_viz.cc`) are complete worked +examples of pages, routes, and create forms; `ctest -R cr_viz_tests` shows +how to drive the server from a test. + ## Configuration and Code Generation ### Overview @@ -3103,6 +3256,30 @@ add_chimod_runtime( ) ``` +#### `clio_add_viz_assets()` Function +Stages and installs a ChiMod's `viz/` web-dashboard pages where the runtime's +`VizServer` looks for them (see [Web Dashboard Integration](#web-dashboard-integration-registerviz)). + +```cmake +clio_add_viz_assets( + MOD_NAME mod_name + [DIR path/to/assets] +) +``` + +**Parameters:** +- **`MOD_NAME`** (required): The ChiMod name as reported by `get_chimod_name()` — the URL (`/viz//`) and directory the assets are served under +- **`DIR`** (optional): Source directory of the assets (default: `${CMAKE_CURRENT_SOURCE_DIR}/viz`) + +**Automatic Behavior:** +- Copies the directory to `/bin/viz/` (a `_viz` target, re-staged when any asset changes) +- Installs it to `/share/clio/viz/` for both the default and `pip_package` components, so packages and wheels ship the pages alongside the library + +**Example:** +```cmake +clio_add_viz_assets(MOD_NAME clio_bdev) +``` + **Function Dependencies:** Both functions automatically handle common dependencies: diff --git a/docs/sdk/context-runtime/20.base-modules/2.bdev.md b/docs/sdk/context-runtime/20.base-modules/2.bdev.md index 118dbeac..a75c5c1c 100644 --- a/docs/sdk/context-runtime/20.base-modules/2.bdev.md +++ b/docs/sdk/context-runtime/20.base-modules/2.bdev.md @@ -11,6 +11,7 @@ The Bdev (Block Device) Module provides a high-performance interface for block d - **Performance monitoring** and statistics collection for both backends - **Memory-aligned I/O operations** for optimal file-based performance - **Block allocation and deallocation management** with unified API +- **Dashboard page**: every bdev pool has a page at `/viz/clio_bdev/` on the runtime's [web dashboard](../../../deployment/monitoring#runtime-dashboard) (capacity meter plus the full stats table), and new bdev pools can be created from the dashboard's **Add Pool** form ## CMake Integration diff --git a/docs/sdk/context-transfer-engine/cte.md b/docs/sdk/context-transfer-engine/cte.md index 65344221..13fdfe6e 100644 --- a/docs/sdk/context-transfer-engine/cte.md +++ b/docs/sdk/context-transfer-engine/cte.md @@ -535,6 +535,17 @@ void example() { } ``` +:::tip From the dashboard +The runtime's [web dashboard](../../deployment/monitoring#runtime-dashboard) +gives every CTE core pool a management page at `/viz/clio_cte_core/`: the +target roster (score, free space, capacity, latency, bandwidth, bytes read/written) with **register** and +**unregister** buttons, backed by +`GET /api/mod/clio_cte_core/{pool}/targets` and +`POST …/register_target` / `…/unregister_target`. `register_target` takes a +fresh `name` + `bdev_type` + `capacity`, or `name` + `attach_pool_id` to +attach an existing pool such as a safe-bdev array. +::: + ### Working with Tags and Blobs #### Using the Core Client Directly diff --git a/sidebars.ts b/sidebars.ts index d2470306..08edcfb0 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -10,6 +10,7 @@ const sidebars: SidebarsConfig = { items: [ 'getting-started/installation', 'getting-started/quick-start', + 'getting-started/dashboard', 'getting-started/developer', ], },