Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,16 @@ lmnr datasets pull <id> # Pull dataset

- All entries in the `[dependency-groups].dev` section of `pyproject.toml` MUST be pinned to a specific version with `==X.Y.Z`. Do NOT use unbounded specifiers (`>=`, `^`, `~`, `<`, ranges, or bare package names). Pinning keeps the test matrix deterministic across developers and CI. When adding a new dev dep, look up the current release on https://pypi.org and pin to that exact version; bumps then go through a normal PR.

## examples/hermes-plugin

- Standalone pip package that bridges [Hermes Agent](https://github.com/nousresearch/hermes-agent) plugin hooks to Laminar spans. Ships via the `hermes_agent.plugins` entry-point group (`lmnr-hermes = "lmnr_hermes:register"`), so `pip install -e examples/hermes-plugin` plus `hermes plugins enable lmnr-hermes` is enough — no pip publish needed.
- The `examples/hermes-plugin` directory is a uv workspace member; adding a new example dir with `tool.uv.sources.lmnr = { workspace = true }` requires updating `[tool.uv.workspace].members` in the root `pyproject.toml` or uv errors with "not a workspace member".
- Hermes calls hooks on different threads (ThreadPoolExecutor for concurrent tool calls; delegate/subagent workers). The plugin keeps a `session_id → turn span` map and parents tool spans via `parent_span_context=Laminar.get_laminar_span_context_dict(turn_span)` rather than relying on the OTel current context, which is thread-local.
- Laminar writes session_id as the attribute `lmnr.association.properties.session_id` (prefix = `ASSOCIATION_PROPERTIES` constant). Tests asserting session scoping must check this exact key, not `session_id` or `SESSION_ID`.
- Hermes drives its own HTTP client for provider calls (see `run_agent.py` ~line 11880 using `_get_transport().normalize_response`), NOT the `anthropic` / `openai` Python SDKs. The raw-SDK instrumentors Laminar auto-enables therefore never see these calls. The plugin must create LLM spans itself in the `pre_api_request` hook (open) and `post_api_request` hook (close + usage), using `span_type="LLM"` and the GenAI semconv attributes (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.cache_read_input_tokens`, `gen_ai.usage.cache_creation_input_tokens`). Without these, traces show tool spans but no model spans, and the Laminar UI reports `$0` cost.
- The `post_api_request` hook's `usage` payload is the dict form of Hermes's `CanonicalUsage` dataclass (`agent/usage_pricing.py`): fields are `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `reasoning_tokens`, `total_tokens`. Map `cache_read_tokens` → `gen_ai.usage.cache_read_input_tokens` and `cache_write_tokens` → `gen_ai.usage.cache_creation_input_tokens` so Laminar's cost computation matches Anthropic's pricing tiers.
- Laminar's SDK strips the port from `LMNR_BASE_URL` ("Ignoring port in base URL: 8000"). To point the plugin at a local dev instance, pass `http_port` / `grpc_port` to `Laminar.initialize()`. The plugin reads `LMNR_HTTP_PORT` and `LMNR_GRPC_PORT` env vars for this. Local app-server gRPC is `8001`, not the SDK's default `8443`.

## Environment Variables

```
Expand Down
52 changes: 52 additions & 0 deletions examples/hermes-plugin/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# lmnr-hermes

Laminar tracing plugin for [Hermes Agent](https://github.com/nousresearch/hermes-agent).

Emits nested OpenTelemetry spans for every Hermes conversation turn, tool call, and
subagent delegation — and lets Laminar's raw-provider instrumentors (OpenAI,
Anthropic, Bedrock) nest their own GenAI spans under each turn for free.

## What gets traced

Per turn (one Hermes `run_conversation` call):

- `hermes.turn` (root span) — input: user message + model/platform; output:
assistant response. Session and user IDs are attached.
- `tool.<name>` (child, `span_type=TOOL`) — one per tool call, with args,
result, and `duration_ms`.
- `subagent.<role>` (child) — one per finished subagent delegation.
- OpenAI / Anthropic / Bedrock spans — nested under the turn automatically,
with token usage and messages.

Span attributes include `hermes.model`, `hermes.provider`, `hermes.api_mode`,
`hermes.finish_reason`, `hermes.usage.*`, and `hermes.tool_duration_ms`.

## Install (local dev, no pip publish)

```bash
# 1. Editable install — exposes the hermes_agent.plugins entry point
pip install -e path/to/lmnr-python/examples/hermes-plugin

# 2. Enable in Hermes
hermes plugins enable lmnr-hermes

# 3. Set your Laminar project key and run
export LMNR_PROJECT_API_KEY=...
hermes run
```

Alternative: drop the `src/lmnr_hermes/` directory as `~/.hermes/plugins/lmnr-hermes/`
and Hermes will discover it on startup (the `plugin.yaml` manifest ships inside
that directory). Then `hermes plugins enable lmnr-hermes`.

## Configuration

The plugin reads standard Laminar environment variables:

| Variable | Purpose |
|------------------------|---------------------------------------------------|
| `LMNR_PROJECT_API_KEY` | Your project key (required unless OTel env is set) |
| `LMNR_BASE_URL` | Override the Laminar endpoint (default: api.lmnr.ai) |

If the key is missing, the plugin no-ops silently — Hermes keeps working
without tracing.
29 changes: 29 additions & 0 deletions examples/hermes-plugin/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
[project]
name = "lmnr-hermes"
version = "0.1.0"
description = "Laminar tracing plugin for Hermes Agent (nousresearch/hermes-agent)"
readme = "README.md"
requires-python = ">=3.10,<4"
license = "Apache-2.0"
authors = [
{ name = "lmnr.ai", email = "founders@lmnr.ai" },
]
dependencies = [
"lmnr>=0.7.0",
]

[project.entry-points."hermes_agent.plugins"]
# Hermes's entry-point loader calls `ep.load()` then looks for a `register`
# attribute on the result, so the entry must resolve to the *module*, not the
# register function itself.
lmnr-hermes = "lmnr_hermes"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
packages = ["src/lmnr_hermes"]

[tool.uv.sources]
lmnr = { workspace = true }
Loading