feat: [Anchor Engine] Add real-time intent preservation integration

[HasRahm]

This PR adds real-time drift monitoring utilizing the @anchor-engine/sdk, providing high-fidelity grounding scores (HHEM-2.1). It includes a +9.2% High-Fidelity Lift benchmark result on CNN/DailyMail.

Summary by CodeRabbit

Release Notes

• New Features
  • Introduced a response validation mechanism in the hallucination prevention library that verifies outputs against provided context information
  • Features configurable threshold settings enabling flexible sensitivity control for different use cases
  • Includes comprehensive error handling with safe defaults if validation services are unavailable
  • Now available as a reusable component that can be easily integrated into applications

[greptile-apps]

Greptile Summary

This PR adds nemoguardrails/library/hallucination/anchor/, a new output rail that calls a configurable on-premise Anchor Engine endpoint to score bot responses for hallucination/drift. The iteration addresses all major concerns from the prior review round: requests is replaced with httpx.AsyncClient, the hardcoded third-party URL is replaced by ANCHOR_BASE_URL (defaulting to http://localhost:3000), the None-context crash is guarded, the SPDX header is present, and both flows.co and flows.v1.co are now included.

Confidence Score: 4/5

Safe to merge once the two P2 items are considered; no blocking P0/P1 issues remain after this revision.

All P0/P1 findings from the previous review round have been addressed. Two P2 items remain: undocumented $anchor_threshold variable with an implicit Python-side default, and relevant_chunks passed to the API without type normalisation. Neither is a runtime blocker in typical usage but both can cause silent misbehaviour in certain configurations.

nemoguardrails/library/hallucination/anchor/actions.py (relevant_chunks type handling) and nemoguardrails/library/hallucination/anchor/flows.co ($anchor_threshold default)

Important Files Changed

Filename	Overview
nemoguardrails/library/hallucination/anchor/actions.py	New async action using httpx; previous blocking requests/third-party URL issues resolved; minor: relevant_chunks type not normalised before JSON serialisation
nemoguardrails/library/hallucination/anchor/flows.co	Colang 2.0 flow wiring added; mirrors self check hallucination pattern correctly; $anchor_threshold has no default/documentation
nemoguardrails/library/hallucination/anchor/flows.v1.co	Colang 1.0 compatibility flow; mirrors v2 flow structure correctly
nemoguardrails/library/hallucination/anchor/init.py	Standard module init with SPDX header; correctly exports check_anchor_drift
tests/test_anchor_check.py	Good coverage of key paths: missing key, empty context, allow/block responses, non-200, and exception handling
nemoguardrails/utils.py	Minor reformatting of a lambda expression for PEP8 line-length compliance; no functional change
tests/guardrails/test_iorails_streaming.py	Updated noqa comment from bare string to F704 code; no functional change

Sequence Diagram

sequenceDiagram
    participant Colang as Colang Runtime
    participant Flow as anchor check hallucination (flows.co)
    participant Action as check_anchor_drift (actions.py)
    participant Env as Environment
    participant API as Anchor Engine API (ANCHOR_BASE_URL)

    Colang->>Flow: output rail triggered ($check_hallucination==True)
    Flow->>Action: await check_anchor_drift(threshold=$anchor_threshold)
    Action->>Env: os.environ.get("ANCHOR_API_KEY")
    alt API key missing
        Action-->>Flow: raise ValueError
    end
    Action->>Env: os.environ.get("ANCHOR_BASE_URL", "http://localhost:3000")
    Action->>Action: extract last_bot_message, relevant_chunks from context
    alt message or context empty
        Action-->>Flow: return True (pass-through)
    end
    Action->>API: POST {base_url}/api/score Bearer token + payload
    alt HTTP 200
        API-->>Action: {allow: true/false}
        Action-->>Flow: return bool(allow)
    else non-200 or exception
        Action-->>Flow: return True (fail-open) + log warning
    end
    alt is_safe == False
        Flow->>Colang: SelfCheckHallucinationRailException / bot inform answer unknown + abort
    else is_safe == True
        Flow->>Colang: continue normally
    end

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: nemoguardrails/library/hallucination/anchor/flows.co
Line: 3-4

Comment:
**`$anchor_threshold` undefined raises instead of defaulting**

The flow invokes `check_anchor_drift(threshold=$anchor_threshold)` but never initialises `$anchor_threshold`. If a caller hasn't set this variable, Colang will pass `None` (or raise a runtime error depending on the runtime version). The Python side does coerce `None` gracefully via `try: float(threshold) except TypeError → 0.95`, so it silently works in practice — but the intent is opaque and the default threshold (0.95) is invisible to the person writing the config.

Adding a default assignment at the top of the flow or documenting the required variable in a docstring would make this explicit.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: nemoguardrails/library/hallucination/anchor/actions.py
Line: 56

Comment:
**`relevant_chunks` type not normalised before JSON serialisation**

`source_context` is passed directly as the `"context"` field in the JSON payload. In many RAG pipelines `relevant_chunks` is stored as a list of strings rather than a single string. A list value serialises to a JSON array, which the API endpoint may not handle. Adding an explicit conversion ensures consistent behaviour:

```suggestion
    source_context = context.get("relevant_chunks", "")
    if isinstance(source_context, list):
        source_context = "\n".join(source_context)
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (10): Last reviewed commit: "fix(pipeline): revert to raw colang assi..." | Re-trigger Greptile}

[coderabbitai]

📝 Walkthrough

Walkthrough

A new async action check_anchor_drift is introduced to detect hallucination drift by validating bot messages against an external anchor API. The action reads API credentials from environment variables, extracts context, makes an HTTP POST request, and returns a boolean indicating whether content passes drift detection.

Changes

Cohort / File(s)	Summary
Anchor drift detection action nemoguardrails/library/hallucination/anchor/__init__.py, nemoguardrails/library/hallucination/anchor/actions.py	Adds new async check_anchor_drift action that integrates with an external anchor API for detecting hallucination drift. Validates ANCHOR_API_KEY from environment, extracts message and context from optional parameters, and returns a drift decision based on API response. Handles missing context and request failures gracefully.

Sequence Diagram

sequenceDiagram
    participant Guardrails as Guardrails<br/>System
    participant Action as check_anchor_drift<br/>Action
    participant Env as Environment
    participant API as Anchor API<br/>Endpoint
    
    Guardrails->>Action: check_anchor_drift(context, threshold)
    Action->>Env: read ANCHOR_API_KEY
    Env-->>Action: API key
    Action->>Action: extract last_bot_message,<br/>relevant_chunks
    alt Missing Context
        Action-->>Guardrails: return True
    else Context Available
        Action->>API: POST /api/score<br/>(message, context, threshold)
        API-->>Action: JSON response
        Action->>Action: parse response.allow
        Action-->>Guardrails: return boolean
    end

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Title check	⚠️ Warning	The title mentions 'real-time intent preservation integration' but the actual changes only add a drift-checking action that validates bot message relevance, not intent preservation.	Revise the title to accurately reflect the changes, such as 'feat: [Anchor Engine] Add real-time drift monitoring for bot responses' or similar.
Test Results For Major Changes	⚠️ Warning	PR introduces major feature (Anchor Engine integration) but only mentions '+9.2% High-Fidelity Lift' without baseline numbers, configuration, methodology, or testing context required by check instructions.	Add before-and-after baseline numbers, detailed configuration, testing methodology, context for CNN/DailyMail benchmark, and reference to validation testing to demonstrate thorough evaluation of this major change.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@nemoguardrails/library/hallucination/anchor/actions.py`:
- Around line 23-24: The code assumes context is a dict and calls
context.get(...) which raises AttributeError if context is None; update the
access in the anchor actions to handle a missing context by using a safe
fallback (e.g., replace context.get(...) with (context or {}).get(...) or
explicitly set context = context or {} at the start of the function), ensuring
variables like last_bot_message and source_context are retrieved from a non-None
mapping; locate these accesses in actions.py around the last_bot_message and
source_context assignments and apply the safe fallback consistently.
- Around line 29-50: The async action currently calls the blocking
requests.post() directly and catches all exceptions; change the code to offload
the blocking HTTP call to a thread using asyncio.to_thread when invoking
requests.post (and response.json()), restrict exception handling to
requests.RequestException, ValueError, and TypeError, and log the caught
exception before returning the safe default; locate the async function that
calls requests.post, the requests.post invocation and the response.json() usage,
and replace direct calls with asyncio.to_thread wrappers while adding logging
for errors and narrowing the except clause to those specific exception types,
then return the same default (True) on handled errors.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: fb30f843-5adc-4d22-98d1-bc1d273dfbeb

📥 Commits

Reviewing files that changed from the base of the PR and between 6a7be49 and 1c1b047.

📒 Files selected for processing (2)

• nemoguardrails/library/hallucination/anchor/__init__.py
• nemoguardrails/library/hallucination/anchor/actions.py

[codecov]

Codecov Report

❌ Patch coverage is 94.11765% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...guardrails/library/hallucination/anchor/actions.py	93.75%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!