feat: Allow for multiple self-check rails

[RobGeada]

Description

Adds an optional $input_task and $output_task variable to the self_check rails, that allows for specification of multiple sequential self-check prompts in a configuration, e.g.:

   prompts:
      - task: check_harmful_content
        content: |
          Your task is to check if the user message contains harmful,
          abusive, or inappropriate content.

          User message: "{{ user_input }}"

          Should this message be blocked (Yes or No)?
          Answer:

      - task: check_off_topic
        content: |
          Your task is to check if the user message is off-topic.
          This bot only handles questions about billing and account management.
          General conversation and greetings are allowed.

          User message: "{{ user_input }}"

          Is this message off-topic and should be blocked (Yes or No)?
          Answer:

    rails:
      input:
        flows:
          - self check input $input_task=check_harmful_content
          - self check input $input_task=check_off_topic

This PR also:

1. Provides support for per-task model configuration, identical to the content_safety rail
2. Standardizes the input and output self-check actions into the same function, to allow for easier future expansion.

Related Issue(s)

Addresses #1873

Checklist

• I've read the CONTRIBUTING guidelines.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

Release Notes

• New Features

  • Support for configuring and executing multiple input and output self-check rails with customizable task parameters.

• Documentation

  • Added configuration guidance for self-check rails, including parameter usage, example configurations, and important context management considerations.

• Tests

  • Added comprehensive test coverage for multiple self-check rail configurations and backward compatibility.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR extends self-check rails to support multiple sequential checks with parameterized task selection. Input and output check actions now accept string task parameters with placeholder mapping to defaults; flows pass task parameters from configuration; and tests validate multi-rail execution and backward compatibility.

Changes

Multiple Self-Check Rails Feature

Layer / File(s)	Summary
Input Check Action String Parameter nemoguardrails/library/self_check/input_check/actions.py	self_check_input converts from Task enum to task: str = "self_check_input", removes Task import, maps "$input_task" placeholder to default, and updates LLM metadata and truncation warnings to use string task directly.
Input Check Flows with Task Parameter nemoguardrails/library/self_check/input_check/flows.co, flows.v1.co	self check input flow now declares $input_task parameter and passes it to the action via task=$input_task, enabling task selection per flow invocation.
Output Check Action String Parameter nemoguardrails/library/self_check/output_check/actions.py	self_check_output converts from Task enum to task: str = "self_check_output", removes Task import, maps "$output_task" placeholder to default, and updates LLM metadata and truncation warnings to use string task directly.
Output Check Flows with Task Parameter nemoguardrails/library/self_check/output_check/flows.co, flows.v1.co	self check output flow now declares $output_task parameter and passes it to the action via task=$output_task, enabling task selection per flow invocation.
Multi-Rail Configuration Guide docs/configure-rails/guardrail-catalog/self-check.md	Documents $input_task and $output_task parameters with Colang v1 global-context warnings, provides example prompt configurations, and shows sequential execution with early stopping on blocking checks.
Multi-Rail & Backward Compatibility Tests tests/test_multiple_self_check_rails.py	New test module with three configs (multi-input, multi-output, default-task) covering eight test functions: multi-rail both-pass/first-blocks/second-blocks scenarios for input and output, plus backward-compatibility tests confirming default task names work when placeholders are omitted.

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	Major feature PR lacks documented test results. While 8 tests were added, the PR description contains only a minimal commit message without evidence of test execution or outcomes.	Document test results in the PR description. For example, state that all 8 tests pass, covering multiple input/output rails and backward compatibility.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main feature: enabling multiple self-check rails through parameterization of input and output tasks.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/test_multiple_self_check_rails.py (1)

75-75: ⚡ Quick win

Strengthen the happy-path input test assertion.

test_multiple_input_rails_both_pass should assert the returned content too, not just role, to catch regressions where the wrong assistant output is produced.

✅ Suggested test hardening

 def test_multiple_input_rails_both_pass():
@@
     new_message = rails.generate(messages=[{"role": "user", "content": "hello"}])

     assert new_message["role"] == "assistant"
+    assert new_message["content"] == "Hey!"

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_multiple_self_check_rails.py` at line 75,
test_multiple_input_rails_both_pass currently only asserts new_message["role"];
add a content assertion to lock the happy-path output by comparing
new_message["content"] (or new_message["content"]["parts"][0] if your message
payload uses parts) to an expected string defined in the test (e.g.,
expected_response) and keep the existing role check; update the test function
test_multiple_input_rails_both_pass to include that additional assertion so
regressions in assistant output are caught.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/test_multiple_self_check_rails.py`:
- Line 75: test_multiple_input_rails_both_pass currently only asserts
new_message["role"]; add a content assertion to lock the happy-path output by
comparing new_message["content"] (or new_message["content"]["parts"][0] if your
message payload uses parts) to an expected string defined in the test (e.g.,
expected_response) and keep the existing role check; update the test function
test_multiple_input_rails_both_pass to include that additional assertion so
regressions in assistant output are caught.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 62074a50-d213-4cbd-acf8-fe8fc813fd7d

📥 Commits

Reviewing files that changed from the base of the PR and between 0952638 and 7117521.

📒 Files selected for processing (8)

• docs/configure-rails/guardrail-catalog/self-check.md
• nemoguardrails/library/self_check/input_check/actions.py
• nemoguardrails/library/self_check/input_check/flows.co
• nemoguardrails/library/self_check/input_check/flows.v1.co
• nemoguardrails/library/self_check/output_check/actions.py
• nemoguardrails/library/self_check/output_check/flows.co
• nemoguardrails/library/self_check/output_check/flows.v1.co
• tests/test_multiple_self_check_rails.py

[greptile-apps]

Greptile Summary

This PR consolidates the input_check and output_check self-check modules into a unified message_check module, adds support for multiple sequential self-check rails via $input_task / $output_task flow parameters, and introduces per-task LLM configuration mirroring the content_safety rail pattern.

• The core shared self_check() helper and both self_check_input / self_check_output actions have been merged into message_check/actions.py, replacing the two now-deleted modules; Colang v2 and v1 flow definitions have been updated to pass the task parameter through to the action.
• A three-level LLM fallback chain is added via _get_llm: task-specific model → default task model (self_check_input / self_check_output) → main LLM injected as llm action param; missing-model cases raise ValueError.
• Comprehensive test coverage is added for multiple-rail sequencing, early abort on block, per-task LLM dispatch, fallback chain ordering, and backward compatibility with the default single-rail configuration.

Confidence Score: 5/5

Safe to merge — consolidation is well-scoped, backward compatibility is preserved, and llms is already registered by _init_llms so the new required parameter is always injected.

The structural change merging two action modules into one is clean: Colang v2 flows add explicit defaults, v1 flows add a null-guard, and the three-level LLM fallback chain correctly uses the already-registered llms dict with the main llm action param as the final fallback. No runtime paths are broken.

nemoguardrails/library/self_check/message_check/actions.py — the misleading default task in the internal self_check helper and the missing space in the error message are minor but worth tidying before merge.

Important Files Changed

Filename	Overview
nemoguardrails/library/self_check/message_check/actions.py	New unified self_check module; minor issues: wrong default task in internal self_check() and missing space in ValueError message.
nemoguardrails/library/self_check/message_check/flows.co	Colang v2 flows for both input and output rails with correct default parameter values; symmetric abort behavior.
nemoguardrails/library/self_check/message_check/flows.v1.co	Colang v1 flows correctly handle unset context variables via explicit null-guard before using default task name.
tests/test_multiple_self_check_rails.py	Thorough test coverage: sequential blocking, early-abort, per-task LLM dispatch, fallback chain ordering, and backward-compat default task.
docs/configure-rails/guardrail-catalog/self-check.md	Documentation added for multiple input/output rail configuration with Colang v1 global-variable warning and example configs.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User Message] --> B[self check input flow 1\n$input_task=check_harmful]
    B --> C{SelfCheckInputAction\ntask=check_harmful}
    C -->|allowed=True| D[self check input flow 2\n$input_task=check_off_topic]
    C -->|allowed=False| E{enable_rails_exceptions?}
    E -->|Yes| F[send InputRailException]
    E -->|No| G[bot refuse to respond]
    F --> H[abort]
    G --> H
    D --> I{SelfCheckInputAction\ntask=check_off_topic}
    I -->|allowed=True| J[Continue to LLM]
    I -->|allowed=False| E2{enable_rails_exceptions?}
    E2 -->|Yes| F2[send InputRailException]
    E2 -->|No| G2[bot refuse to respond]
    F2 --> H2[abort]
    G2 --> H2
    subgraph _get_llm fallback
        K[task in llms?] -->|Yes| L[use task model]
        K -->|No| M[default_task in llms?]
        M -->|Yes| N[use default_task model]
        M -->|No| O[default_llm provided?]
        O -->|Yes| P[use main llm]
        O -->|No| Q[raise ValueError]
    end

Loading

Prompt To Fix All With AI

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
nemoguardrails/library/self_check/message_check/actions.py:56-65
The internal `self_check` helper defaults `task` to `DEFAULT_OUTPUT_TASK` even though it handles both input and output paths. This default is never reached in practice (both public callers always forward their own `task` argument), but if `self_check` were ever invoked directly without an explicit `task` in a `USER_MESSAGE` context, the wrong default would be used to look up the prompt and LLM, silently producing incorrect behaviour.

```suggestion
async def self_check(
    llms: Dict[str, LLMModel],
    message_type: str,
    llm_task_manager: LLMTaskManager,
    default_llm: Optional[LLMModel] = None,
    context: Optional[dict] = None,
    config: Optional[RailsConfig] = None,
    task: Optional[str] = None,
    **kwargs,
):
```

### Issue 2 of 2
nemoguardrails/library/self_check/message_check/actions.py:49-52
Two adjacent f-string literals are concatenated without a separating space, so the rendered message reads `"…found.Please configure…"` with no space after the period.

```suggestion
        error_msg = (
            f"No matching model for task={task} found. "
            f"Please configure a model with type={task}, type={default_task}, or type=main"
        )
```

_{Reviews (6): Last reviewed commit: "Overhaul self-check flows: unify action ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 94.66667% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...drails/library/self_check/message_check/actions.py	94.66%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[github-actions]

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1874