fix(colang): preserve comments above bot say as instructions

[adityasingh2400]

Closes #1322.

In Colang 1.0, the documentation describes comments above a bot say <name> line as natural-language instructions that shape the LLM's generated response. In practice the runtime did attach the comment to the BotIntent event as instructions, but generate_bot_message short-circuited with the predefined sample utterance whenever bot_intent appeared in config.bot_messages, so the comment never reached the model and the bot ignored it.

This change keeps the existing short-circuit for the common case (no comment, predefined utterance used verbatim) but skips it when the BotIntent carries instructions, so the LLM is invoked with the colang history that already includes the # ... instruction. Behavior without a comment is unchanged.

Added regression tests in tests/test_bot_message_instructions.py covering three cases: the comment is attached to the BotIntent, the comment reaches the prompt sent to the LLM, and the predefined-utterance shortcut is preserved when no comment is present.

Summary by CodeRabbit

• Bug Fixes

  • Fixed bot message generation to properly respect instruction directives embedded in bot message definitions. Instructions are now passed to the language model instead of being bypassed when predefined messages are available.

• Tests

  • Added regression tests to verify instruction handling in bot message generation workflows.

[greptile-apps]

Greptile Summary

This PR fixes a long-standing Colang 1.0 bug where a # comment placed above a bot <name> line was supposed to act as a natural-language instruction to the LLM but was silently dropped when a predefined sample utterance existed for that intent. The fix is a minimal one-liner in generate_bot_message that skips the short-circuit path when the BotIntent event carries an instructions field.

• generation.py: Adds a bot_intent_has_instructions guard so that generate_bot_message only uses the predefined utterance when no comment instruction is present; otherwise it falls through to the normal LLM-generation path.
• tests/test_bot_message_instructions.py: Adds three regression tests covering: instruction attachment on the BotIntent event, end-to-end instruction propagation to the LLM prompt, and backward-compat preservation of the predefined-utterance shortcut.

Confidence Score: 4/5

Safe to merge for the common (non-single-call) case; users with single_call mode enabled will not benefit from the fix but will not regress either.

The fix is minimal, correct, and well-tested for the main path. The only gap is that comment instructions remain ineffective when single_call mode is on, because the else-branch short-circuits with the pre-computed single-call message before the normal LLM-generation code runs — but this is a pre-existing limitation, not a regression introduced here.

The generate_bot_message else-branch in generation.py (specifically the single_call early-return block) deserves a follow-up if single_call + comment instructions is an expected use case.

Important Files Changed

Filename	Overview
nemoguardrails/actions/llm/generation.py	Single-line guard added to bypass the predefined-utterance short-circuit when the BotIntent carries instructions; logic is correct for the common path but instructions are still silently ignored when single_call mode is active.
tests/test_bot_message_instructions.py	Three well-structured regression tests covering the new instruction path, end-to-end prompt propagation, and backward-compat shortcut preservation; uses established call.task filtering pattern.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[generate_bot_message called] --> B{BotIntent has instructions?}
    B -- "bool(event.get('instructions')) → True" --> C[bot_intent_has_instructions = True]
    B -- "False / None / empty string" --> D[bot_intent_has_instructions = False]

    C --> E{bot_intent in config.bot_messages AND NOT bot_intent_has_instructions?}
    D --> E

    E -- "True (no instructions, predefined exists)" --> F[Use predefined utterance, skip_output_rails = True]
    E -- "False (has instructions OR no predefined)" --> G{bot_intent starts with '$' and var in context?}

    G -- "True" --> H[Use context variable, Ignores instructions]
    G -- "False" --> I{single_call.enabled?}

    I -- "True" --> J[Use pre-computed single-call message, Instructions still ignored]
    I -- "False" --> K[Call LLM, Instruction in BotIntent event reaches render_task_prompt]

    F --> L[Return bot utterance]
    H --> L
    J --> L
    K --> L

Loading

Comments Outside Diff (1)

1. nemoguardrails/actions/llm/generation.py, line 810-866 (link)

   Instructions silently ignored when single_call mode is active

   When rails.dialog.single_call.enabled is True, the else branch is entered correctly (the new guard works), but the very first thing it does is check whether the pre-computed single-call bot-message matches the current intent and, if so, returns that pre-computed text immediately — before reaching the normal LLM-generation code at the bottom of the else block. The pre-computed message was produced by the single-call LLM prompt, which was assembled before the BotIntent's instructions field existed in the event stream, so those instructions are never incorporated. Users who configure single_call.enabled: true and add comment instructions above bot <name> will see this PR's fix have no effect in their setup.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: nemoguardrails/actions/llm/generation.py
   Line: 810-866
   
   Comment:
   **Instructions silently ignored when `single_call` mode is active**
   
   When `rails.dialog.single_call.enabled` is `True`, the `else` branch is entered correctly (the new guard works), but the very first thing it does is check whether the pre-computed single-call bot-message matches the current intent and, if so, returns that pre-computed text immediately — before reaching the normal LLM-generation code at the bottom of the `else` block. The pre-computed message was produced by the single-call LLM prompt, which was assembled before the BotIntent's `instructions` field existed in the event stream, so those instructions are never incorporated. Users who configure `single_call.enabled: true` and add comment instructions above `bot <name>` will see this PR's fix have no effect in their setup.
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
nemoguardrails/actions/llm/generation.py:810-866
**Instructions silently ignored when `single_call` mode is active**

When `rails.dialog.single_call.enabled` is `True`, the `else` branch is entered correctly (the new guard works), but the very first thing it does is check whether the pre-computed single-call bot-message matches the current intent and, if so, returns that pre-computed text immediately — before reaching the normal LLM-generation code at the bottom of the `else` block. The pre-computed message was produced by the single-call LLM prompt, which was assembled before the BotIntent's `instructions` field existed in the event stream, so those instructions are never incorporated. Users who configure `single_call.enabled: true` and add comment instructions above `bot <name>` will see this PR's fix have no effect in their setup.

_{Reviews (1): Last reviewed commit: "fix(colang): preserve comments above bot..." | Re-trigger Greptile}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cec22393-d479-4922-a154-05f56267a996

📥 Commits

Reviewing files that changed from the base of the PR and between e3715f9 and f040817.

📒 Files selected for processing (2)

• nemoguardrails/actions/llm/generation.py
• tests/test_bot_message_instructions.py

---

📝 Walkthrough

Walkthrough

This PR fixes comment instructions in Colang bot message flows. When a BotIntent carries instructions (from comments), predefined sample utterances are now skipped, allowing the LLM generation path to apply those instructions. The fix adds instruction detection in generate_bot_message and includes comprehensive regression tests covering instruction propagation and backward compatibility.

Changes

Bot Message Instructions

Layer / File(s)	Summary
Instruction detection in message generation nemoguardrails/actions/llm/generation.py	The generate_bot_message method detects instruction directives on incoming BotIntent events. When instructions are present, predefined bot message selection is skipped, allowing the LLM-generation path to use those instructions to guide the response.
Regression tests for bot message instructions tests/test_bot_message_instructions.py	New test module verifies that comments above bot statements become instructions fields in BotIntent events, that instructions propagate to LLM prompts even with predefined utterances available, and that the original behavior is preserved when no instructions exist.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 6

✅ Passed checks (6 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly describes the main change: preserving comments above bot say statements as instructions. It is concise, specific, and directly reflects the primary objective of the changeset.
Linked Issues check	✅ Passed	The PR implementation directly addresses issue #1322 by modifying generate_bot_message to skip the predefined-utterance short-circuit when BotIntent carries instructions, and adds comprehensive regression tests validating that comments become instructions and reach the LLM prompt.
Out of Scope Changes check	✅ Passed	All changes are directly related to the PR objective: the core fix in generation.py handles comment-based instructions, and the new test module validates the complete solution without introducing unrelated modifications.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Test Results For Major Changes	✅ Passed	PR documents regression tests covering comment attachment, LLM prompt delivery, and backward compatibility. Minor code change (+8/-1) in bug fix restoring documented behavior.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!