fix: prevent LLM observation hallucination by properly attributing tool results

[devin-ai-integration]

fix: prevent LLM observation hallucination by properly attributing tool results

Summary

Fixes #4181

When agents use tools, the tool's observation was being appended to the assistant message in conversation history. This caused the LLM to learn that it should generate "Observation:" content itself, leading to hallucinated tool outputs during flow streaming.

The fix: Separate the LLM's actual response from the tool observation in message history:

• LLM's response (Thought/Action/Action Input) → stored as assistant message
• Tool observation → stored as user message

Changes:

• Added llm_response field to AgentAction to preserve the original LLM response
• Modified handle_agent_action_core to store llm_response before appending observation
• Updated all three executors (CrewAgentExecutor, LiteAgent, CrewAgentExecutorFlow) to use proper message attribution
• Fixed add_image_tool special case in both CrewAgentExecutor and CrewAgentExecutorFlow

Review & Testing Checklist for Human

This is a behavioral change to how conversation history is structured. Please verify:

• Message history structure: After a tool call, verify the message list contains an assistant message with the action (no observation), followed by a user message with the observation
• Flow streaming: Test with flow streaming to confirm observations are no longer hallucinated by the LLM
• add_image_tool behavior: The special case for add_image_tool was modified - verify image tools still work correctly
• LiteAgent and CrewAgentExecutorFlow: These executors received the same fix but have no direct tests - consider manual testing
• Existing workflows: Verify existing agent workflows that use tools still function correctly

Recommended test plan:

1. Create a simple crew with an agent that uses a tool
2. Run with verbose logging and inspect the message history
3. Confirm observations appear as user messages, not assistant messages
4. Test with flow streaming if that's your use case

Notes

• The text field on AgentAction still contains the observation (for logging/tracing purposes), but llm_response contains the clean LLM output
• Unit tests pass locally but CI will provide full coverage

Link to Devin run: https://app.devin.ai/sessions/344d8b0e09a0493981fc25eeb2285771
Requested by: João

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[devin-ai-integration]

Closing due to inactivity for more than 7 days. Configure here.