Best Practices for Learning with a RAG-based Chat Agent? #41

Based on the Integration Guide section: Chat-Based Agents

Key Considerations
- Learn from complete conversation (not individual turns)
- Inject playbook context at conversation start
- Allow manual feedback at conversation end

Which recommends learning from complete conversation (not individual turns)

Question:

What's the best practices for the learning phase, specifically for the post-deployment stage.

• Pre-deplopyment
  My understanding is that we can use OfflineAdapter/ACELiteLLM.learn() on each complete conversation history (multi-turns).

• Post-deployment
  Should I still follow the guide and provide feedback only at the end of the complete conversation? Or is it better to send feedback after each turn or N turns to ACE? What are the trade-offs and the recommended approach here?

During RAG process, The search context is updated in every turn based on the user’s latest query and we wrap the latest playbook context into RAG-based Chat Agent's system prompt that causing the agent’s response to change continuously throughout the conversation.