Skip to content

test(recall): cover multi-turn injection growth - #467

Closed
March-77 wants to merge 18 commits into
TencentCloud:mainfrom
March-77:test/120-multiturn-recall-growth
Closed

March-77 wants to merge 18 commits into
TencentCloud:mainfrom
March-77:test/120-multiturn-recall-growth

Conversation

@March-77

@March-77 March-77 commented Jul 11, 2026

Copy link
Copy Markdown

Description | 描述

Adds focused multi-turn regression coverage for the canonical prompt-cache mitigation in #375.

The existing tests validate a single injected message. This follow-up measures the long-session behavior that #120 originally reported:

  • simulates 100 turns with approximately 1,000 dynamic recall characters per turn
  • verifies recall.showInjected=false removes every recall block before persistence
  • verifies persisted question text remains unchanged
  • verifies cumulative persisted history is less than 5% of the showInjected=true history in the benchmark fixture
  • verifies at least 100,000 injected characters are removed across 100 turns
  • covers multipart messages across 50 turns and proves image parts are preserved
  • keeps explicit showInjected=true behavior covered as the opt-in diagnostic path

This PR adds focused regression tests and an acceptance-oriented architecture/measurement note; it does not introduce another runtime injection strategy.

Related Issue | 关联 Issue

Related to #120
Follow-up to / depends on #375

Change Type | 修改类型

  • Bug fix | Bug 修复
  • New feature | 新功能
  • Documentation update | 文档更新
  • Code optimization | 代码优化
  • Test coverage | 测试补充

Self-test Checklist | 自测清单

  • Verified locally | 本地验证通过
  • No existing features affected | 无影响现有功能

Verification:

  • npm.cmd exec vitest run src/adapters/openclaw/recall-injection.test.ts src/adapters/openclaw/recall-injection.multiturn.test.ts: 8 passed
  • npm.cmd test: 78 passed
  • npm.cmd run build:plugin: passed
  • git diff --check: passed

Additional Notes | 其他说明

The runtime mitigation remains the canonical implementation in #375 (prepend by default, opt-in append, and showInjected=false). This follow-up now documents the before/after prompt structure, the R*N*(N-1)/2 aggregate replay-growth model, the deterministic 100-turn evidence, and a controlled DeepSeek/MiMo A/B protocol in docs/prompt-cache-mitigation.md. After #375 merges, this branch should be rebased onto main, retaining the regression and documentation commits. The benchmark is a deterministic persisted-history growth proxy, not a fabricated provider-side cache hit claim.

@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for submitting this PR and participating in Tencent Rhino-bird Open-source Training Program!
We have successfully received your submission. The program is currently in full swing, and we will complete the Code Review for you as soon as possible. Please keep an eye on the status notifications for this PR so you can follow up promptly once the review feedback is provided.
Thanks again for your contribution and open-source spirit! 🚀

@March-77
March-77 marked this pull request as ready for review July 14, 2026 16:16
PorunC and others added 14 commits July 17, 2026 11:07
…tCloud#151)

* feat(hermes): support Windows native setup

Closes TencentCloud#113

Signed-off-by: misaka <09982.misaka@gmail.com>

* test(postinstall): cover cross-platform behavior

* Delete __tests__/postinstall.test.ts

* Delete hermes-plugin/memory/memory_tencentdb/tests/test_memory_tencentdb_recovery.py

---------

Signed-off-by: misaka <09982.misaka@gmail.com>
Co-authored-by: 十五便士 <95488710+Maxwell-Code07@users.noreply.github.com>
Add unit tests for memory-tencentdb provider self-healing functionality, including watchdog behavior and request path recovery.
* Update README.md

* Update README_CN.md
* docs: add Star History chart to README

* docs: replace Star History chart with GitHub badges

* docs: use real Star History chart image in README
…solves TencentCloud#160) (TencentCloud#529)

* fix(store): sanitize FTS5 query tokens to prevent MATCH injection#160

问题
buildFtsQuery() 把用户输入拼成 FTS5 MATCH 表达式,含 FTS5 操作符(`" ' * ( ) AND OR NOT NEAR : ^`)的输入会篡改查询语义或触发语法错误(注入)。

方案
- `sanitizeFtsToken()`:双引号包字面 phrase,内部 `"` 转义为 `""`(修正旧实现"删除双引号"导致的 recall 损伤)
- `sanitizeFtsWhitelist()`:字符级白名单(defence-in-depth)
- `buildFtsQuery` 复用 `sanitizeFtsToken`
- 与项目已有的 `MATCH ?` 参数化(SQL 防注入)组成三层防御

测试
新增 38 个:单元(21)+ 真实 FTS5 召回对比(4)+ 白名单(13)。本地全量 **105 passed**,零回归。

 验收对照
issue 四档全覆盖:基础 / 进阶 / 深入(真实 FTS5 recall 实证)/ 拓展(白名单 + 参数化三层防御)。

* Delete src/core/store/buildFtsQuery.recall.test.ts

* Delete src/core/store/buildFtsQuery.test.ts

* Delete src/core/store/buildFtsQuery.whitelist.test.ts

---------

Co-authored-by: 十五便士 <95488710+Maxwell-Code07@users.noreply.github.com>
…ounters after cleanup (TencentCloud#337)

* fix(data): add CheckpointManager.recalculate() to reconcile drifted counters after cleanup

After cleanup operations (memory-cleaner, manual JSONL pruning, or
pipeline state deletion), checkpoint counters (l0_conversations_count,
total_memories_extracted) can drift from actual data — they only
increment, never decrease.

Add recalculate() to CheckpointManager that accepts actual counts and
overwrites drifted counters under the file lock. Call it once at startup
in wirePipelineRunners() after vector store is initialized.

Closes TencentCloud#157

Signed-off-by: xiyue1753 <xiyuekl@qq.com>

* Delete src/utils/checkpoint.test.ts

---------

Signed-off-by: xiyue1753 <xiyuekl@qq.com>
Co-authored-by: 十五便士 <95488710+Maxwell-Code07@users.noreply.github.com>
@Maxwell-Code07

Copy link
Copy Markdown
Collaborator

Thank you for your hard work on issue [#120] — the solution is well-designed and the code quality is excellent 👏

That said, from a project-stability perspective, we're taking a cautious approach toward "cache hit-rate optimization" changes. This is an experience-level optimization rather than a functional bug or security issue, so to keep the change surface minimal and reduce regression risk, we've decided not to introduce a large-scale implementation at this time and will close the related PRs.

Your ideas and technical approach remain highly valuable, and we truly appreciate the effort you've put in. We welcome you to continue contributing to the Rhino Bird open-source program and look forward to seeing you shine on a more suitable topic 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants