Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions data/entries/ASI06.json
Original file line number Diff line number Diff line change
Expand Up @@ -1086,6 +1086,16 @@
"name": "LAAF v2.0",
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
"type": "open-source"
},
{
"name": "OWASP Agent Memory Guard",
"url": "https://github.com/OWASP/www-project-agent-memory-guard",
"type": "open-source"
},
{
"name": "AgentThreatBench (Inspect Evals)",
"url": "https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/agent_threat_bench",
"type": "open-source"
}
],
"incidents": [
Expand Down
10 changes: 10 additions & 0 deletions data/tools-supplement.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,16 @@
"name": "LAAF v2.0",
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
"type": "open-source"
},
{
"name": "OWASP Agent Memory Guard",
"url": "https://github.com/OWASP/www-project-agent-memory-guard",
"type": "open-source"
},
{
"name": "AgentThreatBench (Inspect Evals)",
"url": "https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/agent_threat_bench",
"type": "open-source"
}
],
"ASI07": [
Expand Down
10 changes: 10 additions & 0 deletions docs/data.js
Original file line number Diff line number Diff line change
Expand Up @@ -20452,6 +20452,16 @@ window.CROSSWALK_DATA = [
"name": "LAAF v2.0",
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
"type": "open-source"
},
{
"name": "OWASP Agent Memory Guard",
"url": "https://github.com/OWASP/www-project-agent-memory-guard",
"type": "open-source"
},
{
"name": "AgentThreatBench (Inspect Evals)",
"url": "https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/agent_threat_bench",
"type": "open-source"
}
],
"incidents": [
Expand Down
2 changes: 2 additions & 0 deletions evals/EXTERNAL_BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ as a profile in this repository.**
| **AgentLSD** | **Adversarial task contamination** of AI security agents — deceptive artifacts in the environment, including non-instructional evidence "such as fake results and decoy endpoints", rather than injected instructions alone | Paired clean and trap-augmented runs on CTF challenges with deterministic trap generation; 6 models on 11 web CTF challenges | LLM01 Prompt Injection · ASI01 Agent Goal Hijack | [arXiv:2609.19140](https://arxiv.org/abs/2609.19140) · release: [Golim/agent-lsd](https://github.com/Golim/agent-lsd) (MIT) |
| **AgentXploit-Bench** | **End-to-end exploitability of AI-agent systems** in authorized white-box pre-deployment auditing: attacks must act through the task-defined attacker interface and be confirmed by an external verifier | 72 reproducible vulnerabilities across 12 open-source AI-agent systems and frameworks | ASI02 Tool Misuse and Exploitation · ASI05 Unexpected Code Execution · LLM01 Prompt Injection | [arXiv:2609.31318](https://arxiv.org/abs/2609.31318) · release: [lwd17/AgentXploit](https://github.com/lwd17/AgentXploit) (README states Apache-2.0; no licence file in the repository) |
| **PrivDrift** | **User-secret leakage under topic drift**: whether secrets a user disclosed earlier in an active conversation remain recoverable after the dialogue moves on, under persuasion-based probing | 1,000 controlled multi-turn dialogues with seeded secrets, content-dense drift turns and standardized extraction probes; 3 LLMs with extended context windows | LLM02 Sensitive Information Disclosure · DSGAI11 Cross-Context Conversation Bleed | [arXiv:2609.30094](https://arxiv.org/abs/2609.30094) · not released: the paper states the authors "plan to release" the generation code, probes, scripts and a sanitized subset |
| **AgentThreatBench** | Whether **LLM agents complete useful work while resisting adversarial content** in tool outputs and memory stores, scored separately for utility and security | 3 Inspect AI tasks: memory poisoning (10 samples), email-triage goal hijack (6 samples), customer-support data exfiltration (8 samples); each includes clean baselines | ASI06 Memory and Context Poisoning · ASI01 Agent Goal Hijack | release: [UKGovernmentBEIS/inspect_evals, agent_threat_bench](https://github.com/UKGovernmentBEIS/inspect_evals/tree/main/src/inspect_evals/agent_threat_bench) (MIT). Contributed by the PR author, who also co-leads OWASP Agent Memory Guard. |

## Why these and not others

Expand All @@ -56,3 +57,4 @@ Routing — incident, catalogue, or noted and closed — is defined once in [`..
|---|---|
| 2026-09-18 | Created with LongPIBench, GenIaC-SecBench and TIER, from the watcher triage of issues #47, #55 and #70. |
| 2026-09-30 | Added APort Vault, ClashBench, AgentLSD, AgentXploit-Bench and PrivDrift, from the watcher triage of issues #125, #137, #144, #160 and #171; added the release and licence note. |
| 2026-10-28 | Added AgentThreatBench, from PR #321. |