Skip to content

feat: add ASI-03 and ASI-05 challenge library expansion#521

Open
ashike24 wants to merge 8 commits into
GenAI-Security-Project:mainfrom
ashike24:main
Open

feat: add ASI-03 and ASI-05 challenge library expansion#521
ashike24 wants to merge 8 commits into
GenAI-Security-Project:mainfrom
ashike24:main

Conversation

@ashike24

@ashike24 ashike24 commented May 31, 2026

Copy link
Copy Markdown

Challenge Library Expansion

Adds 6 new CTF challenges covering ASI-03 (Identity and Impersonation)
and ASI-05 (Indirect Prompt Injection).

ASI-03 Challenges (3)

  • Ghost in the Machine — vendor impersonates orchestrator via profile fields
  • Puppet Master — cross-agent trust abuse via poisoned invoice summary
  • Silver Tongue — role hijack via multi-turn conversation context manipulation

ASI-05 Challenges (3)

  • Trojan Invoice — indirect injection via poisoned FinDrive document
  • Poisoned Inbox — indirect injection via malicious FinMail email
  • Tool Output Hijack — indirect injection via crafted TaxCalc tool output

Each challenge includes

  • YAML definition with hints, labels, difficulty, and points
  • Working detector Python class registered in the detector registry
  • LLM judge prompt with HIGH/MEDIUM/LOW scoring rubric

Tests added (Week 3-4)

  • 78 unit tests across all 6 new detectors
  • Coverage >95% on all detector files
  • Tests located in tests/unit/ctf/detectors/

@ashike24 ashike24 changed the title feat: add ASI-03 and ASI-05 challenge library expansion (GSoC 2026 Week 1-2) feat: add ASI-03 and ASI-05 challenge library expansion Jun 3, 2026
@mekaizen

mekaizen commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Improvement -

  1. Missing Retry-After Header
    File: finbot/core/ratelimit/limiter.py

Problem:
The API returns HTTP 429 correctly but doesn't include the standard Retry-After header. Many clients automatically respect this header when implementing retries.

Recommendation:
Return Retry-After with the remaining TTL.

Redis Initialization Should Be Explicitly Guarded

File: finbot/core/ratelimit/limiter.py

Problem:
The implementation assumes event_bus.redis is initialized. If Redis initialization fails during startup or is unavailable in certain test environments, every request will raise an AttributeError, which is then caught by the generic exception handler. Although the fail-open behavior works, it results in unnecessary exception logging on every request.

  1. Negative TTL Values Can Produce Confusing Responses

File: finbot/core/ratelimit/limiter.py
Problem:
Redis may return -1 or -2 for TTL. Using these values directly could generate messages such as:

@ashike24

ashike24 commented Jul 4, 2026

Copy link
Copy Markdown
Author

Thanks for the review! I've addressed all three issues in commit 9e334e1:

  1. Retry-After header — added headers={"Retry-After": str(ttl)} to the HTTPException so clients automatically know when to retry.

  2. Redis guard — replaced direct event_bus.redis access with getattr(event_bus, "redis", None) and an explicit None check, eliminating AttributeError logging when Redis is unavailable.

  3. Negative TTL — added a guard for ttl < 0 (Redis returns -1 for no expiry, -2 for missing key) that falls back to window_seconds, preventing confusing negative values in the response message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants