Feat(simulator): complete attack simulator infrastructure for Phase 2 (Weeks 3-6) covering all ASI categories

[Jean-Regis-M]

GSoC 2026 - Phase 2 Contributions (Weeks 3-6)

Description

This contribution completes Phase 2 (Attack Simulator) of the GSoC project, covering Weeks 3 through 6. It includes the sandbox harness, adversarial MCP mocks, attack recipes for all ASI categories (ASI01-ASI10), mutation testing framework, and comprehensive unit tests.

---

📅 Week 3 Contributions (Attack Simulator Foundation)

Author: Jean Francois Regis MUKIZA
GSoC Week: 3 | Phase 2: Attack Simulator (Weeks 3-6)

Core Components Implemented:

1. Sandbox harness with subprocess isolation (finbot/aegis/simulator/base.py):

   • Central class for running attack simulations in isolated subprocesses
   • Manages lifecycle of mock MCP servers and recipe execution
   • Provides secure execution environment for attack scenarios

2. Package initialization (finbot/aegis/simulator/__init__.py):

   • Exposes public interface of the simulator module

3. Comprehensive unit test suite (tests/unit/aegis/test_simulator_harness.py):

   • Tests for sandbox initialization and isolation
   • Validation of subprocess creation and cleanup
   • Verification of attack recipe execution within the sandbox

Technical Implementation:

• The SandboxHarness class uses subprocess.Popen to isolate attack simulations
• Recipes are loaded and executed in a controlled environment
• Error handling and timeout mechanisms prevent test harness from hanging
• Implementation follows existing FinBot codebase patterns

Integration Readiness:

• Designed as the foundation for all subsequent attack simulation components
• Used by adversarial MCP servers and recipe executor in later weeks

---

📅 Week 4 Contributions (Attack Simulator Foundation)

Author: Jean Francois Regis MUKIZA
GSoC Week: 4 | Phase 2: Attack Simulator (Weeks 3-6)

Core Components Implemented:

1. Five adversarial MCP server mocks (finbot/aegis/simulator/mcp_mocks/adversarial.py):

   • AdmServer (ASI02): Admin panel manipulation with privileged escalation tools
   • AdviceServer (ASI04): Financial advice manipulation for investment fraud simulation
   • DataExfiltrationServer (ASI05): Sensitive data theft including PII and financial records
   • FileSystemServer: Unauthorized file system access for reading/writing sensitive files
   • ToolPoisoningServer: Malicious tool injection and backdoor implantation capabilities

2. Package initializations:

   • finbot/aegis/simulator/mcp_mocks/__init__.py
   • finbot/aegis/simulator/recipes/__init__.py

3. Attack recipe collections (YAML-defined parametric attack scenarios):

   • 8 ASI01 Prompt Injection recipes (asi01_injection.yaml): System prompt extraction via roleplay, encoding obfuscation, context switching, and authority impersonation
   • 8 ASI02 Tool Misuse recipes (asi02_misuse.yaml): Unauthorized vendor creation, fund transfers, financial report modification, and chained tool misuse attacks

4. Comprehensive unit test suite (tests/unit/aegis/test_mcp_mocks.py):

   • Tests for all five adversarial MCP servers' initialization and tool exposure
   • Validation of specific attack tool executions (e.g., get_admin_panel, get_user_data)
   • Verification of attack simulation fidelity and error handling

Technical Implementation:

• All MCP servers inherit from BaseAdversarialServer with proper MCP protocol implementation
• Servers expose realistic attack surface through standardized MCP tool interfaces
• Recipes follow existing YAML schema with asi, target_agent, description, and executable steps
• Unit tests utilize pytest-asyncio with mocking for isolated server behavior validation
• Implementation adheres to existing FinBot codebase patterns and import conventions

Integration Readiness:

• Components designed for seamless integration with existing SandboxHarness in finbot/aegis/simulator/base.py:
  • MCP servers can be instantiated and run as isolated attack fixtures
  • Recipe collections load automatically via SandboxHarvest._load_recipes()
  • Test suite validates compatibility with attack execution framework

---

📅 Week 5 Contributions (Attack Simulation Expansion)

Author: Jean Francois Regis MUKIZA
GSoC Week: 5 | Phase 2: Attack Simulator (Weeks 3-6)

New Attack Recipe Collections:

1. ASI04 Supply Chain Attacks (asi04_supply.yaml): Supply chain compromise scenarios targeting financial dependencies and third-party integrations
2. ASI05 Remote Code Execution (asi05_rce.yaml): RCE attack vectors including command injection, deserialization exploits, and arbitrary code execution
3. ASI06 Delegation Attacks (asi06_delegation.yaml): Impersonation and privilege abuse through delegated authorization mechanisms

Core Infrastructure Enhancements:

1. Mutation Testing Framework (finbot/aegis/simulator/mutation.py):
   • Implementation of mutation operators for security testing
   • Fault injection mechanisms to evaluate attack detection resilience
   • Configurable mutation strategies for different attack surfaces

Comprehensive Test Suite Expansion:

1. Attack Recipes Unit Tests (tests/unit/aegis/test_attack_recipes.py):

   • Validation of all YAML recipe collections (ASI01-ASI06 at this point, later extended to ASI03, ASI07-ASI10 in Week 6)
   • Testing recipe parameterization and step execution
   • Verification of attack scenario completeness and correctness

2. Mutation Testing Unit Tests (tests/unit/aegis/test_mutation.py):

   • Tests for mutation operator functionality
   • Validation of fault injection mechanisms
   • Assessment of mutation score calculation accuracy

Technical Implementation:

• All new recipe collections follow the established YAML schema with clear asi, target_agent, description, and executable steps
• Mutation framework integrates with existing simulation harness for dynamic attack variation
• Test suites maintain async compatibility with pytest-asyncio and appropriate mocking strategies
• Continued adherence to FinBot codebase patterns, import conventions, and documentation standards

Integration Readiness:

• New recipe collections automatically load via existing SandboxHarvest._load_recipes() mechanism
• Mutation components designed to work with SandboxHarness for dynamic attack scenario generation
• Comprehensive test coverage ensures compatibility with the overall attack execution framework

---

📅 Week 6 Contributions (Remaining ASI Categories and Test Completion)

Author: Jean Francois Regis MUKIZA
GSoC Week: 6 | Phase 2: Attack Simulator (Weeks 3-6)

Core Components Implemented:

1. Remaining attack recipe collections (finbot/aegis/simulator/recipes/asi03_07_08_09_10.yaml):

   • ASI03: Extra safety measure evasion techniques
   • ASI07: Agent impersonation via identity manipulation
   • ASI08: Equity and bias exploitation
   • ASI09: Printer abuse and resource exhaustion
   • ASI010: Model theft and exfiltration
   • Each category includes multiple recipes (total of 15 recipes across these 5 categories) covering various attack vectors.

2. Completed unit test suite for all attack recipes (tests/unit/aegis/test_attack_recipes.py):

   • Extended existing test file to cover ASI03, ASI07, ASI08, ASI09, ASI010 recipes
   • Validates that all recipe collections (ASI01-ASI10) are properly loaded and executable
   • Tests recipe parameterization and step execution for the newly added categories

Technical Implementation:

• The new recipe file follows the same YAML structure as previous collections
• Each recipe includes clear asi tags, target_agent, description, and executable steps
• Test file uses the same parametric test structure to validate all recipe IDs
• No changes to the core simulation infrastructure were required; the new recipes plug into the existing framework

Integration Readiness:

• The new recipe collection is automatically loaded via SandboxHarvest._load_recipes()
• Tests confirm compatibility with the attack execution framework
• Completes the attack simulation coverage for all ASI categories (ASI01-ASI10)

---

🙏 Mentor Acknowledgments

Special thanks to my mentors @mekaizen (Nirupam Ghosh) and @steadhac (Carolina Steadham) for their continuous guidance, insightful feedback, and unwavering support throughout this GSoC journey. Their expertise has been invaluable in shaping these contributions to align with project goals and maintain high code quality standards.

---

🔗 Phase 2 Progress Summary

Weeks 3-6 Combined Deliverables:

• ✅ Sandbox Harness with Subprocess Isolation (base.py)
• ✅ Five Adversarial MCP Servers (ASI02, ASI04, ASI05, FileSystem, ToolPoisoning)
• ✅ Attack Recipe Collections for ALL ASI Categories (ASI01-ASI10):
  • ASI01: Prompt Injection (8 recipes)
  • ASI02: Tool Misuse (8 recipes)
  • ASI03: Extra Safety Evasion (X recipes) [from asi03_07_08_09_10.yaml]
  • ASI04: Supply Chain Attacks
  • ASI05: Remote Code Execution
  • ASI06: Delegation/Privilege Abuse
  • ASI07: Agent Impersonation (from asi03_07_08_09_10.yaml)
  • ASI08: Equity and Bias Exploitation (from asi03_07_08_09_10.yaml)
  • ASI09: Printer Abuse and Resource Exhaustion (from asi03_07_08_09_10.yaml)
  • ASI010: Model Theft and Exfiltration (from asi03_07_08_09_10.yaml)
• ✅ Mutation Testing Framework for security resilience evaluation
• ✅ Comprehensive Test Suite validating all components:
  • test_simulator_harness.py (Week 3)
  • test_mcp_mocks.py (Week 4)
  • test_attack_recipes.py (Weeks 5-6, now covering ASI01-ASI10)
  • test_mutation.py (Week 5)
• ✅ Seamless Integration with a unified attack execution framework

This work establishes a complete attack simulation foundation for Phase 2, enabling comprehensive testing across all ASI categories (ASI01-ASI10) with capabilities for both static attack scenarios and dynamic mutation-based testing. The infrastructure is now ready for subsequent phases to implement advanced attack chaining and defense mechanisms.

---

Author: Jean Francois Regis MUKIZA
GSoC Weeks: 3-6 | Phase 2: Attack Simulator (Weeks 3-6)