test(email): two phishing-guard tests assert against the retired agent's prompt

[itomek]

Problem

Two phishing-guard tests assert against the retired email agent's system prompt, so they guard a surface no user reaches. TestPhishingBodyPromptInjection in tests/unit/agents/email/test_phishing_block_tool.py checks that gaia_agent_email.agent._SYSTEM_PROMPT contains the untrusted-body warning and tells the model to refuse acting on body instructions. Both of those properties now matter on the flagship's EmailToolsMixin path instead, where PR #4155 adds a real equivalent (tests/unit/agents/tools/test_email_phishing_guard.py, no import gate).

What makes them actively misleading rather than merely obsolete: the module is gated behind pytest.importorskip("gaia_agent_email"), so outside the one CI lane that installs the hub wheel they vanish from the run without a word. Anyone auditing phishing coverage locally, or reading the main unit lane, sees green and a test file with the right name.

Part of #3814. Related: #4150 (the defect), #4155 (the flagship guard), #4121 (the general collected-then-skipped sweep).
Spec: none: searched docs/spec/ — no spec governs test placement; the retirement is stated in milestone 59's description.

Outcome

Phishing-guard coverage lives on the path that actually serves mail, and no test file reads as coverage for a surface that has been retired.

Acceptance criteria

• TestPhishingBodyPromptInjection (tests/unit/agents/email/test_phishing_block_tool.py:325) is deleted, or retargeted at the flagship's guard so it asserts against src/gaia/agents/tools/_email/phishing.py / EmailToolsMixin rather than gaia_agent_email.agent._SYSTEM_PROMPT
• If retargeted, the file it lands in has no module-level importorskip — it must run in test_unit.yml, which installs only .[api]
• The two behaviours stay covered somewhere that runs by default: an untrusted-body warning is present, and the model is told to refuse acting on body instructions — today asserted at :334 and :344 against the retired agent
• pytest tests/unit/agents/tools/test_email_phishing_guard.py passes with no gaia_agent_email installed — expected 0 skipped

Scope & expectations

• Where: tests/unit/agents/email/test_phishing_block_tool.py (the TestPhishingBodyPromptInjection class only, :325-352); the flagship guard tests added by fix(email): stop the flagship repeating a phishing lure's instructions #4155 under tests/unit/agents/tools/
• Out of scope: the other ~10 tests in that file, which exercise the retired agent's quarantine/undo tools and are a separate retirement decision; the general "a whole-file skip is invisible" guard, which is ci: nothing verifies that a test file is actually run by any CI lane #4121's ask, not this issue's
• Constraints: land after fix(email): stop the flagship repeating a phishing lure's instructions #4155, so the replacement coverage exists before the old assertions go

How to verify

CLI surface: pytest tests/unit/agents/tools/ -q in an environment with no gaia_agent_email installed — show the count and that nothing skips. Then the email lane's own run, showing the retired-agent class is gone rather than silently skipping.

🔍 Technical details

Claim class: reproduced (executed) — the skip below was run in this checkout at 572e8b79; the code and workflow citations were opened and checked.

$ python -m pytest "tests/unit/agents/email/test_phishing_block_tool.py::TestPhishingBodyPromptInjection" -q
1 skipped in 0.09s

import gaia_agent_email → ModuleNotFoundError in this environment. The gate is tests/unit/agents/email/test_phishing_block_tool.py:33.

The two tests, at :334 and :344:

def test_system_prompt_contains_untrusted_input_warning(self):
    from gaia_agent_email.agent import _SYSTEM_PROMPT
    assert "UNTRUSTED" in _SYSTEM_PROMPT

Correction to how this was first framed. These are not tests that "cannot fail". They do run — test_email_agent_unit.yml:143 installs -e hub/agents/email/python and :171 runs tests/unit/agents/email/ as a directory. #4121 already measured this and explicitly cleared the email-gated files as near-misses for exactly that reason. Where they silently skip is test_unit.yml, which installs only .[api] (:117) and runs tests/unit/ (:137) — and any dev machine without the hub wheel, as above.

So the defect is the target, not the lane: green assertions about a retired agent's prompt, in a file named as if it guards phishing today. The skip is what stops that being obvious.

[github-actions]

Confirmed, and the simplest resolution is a plain deletion rather than a retarget.

The two assertions check the retired agent's prompt text for a warning and the word "refuse". The flagship's replacement in #4155 isn't a prompt string at all — it fences every message body and attaches a stated verdict on the tool result itself, so the safeguard holds even when no skill is loaded. Porting the string checks over would give you a weaker duplicate of coverage that already exists there, including the "don't repeat what the mail told the user to do" property.

The blocker is still real: #4155 is open, so nothing should be removed yet. Once it merges, deleting the class at :325-352 satisfies all four acceptance criteria — the new guard file has no import gate, so its 0 skipped requirement is met as written.

Next step: wait on #4155, then a delete-only PR touching that one class.

🔍 Technical details

Reproduced the skip at tests/unit/agents/email/test_phishing_block_tool.py:33 — pytest.importorskip("gaia_agent_email") at module level, so the whole file vanishes under test_unit.yml, which installs only .[api].

#4155 adds src/gaia/agents/tools/_email/phishing.py with UNTRUSTED_BODY_OPEN/CLOSE and a SUSPICIOUS_GUIDANCE constant emitted on the tool result. The two behaviours the issue asks to preserve map onto:

• untrusted-body warning → test_read_email_fences_the_body_and_flags_it, test_every_body_is_fenced_not_just_a_suspicious_one
• refuse acting on body instructions → test_listing_forbids_the_action_bucket_and_the_restated_ask, plus test_the_guard_holds_with_no_skill_loaded

tests/unit/agents/tools/test_email_phishing_guard.py in that PR has no module-level importorskip, so criterion 4 holds without further work.

The remaining ~10 tests in the email file exercise the quarantine/undo tools and stay, per the issue's scope note.