Skip to content

AI spam/phishing scan of templates and broadcasts with auto-suspend (hosted) #72

Closed

Description

@clbotdev

Background

Nothing checks what hosted customers send. Templates and broadcasts go straight to the ESP after rendering (apps/api/src/mail/render.ts, apps/api/src/mail/worker.ts). Searching apps/api/src for "phish", "moderat" or spam scanning finds nothing. Spam and phishing from one hosted account can get our shared ESP connections, domains and payment account flagged. Source: Jesse Hanley's anti-abuse checklist (shared by Marc Lou on X, 18 Oct 2026).

Building blocks that exist: teams have a sending_suspended status and organizations a suspended status (apps/api/src/db/schema.ts, apps/api/src/team/routes.ts:77), so the mechanism to stop sending is there. Nothing triggers it automatically yet.

Part of #75 (hosted abuse prevention umbrella).

Proposed fix

  • When a template, broadcast or sequence email is created or updated, and again at send/start time, run an AI classifier over the subject, body, links and sender. It checks for spam, phishing, credential harvesting and malware links, and known bad link domains.
  • Store the verdict and reasons on the email/template.
  • High-confidence phishing/malware: block the send, set the team to sending_suspended, notify the team and alert admins.
  • Medium risk: hold the broadcast for admin review and warn the user.
  • Provide an admin review queue with release or confirm-suspension, and an appeal path for the user.
  • Hosted-only, feature-flagged; skip or relax for trusted, paid, long-standing orgs to control cost.

Acceptance criteria

  • Every new or edited marketing email is scanned before it can be started or sent
  • High-risk content blocks the send and sets the team to sending_suspended, with a user notice and an admin alert (tested)
  • Medium-risk content is held for admin review; admins can release it
  • Verdicts and reasons are stored and visible to admins
  • Hosted-only flag; self-host unaffected by default

Activity

  1. clbotdev commented on Oct 19, 2026

    @clbotdev
    ContributorAuthor

    Implemented in #91 (merged). Its PR body listed this issue after "Closes #71,", which GitHub doesn't auto-close.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestseverity: mediumNotable gap with a workaround, or competitive parity

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions