feat: Add Flue v2 issue triage shadow mode - #19079
Open
sfanahata wants to merge 21 commits into
Open
Conversation
Adds a Flue agent that classifies incoming GitHub issues using existing label taxonomy (Platform, Product Area, Team, Impact, Effort) and generates structured triage reports. Runs on issue open events and via manual workflow_dispatch trigger. - .flue/agents/triage-issue.ts: Agent entry point using Sonnet - .agents/skills/classify-docs-issue.md: Classification skill with full label mapping from issue templates - .flue/AGENTS.md: Project context for the agent - .github/workflows/flue-triage-issue.yml: GitHub Actions workflow - DRY_RUN=true by default, set DRY_RUN=false for live runs Co-Authored-By: Claude <noreply@anthropic.com>
- Remove issues.opened trigger — manual dispatch only until prompt injection detection is added - Remove LINEAR_API_KEY from agent sandbox — Linear ticket creation will be a separate post-agent step - Downgrade permissions to issues: read (no write needed yet) - Fix concurrency group to match dispatch-only trigger Co-Authored-By: Claude <noreply@anthropic.com>
GitHub API calls are now wrapped as custom ToolDefs (fetch_issue, search_issues) that run in the Node process, not the sandbox. The agent never sees GH_TOKEN — it calls the tools by name and gets structured results back. Also fixes import path to @flue/runtime/client (what the CLI bundles internally, vs @flue/sdk/client which requires separate install). Co-Authored-By: Claude <noreply@anthropic.com>
Issue content is now fetched and validated in the TypeScript handler before the LLM ever sees it. If injection patterns are detected, the agent returns a flagged report and skips AI triage entirely. The skill now receives pre-parsed fields (title, body, labels, author) as arguments instead of fetching the issue itself, so the LLM never processes raw untrusted API responses. Co-Authored-By: Claude <noreply@anthropic.com>
- Import from @flue/runtime (client entrypoint was folded in) - Use local() factory instead of 'local' string - Move skill to .agents/skills/<name>/SKILL.md convention Verified: dry run against issue #17799 produces correct triage. Co-Authored-By: Claude <noreply@anthropic.com>
- Use Linear priority scale (urgent/high/medium/low) instead of impact (small/medium/large) to match the Docs team's workflow - Update existing DOCS-XXXX Linear ticket instead of creating duplicates — finds the auto-synced ticket and sets priority + labels - Fix get_linked_prs tool to return JSON strings (Flue tools must return strings) - Add PR-first check: skip deep RCA when a linked PR already exists - Handle unstructured issues (no template labels) by classifying from content alone Co-Authored-By: Claude <noreply@anthropic.com>
When DRY_RUN=false, the agent now posts the full triage report as a comment on the existing DOCS-XXXX ticket in addition to setting priority and labels. This gives Shannon/Alex the classification, related docs, suggested labels, and recommended action directly in Linear. Co-Authored-By: Claude <noreply@anthropic.com>
- Sort imports per simple-import-sort - Name the default export function (import/no-anonymous-default-export) - Apply Prettier formatting Co-Authored-By: Claude <noreply@anthropic.com>
- Use Docs team label IDs instead of DevEx team IDs - Separate priority, label, and comment into independent API calls so one failure doesn't block the others - Add error logging for failed Linear mutations Tested live: priority updates correctly, triage comments post to existing DOCS tickets. Co-Authored-By: Claude <noreply@anthropic.com>
Fetch the issue's current labels in the search query and skip issueAddLabel when the target label is already present. This avoids Linear's label-group exclusivity errors (e.g., trying to add 'Docs' when it's already on the issue from GitHub sync). Co-Authored-By: Claude <noreply@anthropic.com>
Major architecture change for security, idempotency, and debuggability: - Agent is now purely read-only — returns JSON, no secrets, no writes - New apply-triage.sh handles all writes (GitHub labels, Linear update, GitHub comment fallback) with idempotency checks - Uses <!-- flue-triage --> marker to prevent duplicate comments - Checks Linear for existing triage before posting - Falls back to GitHub comment if Linear ticket not found yet - Re-enables issues.opened trigger (agent has no write access) - Simplified triage report format — no more redundant fields - Removed suggestedLabels and dryRun (agent is always read-only) Co-Authored-By: Claude <noreply@anthropic.com>
Consolidate all orchestration (triage + writes) in the TypeScript handler instead of a separate bash script. This follows Flue's recommended pattern where the handler is the orchestration layer. Keeps all improvements from the bash approach: - Idempotency via triage marker on GitHub comments - Linear comment dedup check - Label conflict handling (skip if already present) - GitHub fallback when Linear ticket not found - Error logging without crashing Removes: - .flue/scripts/apply-triage.sh - JSON extraction logic in workflow - Multi-step workflow (now single step) Co-Authored-By: Claude <noreply@anthropic.com>
Add author_association check so the agent only runs for issues opened by MEMBER, COLLABORATOR, or OWNER. External users' issues are still triaged via manual workflow_dispatch. This prevents unbounded Anthropic API costs from spam issue creation. Addresses Warden finding D7Y-HH3. Co-Authored-By: Claude <noreply@anthropic.com>
Match sentry-javascript's pattern: only run triage when the issue has template-applied labels (Docs, Docs Platform, or Bug). Issues without templates (spam, bots) skip auto-triage but can still be triaged via workflow_dispatch. Co-Authored-By: Claude <noreply@anthropic.com>
Fixes: - Team field now uses v.picklist with allowlist matching labels.yml instead of v.optional(v.string()) — prevents injection of arbitrary GitHub labels (Warden VK2-ZRW) - Linear fallback checks commentCreate success before setting linearOk — failed mutations now correctly trigger GitHub fallback (Cursor) - get_linked_prs fetches full PR details via /pulls/:number API to get accurate merged status (Cursor) - Idempotency fix: GitHub fallback comment uses TRIAGE_MARKER but Linear success does not, so re-runs can retry Linear when ticket appears later (Sentry bot) - Removed overly broad injection patterns (act as, curl|sh, echo, eval, base64) that would false-positive on legitimate issue content (Sentry bot) - Fixed SKILL.md step numbering (duplicate Step 2, missing Step 8) and added explicit summary instruction (Sentry bot) - linearQuery now catches fetch errors instead of crashing Not a bug: "missing filesystem tools" — sandbox: local() provides bash/grep/find via Flue's built-in tools, custom tools are additional. Co-Authored-By: Claude <noreply@anthropic.com>
- Global concurrency group (flue-triage) so only one triage runs at a time — queues instead of parallelizing - AbortSignal.timeout(120s) on the skill call — hard cap on LLM processing time per issue Combined with label gate and 10-min workflow timeout, worst case for 100 spam issues is now ~$5-10 processed sequentially (~2 min each) instead of in parallel. The ultimate backstop is setting a spend limit on the Anthropic API key itself. Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
🏷 The following changes will be made to the repository labels |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Contributor
Author
|
Addressed the label-sync dry-run warning in 4e50a7a by removing the |
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f4dec0c. Configure here.
| '<!-- sentry-docs-parking-review:v1 -->', | ||
| `${docsMention} review requested: this issue is proposed for Parking Lot (${result.decision.parkingLotReason}). GitHub remains open and Linear remains active until a person approves or reprioritizes it.` | ||
| ); | ||
| } |
Contributor
There was a problem hiding this comment.
Parking Lot label never cleared
Medium Severity
applyTriageResult only adds the Parking Lot label for immediate no-priority review and never removes it. After a human reprioritizes or a later triage assigns a real priority, GitHub can keep showing Parking Lot while Linear is active again, which contradicts the review comment that reprioritization leaves the issue active.
Reviewed by Cursor Bugbot for commit f4dec0c. Configure here.
| if (response.status < 300) { | ||
| throw new Error(`The reported broken URL currently resolves: ${response.url}`); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Redirects treated as broken URLs
Medium Severity
verifyBroken uses redirect: 'manual' and only rejects responses with status below 300. A 301/302 therefore counts as broken, so the autofixer can open content-edit or redirect PRs for URLs that already resolve through an existing redirect.
Reviewed by Cursor Bugbot for commit f4dec0c. Configure here.
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Comment on lines
+52
to
+70
| export function parseTriageState(body: string): PersistedTriageState | undefined { | ||
| const match = body.match(/<!-- sentry-docs-triage-state:v2:([A-Za-z0-9_-]+) -->/); | ||
| if (!match) return undefined; | ||
| const parsed = JSON.parse(Buffer.from(match[1], 'base64url').toString('utf8')) as { | ||
| policyVersion?: number; | ||
| triagedAt?: string; | ||
| decision?: unknown; | ||
| applied?: PersistedTriageState['applied']; | ||
| overrides?: PersistedTriageState['overrides']; | ||
| }; | ||
| if (parsed.policyVersion !== 2 || !parsed.triagedAt) return undefined; | ||
| return { | ||
| policyVersion: 2, | ||
| triagedAt: parsed.triagedAt, | ||
| decision: v.parse(TriageDecisionSchema, parsed.decision), | ||
| ...(parsed.applied ? {applied: parsed.applied} : {}), | ||
| ...(parsed.overrides ? {overrides: parsed.overrides} : {}), | ||
| }; | ||
| } |
There was a problem hiding this comment.
Untrusted Linear comments control triage state and lifecycle closures
Anyone who can comment on a linked Linear issue can plant a forged sentry-docs-triage-state:v2 marker that apply/lifecycle treat as authoritative, including forcing issue closures and Parking Lot moves after crafted due dates.
Evidence
parseTriageState()accepts any Linear comment body matching<!-- sentry-docs-triage-state:v2:... -->and only validatespolicyVersion,triagedAt, andTriageDecisionSchema; it does not check comment author, bot identity, or signature.applyTriageResult()selectsexistingStatewithlinear.comments.toReversed().map(parseTriageState).find(Boolean), so the newest matching comment wins regardless of who wrote it.run-lifecycle.tsuses the same unauthenticated lookup and, whenstate.decision.actionability === 'needs-information'andneedsInformationCloseDueAt(derived from attacker-controlledtriagedAt) is due, cancels Linear and closes the GitHub issue asnot_planned.- The same path can also force Parking Lot labeling plus GitHub/Linear closure for inactive medium/low issues once the forged
triagedAt/decision makeparkingLotEligibleAtdue.
Also found at 1 additional location
.flue/run-lifecycle.ts:40-44
Identified by Warden · security-review · YGA-AN7
| '--title', | ||
| `fix(docs): Resolve broken link from #${issue.number}`, | ||
| '--body', | ||
| `Automated, validated broken-link fix.\n\n${decision.quickFix!.description}\n\nFixes #${issue.number}${linearReference}\n\nValidation: redirect rules, redirect-chain lint, focused tests, formatting, and git diff checks passed.`, |
There was a problem hiding this comment.
Model-controlled quickFix description can inject GitHub closing keywords into bot PRs
Untrusted model quickFix.description is interpolated into the automated PR body without neutralizing GitHub closing keywords, so a crafted broken-link triage result can make the bot PR auto-close unrelated issues on merge. Sanitize or omit model text from the body (e.g. strip Fixes/Closes/Resolves forms) and keep only server-generated references.
Evidence
createPullRequest()buildsgh pr create --bodywith${decision.quickFix!.description}inserted verbatim before the intentionalFixes #${issue.number}line.decisionis parsed only withTriageDecisionSchema;quickFix.descriptionis a free-form string up to 500 chars and may contain newlines andFixes/Closes/Resolves #Nkeywords.main()feeds that model decision intocreatePullRequest()after eligibility checks, and the workflow runs this with a GitHub App token that haspull-requests: write.- No sanitization or allowlist removes closing-keyword patterns from the model-controlled description before the privileged PR is opened.
Identified by Warden · security-review · HRU-Q2H
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Builds the GitHub issue triage bot proposed in #17811 on Flue v2. The complete bot is included, but it merges in read-only shadow mode. We review backlog results before enabling writes.
Behavior
Parking Lot, leave open, and request Docs review in Linear.Parking Lot, moved to LinearCanceled, commented, and closed.Safety gates
FLUE_TRIAGE_MODE=shadowFLUE_TRIAGE_MODE=applyFLUE_TRIAGE_AUTO_FIX_ENABLED=trueApply and auto-fix remain disabled unless explicitly enabled. Analysis, issue mutations, and repository-write PR creation run in separately permissioned jobs.
Validation plan
After merge:
FLUE_TRIAGE_MODE=shadow.Triage Backtestfor a calibration sample, then the open backlog.The backtest never writes. Immediate “not worth prioritizing” decisions remain open for human Parking Lot review even in apply mode.
Before apply mode, create the GitHub
Parking Lotlabel outside the currently incomplete declarative label catalog.Verification