feat: Flue-based triage agent for docs issues - #17811
Conversation
Adds a Flue agent that classifies incoming GitHub issues using existing label taxonomy (Platform, Product Area, Team, Impact, Effort) and generates structured triage reports. Runs on issue open events and via manual workflow_dispatch trigger. - .flue/agents/triage-issue.ts: Agent entry point using Sonnet - .agents/skills/classify-docs-issue.md: Classification skill with full label mapping from issue templates - .flue/AGENTS.md: Project context for the agent - .github/workflows/flue-triage-issue.yml: GitHub Actions workflow - DRY_RUN=true by default, set DRY_RUN=false for live runs Co-Authored-By: Claude <noreply@anthropic.com>
- Remove issues.opened trigger — manual dispatch only until prompt injection detection is added - Remove LINEAR_API_KEY from agent sandbox — Linear ticket creation will be a separate post-agent step - Downgrade permissions to issues: read (no write needed yet) - Fix concurrency group to match dispatch-only trigger Co-Authored-By: Claude <noreply@anthropic.com>
GitHub API calls are now wrapped as custom ToolDefs (fetch_issue, search_issues) that run in the Node process, not the sandbox. The agent never sees GH_TOKEN — it calls the tools by name and gets structured results back. Also fixes import path to @flue/runtime/client (what the CLI bundles internally, vs @flue/sdk/client which requires separate install). Co-Authored-By: Claude <noreply@anthropic.com>
Issue content is now fetched and validated in the TypeScript handler before the LLM ever sees it. If injection patterns are detected, the agent returns a flagged report and skips AI triage entirely. The skill now receives pre-parsed fields (title, body, labels, author) as arguments instead of fetching the issue itself, so the LLM never processes raw untrusted API responses. Co-Authored-By: Claude <noreply@anthropic.com>
- Import from @flue/runtime (client entrypoint was folded in) - Use local() factory instead of 'local' string - Move skill to .agents/skills/<name>/SKILL.md convention Verified: dry run against issue #17799 produces correct triage. Co-Authored-By: Claude <noreply@anthropic.com>
- Use Linear priority scale (urgent/high/medium/low) instead of impact (small/medium/large) to match the Docs team's workflow - Update existing DOCS-XXXX Linear ticket instead of creating duplicates — finds the auto-synced ticket and sets priority + labels - Fix get_linked_prs tool to return JSON strings (Flue tools must return strings) - Add PR-first check: skip deep RCA when a linked PR already exists - Handle unstructured issues (no template labels) by classifying from content alone Co-Authored-By: Claude <noreply@anthropic.com>
When DRY_RUN=false, the agent now posts the full triage report as a comment on the existing DOCS-XXXX ticket in addition to setting priority and labels. This gives Shannon/Alex the classification, related docs, suggested labels, and recommended action directly in Linear. Co-Authored-By: Claude <noreply@anthropic.com>
- Sort imports per simple-import-sort - Name the default export function (import/no-anonymous-default-export) - Apply Prettier formatting Co-Authored-By: Claude <noreply@anthropic.com>
- Use Docs team label IDs instead of DevEx team IDs - Separate priority, label, and comment into independent API calls so one failure doesn't block the others - Add error logging for failed Linear mutations Tested live: priority updates correctly, triage comments post to existing DOCS tickets. Co-Authored-By: Claude <noreply@anthropic.com>
Fetch the issue's current labels in the search query and skip issueAddLabel when the target label is already present. This avoids Linear's label-group exclusivity errors (e.g., trying to add 'Docs' when it's already on the issue from GitHub sync). Co-Authored-By: Claude <noreply@anthropic.com>
Major architecture change for security, idempotency, and debuggability: - Agent is now purely read-only — returns JSON, no secrets, no writes - New apply-triage.sh handles all writes (GitHub labels, Linear update, GitHub comment fallback) with idempotency checks - Uses <!-- flue-triage --> marker to prevent duplicate comments - Checks Linear for existing triage before posting - Falls back to GitHub comment if Linear ticket not found yet - Re-enables issues.opened trigger (agent has no write access) - Simplified triage report format — no more redundant fields - Removed suggestedLabels and dryRun (agent is always read-only) Co-Authored-By: Claude <noreply@anthropic.com>
Consolidate all orchestration (triage + writes) in the TypeScript handler instead of a separate bash script. This follows Flue's recommended pattern where the handler is the orchestration layer. Keeps all improvements from the bash approach: - Idempotency via triage marker on GitHub comments - Linear comment dedup check - Label conflict handling (skip if already present) - GitHub fallback when Linear ticket not found - Error logging without crashing Removes: - .flue/scripts/apply-triage.sh - JSON extraction logic in workflow - Multi-step workflow (now single step) Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
Add author_association check so the agent only runs for issues opened by MEMBER, COLLABORATOR, or OWNER. External users' issues are still triaged via manual workflow_dispatch. This prevents unbounded Anthropic API costs from spam issue creation. Addresses Warden finding D7Y-HH3. Co-Authored-By: Claude <noreply@anthropic.com>
Match sentry-javascript's pattern: only run triage when the issue has template-applied labels (Docs, Docs Platform, or Bug). Issues without templates (spam, bots) skip auto-triage but can still be triaged via workflow_dispatch. Co-Authored-By: Claude <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
Fixes: - Team field now uses v.picklist with allowlist matching labels.yml instead of v.optional(v.string()) — prevents injection of arbitrary GitHub labels (Warden VK2-ZRW) - Linear fallback checks commentCreate success before setting linearOk — failed mutations now correctly trigger GitHub fallback (Cursor) - get_linked_prs fetches full PR details via /pulls/:number API to get accurate merged status (Cursor) - Idempotency fix: GitHub fallback comment uses TRIAGE_MARKER but Linear success does not, so re-runs can retry Linear when ticket appears later (Sentry bot) - Removed overly broad injection patterns (act as, curl|sh, echo, eval, base64) that would false-positive on legitimate issue content (Sentry bot) - Fixed SKILL.md step numbering (duplicate Step 2, missing Step 8) and added explicit summary instruction (Sentry bot) - linearQuery now catches fetch errors instead of crashing Not a bug: "missing filesystem tools" — sandbox: local() provides bash/grep/find via Flue's built-in tools, custom tools are additional. Co-Authored-By: Claude <noreply@anthropic.com>
- Global concurrency group (flue-triage) so only one triage runs at a time — queues instead of parallelizing - AbortSignal.timeout(120s) on the skill call — hard cap on LLM processing time per issue Combined with label gate and 10-min workflow timeout, worst case for 100 spam issues is now ~$5-10 processed sequentially (~2 min each) instead of in parallel. The ultimate backstop is setting a spend limit on the Anthropic API key itself. Co-Authored-By: Claude <noreply@anthropic.com>
| | Kotlin Multiplatform SDK | `Platform: KMP` | | ||
| | Native SDK | `Platform: Native` | | ||
| | .NET SDK | `Platform: .NET` | | ||
| | PHP SDK | `Platform: PHP` | | ||
| | Python SDK | `Platform: Python` | | ||
| | React Native SDK | `Platform: React-Native` | | ||
| | Ruby SDK | `Platform: Ruby` | | ||
| | Rust SDK | `Platform: Rust` | | ||
| | Unity SDK | `Platform: Unity` | | ||
| | Unreal Engine SDK | `Platform: Unreal` | | ||
| | Sentry CLI | `Platform: CLI` | |
There was a problem hiding this comment.
Bug: The SDK mapping table in SKILL.md is missing several options from the issue template, causing incorrect issue classification and team assignment by the triage agent.
Severity: MEDIUM
Suggested Fix
Update the mapping table in step 3 of .agents/skills/classify-docs-issue/SKILL.md to include all SDK options available in the .github/ISSUE_TEMPLATE/issue-content-01-sdks.yml dropdown. Ensure each new entry maps to a valid GitHub label.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .agents/skills/classify-docs-issue/SKILL.md#L56-L78
Potential issue: The mapping table in `.agents/skills/classify-docs-issue/SKILL.md` is
missing several SDK options present in the
`.github/ISSUE_TEMPLATE/issue-content-01-sdks.yml` issue template, such as `PowerShell
SDK`, `All JavaScript SDKs`, and `Other`. When a user selects one of these missing
options, the AI agent fails to find a corresponding platform label. This results in
incorrect team assignment, often defaulting to the general `Team: Docs` instead of the
appropriate SDK-specific team, reducing the effectiveness of the automated triage
process.
| // No TRIAGE_MARKER so re-runs can retry Linear when ticket exists | ||
| if (!linearOk) { | ||
| const commentsRes = await fetch( | ||
| `https://api.github.com/repos/${REPO}/issues/${issue.number}/comments?per_page=100`, |
There was a problem hiding this comment.
Bug: The get_linked_prs tool hardcodes the repository name, causing it to fail or fetch incorrect data for pull requests from other repositories.
Severity: MEDIUM
Suggested Fix
Modify the get_linked_prs tool to extract the repository owner and name from the cross-referenced event payload instead of using a hardcoded value. Additionally, add a check for res.ok to handle failed API requests gracefully before attempting to parse the JSON response.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.
Location: .flue/agents/triage-issue.ts#L267-L270
Potential issue: The `get_linked_prs` tool in `.flue/agents/triage-issue.ts` incorrectly
assumes that all linked pull requests originate from the same repository
(`getsentry/sentry-docs`). It hardcodes this repository when making API calls to fetch
PR details. If a PR from another repository (e.g., `getsentry/sentry`) references a docs
issue, the API call will either fail with a 404 or fetch an unrelated PR. Because the
code does not check if the fetch was successful before parsing the JSON, the agent
receives incorrect data (e.g., `state: undefined`), leading to flawed analysis of the
issue's status.
|
This pull request has been automatically marked as stale because it has not had recent activity. If this PR is still relevant, please:
Thank you for your contribution! |
|
This pull request has been automatically closed due to inactivity. If you'd like to continue working on this, feel free to reopen the PR or create a new one. |
|
fwiw theres a new version of flue (v2) that has a better developer experience imo |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 52000b4. Configure here.
| filter: { | ||
| team: {key: {eq: 'DOCS'}}, | ||
| attachments: {url: {contains: `sentry-docs/issues/${issue.number}`}}, | ||
| }, |
There was a problem hiding this comment.
Linear issue URL prefix collision
High Severity
The Linear lookup uses contains on sentry-docs/issues/${issue.number}, so issue numbers that are prefixes of others can match the wrong ticket. With first: 1, triage may set priority, labels, and comments on a different DOCS issue.
Reviewed by Cursor Bugbot for commit 52000b4. Configure here.
| ## Step 8: Determine Linear Label | ||
|
|
||
| - If classification is `platform-bug` or `platform-improvement` → `Docs Platform` | ||
| - Everything else → `Docs Content` |
There was a problem hiding this comment.
Broken-link Linear label wrong
Medium Severity
Step 8 maps only platform-bug and platform-improvement to the Linear Docs Platform label. broken-link falls through to Docs Content, even though 404 issues come from the Docs Platform template and are classified separately from content issues.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 52000b4. Configure here.
| linearQuery( | ||
| env.LINEAR_API_KEY, | ||
| `mutation($input: CommentCreateInput!) { | ||
| commentCreate(input: $input) { success } | ||
| }`, | ||
| { | ||
| input: { | ||
| issueId: linearIssue.id, | ||
| body: `🤖 **Auto-triage report**\n\n${data.triageReport}`, | ||
| }, |
There was a problem hiding this comment.
Untrusted issue content can influence privileged Linear priority updates and bot comments
A public GitHub issue can contain instruction-like content that evades the narrow prompt-injection regexes and influences the model's allowed priority and triageReport fields. Those fields are then used by the workflow's privileged credentials to change the DOCS Linear ticket priority and publish model-attributed comments containing attacker-controlled Markdown, enabling limited internal triage manipulation and misleading bot communications.
Evidence
triageIssue()passes the attacker-controlled issue title and body directly tosession.skill('classify-docs-issue');detectInjection()only blocks a small set of literal phrases and does not enforce that the content is non-instructional.- The Valibot schema restricts
priorityto four valid values andtriageReportto a string, but does not verify that the priority matches trusted issue metadata or constrain the report's content or length. applyTriage()uses the workflow'sLINEAR_API_KEYto setissueUpdate(... priority: PRIORITY_MAP[data.priority])and posts${data.triageReport}as an auto-triage Linear comment.- If Linear is unavailable, the same model output is posted to the GitHub issue with a bot-style header and no content or length guard.
Identified by Warden · security-review · LYT-XDV
| const titleFlagged = detectInjection(issue.title); | ||
| const bodyFlagged = detectInjection(issue.body ?? ''); | ||
|
|
||
| if (titleFlagged || bodyFlagged) { | ||
| return { | ||
| classification: 'support-question' as const, | ||
| issueNumber: issue.number, | ||
| flagged: true, | ||
| summary: `Issue #${issue.number} flagged for potential prompt injection. Skipping AI triage.`, | ||
| }; | ||
| } | ||
|
|
||
| const agent = await init({ | ||
| model: 'anthropic/claude-sonnet-4-6', | ||
| sandbox: local(), | ||
| tools: githubTools(token), | ||
| }); | ||
|
|
||
| const session = await agent.session(); | ||
|
|
||
| const {data} = await session.skill('classify-docs-issue', { | ||
| signal: AbortSignal.timeout(120_000), | ||
| args: { | ||
| issueNumber: issue.number, | ||
| title: issue.title, | ||
| body: issue.body ?? '', | ||
| labels: issue.labels.map(l => l.name), | ||
| author: issue.user.login, | ||
| createdAt: issue.created_at, | ||
| }, |
There was a problem hiding this comment.
Untrusted issue content reaches local-sandbox agent with CI secrets
Public issue title/body are passed into an AI agent running sandbox: local() in a job that also has LINEAR_API_KEY, ANTHROPIC_API_KEY, and GH_TOKEN in the process environment. The regex injection check is a small denylist and is easy to bypass, so a crafted issue can steer the agent (which is documented to have shell/gh and filesystem tools) toward reading or exfiltrating those secrets via comments or tool output.
Evidence
triageIssueloads attacker-controlledissue.title/issue.bodyand only gates ondetectInjection()(a short keyword denylist), then passes that content intosession.skill('classify-docs-issue', …).- The agent is started with
sandbox: local()while the workflow step exportsANTHROPIC_API_KEY,LINEAR_API_KEY, andGH_TOKENinto the same process environment. .flue/AGENTS.mddocuments that the agent hasghCLI and local filesystem access under this setup.applyTriagelater posts unconstraineddata.triageReportto Linear/GitHub, which is a viable exfil channel if the agent can observe secrets or tool/shell output.
Also found at 1 additional location
.github/workflows/flue-triage-issue.yml:55-64
Identified by Warden · security-review · CPL-TKH
| - name: Install Flue | ||
| run: npm install -g @flue/cli | ||
|
|
||
| - name: Parse issue number | ||
| id: issue | ||
| env: | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }} | ||
| INPUT_ISSUE_NUMBER: ${{ github.event.inputs.issue_number }} | ||
| run: | | ||
| if [ "$EVENT_NAME" = "issues" ]; then | ||
| echo "number=$EVENT_ISSUE_NUMBER" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "number=$INPUT_ISSUE_NUMBER" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Run triage agent | ||
| env: | ||
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }} | ||
| ISSUE_NUMBER: ${{ steps.issue.outputs.number }} | ||
| run: | | ||
| npx flue run triage-issue --target node \ | ||
| --id "triage-${ISSUE_NUMBER}" \ | ||
| --payload "{\"issueNumber\": ${ISSUE_NUMBER}}" |
There was a problem hiding this comment.
Unpinned Flue CLI executes in a privileged GitHub Actions job
The workflow resolves @flue/cli from npm without a version or integrity pin, then executes it with ANTHROPIC_API_KEY, LINEAR_API_KEY, and an issues:write GITHUB_TOKEN; a malicious release could execute code in the trusted runner and exfiltrate or misuse those credentials. Pin the package to an integrity-checked version and invoke only the verified artifact.
Evidence
- The
Install Fluestep runsnpm install -g @flue/cliwithout a version, lockfile, integrity hash, or--ignore-scripts, allowing mutable registry code and install scripts to execute on the runner. - The following
Run triage agentstep invokes the installed CLI throughnpx flue run triage-issuewhile exposingANTHROPIC_API_KEY,LINEAR_API_KEY, andGH_TOKEN. - The job grants
issues: write; the triage agent uses that token to add labels and post comments, whileLINEAR_API_KEYis used for Linear mutations, so compromised CLI code would run across these security boundaries.
Identified by Warden · security-review · JNW-S73
2 participants
DESCRIBE YOUR PR
Adds an AI-powered triage agent using Flue that automatically classifies incoming GitHub issues and enriches the corresponding Linear ticket.
What it does:
Architecture:
.flue/agents/triage-issue.ts— agent handler (read-only AI triage + write orchestration).agents/skills/classify-docs-issue/SKILL.md— classification skill with label mapping.flue/AGENTS.md— project context for the agent.github/workflows/flue-triage-issue.yml— triggers onissues.opened+ manual dispatchSecurity:
Tested against: #17799, #17412, #17743, #17707, #17716, #17568, #17438, #17601 — all classified correctly.
Requires secrets:
ANTHROPIC_API_KEY,LINEAR_API_KEY(optional, falls back to GitHub)IS YOUR CHANGE URGENT?
PRE-MERGE CHECKLIST