feat(mcp): add run_rank_tracker to trigger a check off-schedule - #157
shanegringdotcom wants to merge 1 commit into
Conversation
Rank tracking was the only MCP surface with no way to act: get_rank_tracker reads configs and positions, but its description ends "To trigger a new check, use the dashboard." An agent that notices a tracker is stale has to hand the task back to a human. That gap is visible in practice. A weekly tracker whose scheduled run does not fire leaves `nextCheckAt` in the past and positions frozen, and an agent building a weekly report cannot tell the difference between "no movement" and "no check ran" — or fix it. Two consecutive snapshots 14 days apart get reported as week-over-week movement. RankTrackingService.triggerCheck already implements this for the dashboard and withMcpProjectAuth already supplies context.billing, so this wires the existing service up the same way run_site_audit does. - `already_running` returns started:false with an explanatory message rather than throwing, so a caller polls instead of retrying and double-charging. - A tracker outside the project throws NOT_FOUND before triggerCheck, so a bad id cannot spend credits. - Telemetry only fires for a run that actually began. - get_rank_tracker's description now points at run_rank_tracker instead of the dashboard, and mentions lastCheckedAt as the freshness signal. Credits: one SERP check per keyword per device, same as a scheduled run — the description says so, since this is the rare MCP tool that spends money. pnpm ci:check and pnpm test:ci pass (736 tests).
sorcerai
left a comment
There was a problem hiding this comment.
Blocking: this MCP path bypasses the paid-plan gate used by both existing rank-check entry points. The handler calls RankTrackingService.triggerCheck after ownership lookup, but unlike src/serverFunctions/rank-tracking.ts and scheduledRankChecks.ts it never checks hosted mode + customerHasPaidPlan. The workflow checks usage credits, not paid-plan status, so a hosted non-paid MCP caller can reach provider spend. Please gate after the project-scoped lookup and before triggerCheck, reuse the existing PAYMENT_REQUIRED message, and add a focused test proving triggerCheck/provider work is not called for a hosted unpaid org. Also align successful telemetry with rank_tracking:check_trigger + runId, or document/test a deliberate new event contract.
|
Hey, Thanks for contributing! For the next few months, I'm going to stop accepting external PRs. Reviewing and testing them has been slowing down progress against the roadmap. Right now, the best way to contribute is through creating quality "Issues" which are easier for me to review and prioritize right now. Here is our updated contributing guide: https://github.com/every-app/open-seo/blob/main/docs/CONTRIBUTING.md Converting, this PR to an issue would be the best way to get it prioritized. You can use the the |
Problem
Rank tracking is the only MCP surface with no way to act.
get_rank_trackerreads configs and positions, but its description ends "To trigger a new check, use the dashboard." An agent that notices a tracker is stale has to hand the task back to a human.This shows up in practice. I run weekly SEO reports for ~12 projects off this MCP. One project's weekly tracker stopped firing —
nextCheckAtsat 2 days in the past withisActive: trueandlastSkipReason: null, positions frozen at the previous run. From the MCP alone an agent can see the staleness but can't fix it, and can't distinguish "no movement" from "no check ran". Six other trackers had drifted to 14-day gaps, which is a subtle correctness problem: two consecutive snapshots reported as week-over-week movement are simply wrong, and the number itself looks real enough to survive review.What this adds
run_rank_tracker(projectId, trackerId)— triggers an on-demand check.RankTrackingService.triggerCheckalready implements this for the dashboard andwithMcpProjectAuthalready suppliescontext.billing, so this wires the existing service up exactly the wayrun_site_auditdoes. No new service or repository code.Behaviour worth calling out:
already_runningis not an error. It returnsstarted: falsewith an explanatory message instead of throwing, so a caller polls rather than retrying and double-charging.NOT_FOUNDbeforetriggerCheck, so a bad id can't burn credits.Also updates
get_rank_tracker's description to point atrun_rank_trackerinstead of the dashboard, and to mentionlastCheckedAtas the freshness signal — an agent needs that to decide whether a check is worth paying for.Testing
pnpm ci:checkandpnpm test:ciboth pass (736 tests, 92 files).New
run-rank-tracker.test.tscovers the three outcomes that matter: a started run returns its id, an in-flight run reportsstarted: falseand skips telemetry, and an unknown tracker throws beforetriggerCheckis reached.Notes
Kept to one tool per the contributing guide.
create_rank_trackerwould be the natural follow-up — I also had to create a tracker by hand — but it pulls in keyword management, so it seemed better as a separate PR if you'd want it at all.Happy to adjust naming, the description wording, or the
already_runningshape if you'd prefer it throw.