Skip to content

feat(mcp): add run_rank_tracker to trigger a check off-schedule - #157

Open
shanegringdotcom wants to merge 1 commit into
every-app:mainfrom
shanegringdotcom:feat/mcp-run-rank-tracker
Open

shanegringdotcom wants to merge 1 commit into
every-app:mainfrom
shanegringdotcom:feat/mcp-run-rank-tracker

Conversation

@shanegringdotcom

Copy link
Copy Markdown

Problem

Rank tracking is the only MCP surface with no way to act. get_rank_tracker reads configs and positions, but its description ends "To trigger a new check, use the dashboard." An agent that notices a tracker is stale has to hand the task back to a human.

This shows up in practice. I run weekly SEO reports for ~12 projects off this MCP. One project's weekly tracker stopped firing — nextCheckAt sat 2 days in the past with isActive: true and lastSkipReason: null, positions frozen at the previous run. From the MCP alone an agent can see the staleness but can't fix it, and can't distinguish "no movement" from "no check ran". Six other trackers had drifted to 14-day gaps, which is a subtle correctness problem: two consecutive snapshots reported as week-over-week movement are simply wrong, and the number itself looks real enough to survive review.

What this adds

run_rank_tracker(projectId, trackerId) — triggers an on-demand check.

RankTrackingService.triggerCheck already implements this for the dashboard and withMcpProjectAuth already supplies context.billing, so this wires the existing service up exactly the way run_site_audit does. No new service or repository code.

Behaviour worth calling out:

  • already_running is not an error. It returns started: false with an explanatory message instead of throwing, so a caller polls rather than retrying and double-charging.
  • Ownership is checked before spending. A tracker outside the project throws NOT_FOUND before triggerCheck, so a bad id can't burn credits.
  • Telemetry only fires for a run that actually began.
  • The cost is stated in the tool description — one SERP check per keyword per device. This is the rare MCP tool that spends money, so the model should see that before calling it.

Also updates get_rank_tracker's description to point at run_rank_tracker instead of the dashboard, and to mention lastCheckedAt as the freshness signal — an agent needs that to decide whether a check is worth paying for.

Testing

pnpm ci:check and pnpm test:ci both pass (736 tests, 92 files).

New run-rank-tracker.test.ts covers the three outcomes that matter: a started run returns its id, an in-flight run reports started: false and skips telemetry, and an unknown tracker throws before triggerCheck is reached.

Notes

Kept to one tool per the contributing guide. create_rank_tracker would be the natural follow-up — I also had to create a tracker by hand — but it pulls in keyword management, so it seemed better as a separate PR if you'd want it at all.

Happy to adjust naming, the description wording, or the already_running shape if you'd prefer it throw.

Rank tracking was the only MCP surface with no way to act: get_rank_tracker
reads configs and positions, but its description ends "To trigger a new check,
use the dashboard." An agent that notices a tracker is stale has to hand the
task back to a human.

That gap is visible in practice. A weekly tracker whose scheduled run does not
fire leaves `nextCheckAt` in the past and positions frozen, and an agent
building a weekly report cannot tell the difference between "no movement" and
"no check ran" — or fix it. Two consecutive snapshots 14 days apart get
reported as week-over-week movement.

RankTrackingService.triggerCheck already implements this for the dashboard and
withMcpProjectAuth already supplies context.billing, so this wires the existing
service up the same way run_site_audit does.

- `already_running` returns started:false with an explanatory message rather
  than throwing, so a caller polls instead of retrying and double-charging.
- A tracker outside the project throws NOT_FOUND before triggerCheck, so a bad
  id cannot spend credits.
- Telemetry only fires for a run that actually began.
- get_rank_tracker's description now points at run_rank_tracker instead of the
  dashboard, and mentions lastCheckedAt as the freshness signal.

Credits: one SERP check per keyword per device, same as a scheduled run — the
description says so, since this is the rare MCP tool that spends money.

pnpm ci:check and pnpm test:ci pass (736 tests).

@sorcerai sorcerai left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: this MCP path bypasses the paid-plan gate used by both existing rank-check entry points. The handler calls RankTrackingService.triggerCheck after ownership lookup, but unlike src/serverFunctions/rank-tracking.ts and scheduledRankChecks.ts it never checks hosted mode + customerHasPaidPlan. The workflow checks usage credits, not paid-plan status, so a hosted non-paid MCP caller can reach provider spend. Please gate after the project-scoped lookup and before triggerCheck, reuse the existing PAYMENT_REQUIRED message, and add a focused test proving triggerCheck/provider work is not called for a hosted unpaid org. Also align successful telemetry with rank_tracking:check_trigger + runId, or document/test a deliberate new event contract.

HasimEmre pushed a commit to HasimEmre/open-seo that referenced this pull request Jul 30, 2026
@bensenescu

Copy link
Copy Markdown
Contributor

Hey,

Thanks for contributing!

For the next few months, I'm going to stop accepting external PRs. Reviewing and testing them has been slowing down progress against the roadmap.

Right now, the best way to contribute is through creating quality "Issues" which are easier for me to review and prioritize right now.

Here is our updated contributing guide: https://github.com/every-app/open-seo/blob/main/docs/CONTRIBUTING.md

Converting, this PR to an issue would be the best way to get it prioritized.

You can use the the /simple-issue-description skill described in the guide to convert the PR to an issue.

ramonmnavarro-byte pushed a commit to ramonmnavarro-byte/open-seo that referenced this pull request Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants