Skip to content

Cut over the k8s CronJob from cronjob-test/ to 2026/ (and retire the browser extension) #56

Description

@rtanglao

Ask

@aatchison — once you're happy with the CronJob from #44, please flip its data root from cronjob-test/ to the real 2026/ tree so it becomes the live producer of AAQ data.

Why now

The browser extension (#29) is currently the only thing writing to 2026/. All three GitHub workflows are disabled_manually, so if the extension stops before the cutover, the dataset simply stops. Right now both writers coexist safely only because the CronJob is pointed at cronjob-test/.

Validation so far (2026-07-28, comparing cronjob-test/2026/ against 2026/ at da97f02)

Across all 12 days the CronJob has produced (2026-01-25 … 2026-07-15), the output looks correct:

  • Question row counts match exactly on every day — 01-25, 02-18, 03-05, 04-02, 04-22, 05-20, 06-04, 06-30, 07-12, 07-13, 07-14, 07-15.
  • Answers are a superset, e.g. 07-12: 97 rows (CronJob) vs 82 (extension); 07-13: 72 vs 65. This is the CronJob correctly re-refreshing older days that the extension's 7-day trailing window has moved past — updates the extension structurally cannot see.
  • updated timestamps are fresher for the same reason (e.g. Q1592638: 2026-07-22T11:31:40Z in cronjob-test/ vs 2026-07-19T06:24:22Z in 2026/).
  • Line endings are clean — both committed blobs are LF (core.autocrlf=input); the CRLF that shows up in a local working tree is a checkout artifact, not committed content.

The superset behaviour is a genuine argument for the cutover: the extension's trailing window permanently misses late edits and answers on older days, which is also the root of #55.

Proposed sequence

  1. Repoint the CronJob's data root from cronjob-test/ to 2026/ and unsuspend (per the plan in docs/superpowers/plans/2026-07-13-k8s-argocd-scraper-deployment.md).
  2. Let a few hourly commits land on 2026/ and confirm they're sane.
  3. Only then disable the extension's daily alarm — until step 2 passes, keep running it.
  4. Decide what to do with the now-redundant cronjob-test/ tree (delete, or keep as a staging target for future changes).

Open question this resolves

CLAUDE.md currently records an undecided point: whether the CronJob replaces the extension or runs alongside it. Both writing the same day-CSVs on main duplicates work and races (the pod's rebase-retry copes, but it isn't a plan). This issue proposes replace, with the extension retained as a manual fallback should the pod's egress allowlisting ever lapse.

Context

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions