What
Stand up automated detection of documentation drift between the platform docs and the Apify Console UI. UI claims in the docs — a route resolves, a tab/button is named X, a heading is visible on page Y — are extracted as testable assertions and verified against Console staging, so when the UI changes and the docs don't, we find out automatically.
Approach: an adjustable list of docs pages → assertions extracted with an LLM → stored as a committed, human-reviewed baseline → evaluated against staging with Playwright. Failures point back to source_file:line.
Status
Initial harness and a baseline for a 4-page Console test set (index, settings, billing, store — 53 assertions: routes, tabs, buttons, headings) are up in #2669.
Remaining work
Source
Internal plan "AI-based testing for docs"; implementation tracked in #2669.
What
Stand up automated detection of documentation drift between the platform docs and the Apify Console UI. UI claims in the docs — a route resolves, a tab/button is named X, a heading is visible on page Y — are extracted as testable assertions and verified against Console staging, so when the UI changes and the docs don't, we find out automatically.
Approach: an adjustable list of docs pages → assertions extracted with an LLM → stored as a committed, human-reviewed baseline → evaluated against staging with Playwright. Failures point back to
source_file:line.Status
Initial harness and a baseline for a 4-page Console test set (index, settings, billing, store — 53 assertions: routes, tabs, buttons, headings) are up in #2669.
Remaining work
lychee.yml), with all secrets in GitHub Secrets and nothing in.envSource
Internal plan "AI-based testing for docs"; implementation tracked in #2669.