Harden Apple Ads agent preflight and metric semantics

docs/agent/asa-metrics.md

@@ -48,7 +48,8 @@ names costs at most one call and should never be done on purpose.
 Cohort roots — `revenue`, `arpu`, `arppu`, `arpas` (alias `cohort_arpas`), `roas`, `roi` —
 expand to `gross_`, `proceeds_`, and `net_` variants (`gross_roas`, `proceeds_revenue`,
 `net_arpu`, and so on). To rank by a cohort metric with `--order-by`, use the expanded name
-— `--order-by gross_roas`, not `--order-by roas`. There is no `ltv` metric; see [Cohort
+— `--order-by net_roas`, not `--order-by roas`. Agent workflows use the `net_` variant and do
+not ask users to choose gross or proceeds. There is no `ltv` metric; see [Cohort
 windows](#cohort-windows).
 
 `metrics overview` accepts the root names only (`revenue`, `roas`, `arpu`, …) — no
@@ -79,17 +80,21 @@ The single most misunderstood part of the surface: there is no `ltv` metric. Lif
 is a cohort metric read at a renewal window, so `--by-days` is how you ask for a day-7 or
 day-90 value.
 
-- `--by-days` is repeatable, at most 16 windows per call, on both `metrics` and `metrics
-  overview`. Omit it entirely to get the dashboard's default figures instead of cohort
-  values.
+- `--by-days` applies only to the cohort roots `revenue`, `roas`, `arpu`, `arppu`, `arpas`, and
+  `roi`. It is repeatable, at most 16 windows per call, on both `metrics` and `metrics overview`.
+  Omit it entirely to get the dashboard's default figures instead of cohort values.
 - `--order-by-day` ranks rows by one of those windows — the way to get top campaigns by
   day-90 ROAS in a single call. Its value must be one of the windows passed to `--by-days`.
-- Ranking by a cohort metric takes the expanded name — `--order-by gross_roas`, not
-  `--order-by roas` — see [Metric vocabulary](#metric-vocabulary).
+- Ranking by a cohort metric takes the expanded name. Agent workflows use `--order-by net_roas`,
+  not `--order-by roas` — see [Metric vocabulary](#metric-vocabulary).
+- Apple spend, Adapty attribution, count, conversion, and cost metrics are not cohort-windowed.
+  In particular, `cost_per_paid` and `cost_per_trial` are values for the requested date window;
+  `--by-days` does not turn either into a day-X metric. Never ask for a cohort day, pass
+  `--by-days` / `--order-by-day`, or attach a `day-X` label for a non-cohort metric.
 
 ```sh
 adapty asa metrics --entity campaign --date-from 2026-07-01 --date-to 2026-07-31 \
-  --metric roas --by-days 7 --by-days 90 --order-by gross_roas --order-by-day 90
+  --metric roas --by-days 7 --by-days 90 --order-by net_roas --order-by-day 90
 ```
 
 ## The analytics pool

docs/agent/skills/adapty-cli-setup/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: adapty-cli-setup
 license: MIT
-description: Use when the Adapty CLI needs installing or authenticating before any Apple Search Ads work — a fresh Cowork or cloud session, "adapty: command not found", AuthRequiredError, an expired auth code, connecting an Apple Search Ads account, or a 402 ads_manager_subscription_required. Triggers on "install the Adapty CLI", "set up adapty", "adapty auth login", "connect Apple Search Ads", "not authenticated", or any Apple Ads request made in a session where the CLI is not yet installed.
+description: Use when the Adapty CLI needs installing or authenticating before any Apple Search Ads work — a fresh Cowork or cloud session, "adapty: command not found", AuthRequiredError, an expired auth code, or connecting an Apple Search Ads account. Triggers on "install the Adapty CLI", "set up adapty", "adapty auth login", "connect Apple Search Ads", "not authenticated", or any Apple Ads request made in a session where the CLI is not yet installed.
 ---
 
 # Adapty CLI setup
@@ -13,6 +13,24 @@ off to `apple-ads`. Nothing here spends money.
 from a previous session, and nothing in this one carries into the next. Never skip the run because
 setup "was already done".
 
+## Entry boundary
+
+Open this skill only when `adapty` is missing, `adapty asa whoami` returns
+`AuthRequiredError`, or Apple credentials need connecting. A failed preflight is not automatically
+an authentication failure:
+
+- `402 ads_manager_subscription_required` means authentication succeeded. Do not install or log in.
+- `NetworkError` means the CLI could not reach the API. Do not install or log in.
+- Repeated `failed to copy trust settings of system certificate-25291` lines mean a macOS sandbox
+  cannot read the system Keychain. The caller must suppress those lines, retry the same read once
+  with `NODE_USE_SYSTEM_CA=0`, then retry that read once outside the sandbox when the platform
+  explicitly supports it. Never route either failure into this setup flow.
+
+If both network retries fail, stop with exactly this user-facing message and no certificate dump:
+
+> Adapty API is unreachable from this sandbox. Allow network access for `adapty.io` and
+> `*.adapty.io`, then start a new task.
+
 ## Run this
 
 Two scripts do the whole job. Run step 1, surface what it prints, run step 2. Do not improvise
@@ -112,6 +130,7 @@ the dashboard usually means the token is scoped elsewhere.
 | `User code not found or expired` in the browser | The waiter died, or the TTL ran out | Step 1 again, and surface the new code faster |
 | `AuthRequiredError` after `AUTHED` | Almost always a stale `ADAPTY_TOKEN` | `unset ADAPTY_TOKEN`, re-check |
 | `402 ads_manager_subscription_required` | Authenticated fine; the company has no Ads Manager subscription | **Not a setup bug.** Say so plainly and stop. No flag works around it |
+| `NetworkError`, including certificate `-25291` noise | Network or sandbox trust-store access failed before authentication could be checked | Return to the caller's quiet preflight retry. Never install or log in |
 
 ## ADAPTY_TOKEN
 
@@ -145,6 +164,9 @@ printing the link.
 - **Never treat `adapty auth status` as proof of a working token.** It never touches the network;
   `adapty asa whoami` is the first call that asks the server.
 - **Never present a `402` as a broken install**, and never hunt for a flag that bypasses it.
+- **Never present a `NetworkError` as an authentication or install failure.** In particular, do not
+  reset Keychain, reinstall Xcode, run `sudo`, reinstall the CLI, or start `auth login` for
+  certificate `-25291` noise.
 - **Never echo a full token into chat**, and never write one into a file that could be committed.
 - **Never run a write command from this skill.** Setup is reads plus `asa connect`. A campaign, bid,
   budget or keyword means `apple-ads` owns the next step.

scripts/check-agent-docs.mjs

@@ -10,13 +10,16 @@ const INVENTORY_DOCS = [
   fileURLToPath(new URL('../docs/agent/asa-management.md', import.meta.url)),
   fileURLToPath(new URL('../docs/agent/asa-metrics.md', import.meta.url)),
 ]
+const SETUP_DOC = fileURLToPath(new URL('../docs/agent/skills/adapty-cli-setup/SKILL.md', import.meta.url))
+const METRICS_DOC = fileURLToPath(new URL('../docs/agent/asa-metrics.md', import.meta.url))
 const EXAMPLE_DOCS = [
   ...INVENTORY_DOCS,
-  fileURLToPath(new URL('../docs/agent/skills/adapty-cli-setup/SKILL.md', import.meta.url)),
+  SETUP_DOC,
 ]
 
 const errors = []
 const fail = (file, line, message) => errors.push(`${relative(ROOT, file)}:${line}: ${message}`)
+const normalizeNewlines = (text) => text.replaceAll(/\r\n?/g, '\n')
 
 if (!existsSync(MANIFEST)) {
   console.error('oclif.manifest.json is missing; run `pnpm build && pnpm exec oclif manifest` first.')
@@ -144,6 +147,41 @@ for (const file of EXAMPLE_DOCS) {
   }
 }
 
+const setupText = normalizeNewlines(readFileSync(SETUP_DOC, 'utf8'))
+const setupDescription = setupText.match(/^description:\s*(.+)$/m)?.[1] ?? ''
+const setupContracts = [
+  ['## Entry boundary', 'explicit setup entry boundary'],
+  ['`NetworkError` means the CLI could not reach the API. Do not install or log in.', 'NetworkError routing'],
+  ['failed to copy trust settings of system certificate-25291', 'Cowork certificate-noise signature'],
+  ['NODE_USE_SYSTEM_CA=0', 'system CA fallback'],
+  [
+    'Adapty API is unreachable from this sandbox. Allow network access for `adapty.io` and',
+    'concise network failure message',
+  ],
+]
+for (const [contract, label] of setupContracts) {
+  if (!setupText.includes(contract)) fail(SETUP_DOC, 1, `missing setup contract: ${label}`)
+}
+
+if (setupDescription.includes('402') || setupDescription.includes('ads_manager_subscription_required')) {
+  fail(SETUP_DOC, 1, '402 must not trigger the setup skill')
+}
+
+if (setupText.indexOf('## Entry boundary') > setupText.indexOf('## Run this')) {
+  fail(SETUP_DOC, 1, 'setup entry boundary must be read before install and login instructions')
+}
+
+const metricsText = normalizeNewlines(readFileSync(METRICS_DOC, 'utf8'))
+const metricContracts = [
+  ['`revenue`, `roas`, `arpu`, `arppu`, `arpas`, and\n  `roi`', 'complete cohort-window metric family'],
+  ['Agent workflows use the `net_` variant', 'net revenue-family workflow default'],
+  ['`cost_per_paid` and `cost_per_trial` are values for the requested date window', 'cost metric date-window semantics'],
+  ['`--by-days` does not turn either into a day-X metric', 'non-cohort day-X prohibition'],
+]
+for (const [contract, label] of metricContracts) {
+  if (!metricsText.includes(contract)) fail(METRICS_DOC, 1, `missing metrics contract: ${label}`)
+}
+
 if (errors.length > 0) {
   for (const error of errors) console.error(`ERROR ${error}`)
   console.error(`\n${errors.length} agent documentation drift error(s)`)