feat: store canonical ticket URLs, assemble affiliate wrapper at resolve time

[chubes4]

Problem

The Ticketmaster Discovery API returns a fully-assembled Impact Radius affiliate wrapper (our API key is affiliate-linked). Ticketmaster.php stored it verbatim into the Event Details block's ticketUrl attribute, freezing the affiliate ID, campaign ID, and ad ID into ~74,600 rows of post_content (74,643 published events with an evyy.net URL, verified by read-only query against blog 7 as of this PR). Rotating any of those IDs — affiliate ID reissue, network migration, a tracking-param change — was a mass content migration rather than a config edit.

The fix

• Store canonical, assemble at resolve time. The TM import handler (Ticketmaster.php) now stores the canonical vendor URL. inc/Core/ticket-destination.php adds one filterable config (data_machine_events_ticket_wrapper_config() — affiliate ID / campaign ID / ad ID / template / apply_to_hosts) and assembles the wrapper from it at resolve time (data_machine_events_assemble_affiliate_wrapper()).
• Read path tolerant of both shapes. ResolveTicketDestinationAbilities (and every compliance-gated render/REST surface) handles wrapper-stored rows (return as-is) and canonical-stored rows (assemble) identically — the backfill is therefore non-blocking and independently revertible.
• New dedicated backfill. TicketUrlCanonicalBackfillAbilities / TicketUrlCanonicalBackfillCommand — a sibling of TicketUrlResyncAbilities/-Command (not a modification of either) — converts wrapper-stored rows to canonical, dry-run by default.
• Byte-identity gate. A row is only converted when unwrap → re-assemble reproduces the normalized stored wrapper byte-for-byte. Otherwise it's reported skipped_roundtrip_mismatch (or mangled_unrecoverable for the known v0.8.39-era mangled shape) and left wrapper-stored. A partial, provably-safe migration beats a complete, possibly-lossy one.

Proof: rotating the affiliate ID is a config edit, zero content writes

AffiliateWrapperAssemblyTest::test_rotating_affiliate_id_changes_output_without_touching_content and TicketUrlCanonicalBackfillAbilitiesTest::test_affiliate_id_rotation_is_config_only add a filter on data_machine_events_ticket_wrapper_config, re-resolve the same event, assert the wrapper reflects the new ID, and assert post_content is byte-identical before/after. No wp_update_post call anywhere in the rotation path.

Byte-identity evidence

AffiliateWrapperAssemblyTest::test_reassembly_from_real_stored_values_is_byte_identical runs the full normalize → unwrap → assemble pipeline against real stored production wrapper values (fetched via read-only query against blog 7) across all three observed stored shapes (raw &, HTML-entity & — ~54% of rows, verified: 40,560 / 74,643 — and the legacy \u0026amp; combined artifact) and asserts byte-for-byte equality with the original.

I additionally ran a standalone, non-test verification harness (not part of the PHPUnit suite — a throwaway script, described in "Verification status" below) against 800 random real production rows, sampled fresh via read-only wp db query. Results:

• 100% byte-identical round-trip for every ticketmaster.com and ticketweb.com destination in the sample.
• The only mismatches were a long tail of one-off third-party box-office hosts (axs.com, evenue.net, etix.com, eventim.us, dice.fm, seatgeek.com, universe.com, tixr.com, prekindle.com, fgtix.com) — 1-4 rows each — which correctly fail the round-trip guard because they're not in apply_to_hosts and stay wrapper-stored. Zero silent corruption anywhere in the sample.

A real gap found and fixed during verification: ticketweb.com

The original apply_to_hosts list shipped with only ticketmaster.com, on the assumption (stated in a code comment) that it was "the only vendor whose URLs arrive affiliate-wrapped via the Discovery API today." That assumption was wrong. A read-only production query showed ticketweb.com — Ticketmaster's own sub-brand for smaller venues — is ~33% of the entire wrapped corpus (24,620 of 74,643 published rows). Under the original list, every one of those rows would have permanently failed the round-trip guard (not because of an encoding problem — the guard's own safety net caught this correctly — but purely because the destination host wasn't in the allowlist) and stayed wrapper-stored forever, defeating a third of the migration's purpose.

Fixed: apply_to_hosts now includes both ticketmaster.com and ticketweb.com. Verified against the 800-row sample: round-trip match rate went from 524/798 (66%) to 779/799 (97.5%) once ticketweb.com was added, with the remaining ~2.5% being the genuine long-tail third-party hosts described above, which are deliberately not added (an unbounded, ever-growing per-venue host allowlist is not "vendors we actually monetize," it's guessing at an open set). New test coverage: test_ticketweb_reassembly_is_byte_identical (including the ?REFERRAL_ID=tmfeed-bearing shape) and test_third_party_box_office_hosts_are_not_wrapped.

Regression check: #820 Ticketmaster-compliance guarantees

Before touching anything, I diffed the working tree's render.php and ResolveTicketDestinationAbilities.php against origin/main (which already has #820 merged) to confirm the prior agent session hadn't weakened them. Result: no regression.

• No-href gating on affiliate buttons — intact. render.php still emits data-ticket-ref, no destination URL in markup, <a ... data-ticket-ref="..." role="button" ...> with no href attribute for gated tickets. The only change is swapping the gating predicate from data_machine_events_is_affiliate_ticket_url() to the new, broader data_machine_events_is_gated_ticket_url() — needed so a canonical-stored monetized row (post-backfill) still gates instead of silently demoting to a direct href.
• normalizeResolvedUrl() entity decoding (& and \u0026) — intact and generalized: the logic moved verbatim into the shared data_machine_events_normalize_ticket_url_entities() in ticket-destination.php so the resolve ability, the backfill, and any future consumer share one implementation instead of private copies.
• <noscript> permalink fallback — untouched, still present at the same location in render.php.

DisplayVars, the Calendar REST controller (Calendar.php), and render.php all got the same one-line predicate swap (is_affiliate_ticket_url → is_gated_ticket_url), each with an inline comment explaining why.

How the read path stays tolerant of both shapes, and why the backfill is revertible

ResolveTicketDestinationAbilities::resolveTicketUrl():

• Wrapper-stored URL → data_machine_events_is_affiliate_ticket_url() is true → returned as-is (pre-Store canonical ticket URLs and assemble affiliate wrappers at resolve time #818 behavior, unchanged).
• Canonical-stored URL for a monetized vendor → data_machine_events_assemble_affiliate_wrapper() re-assembles the wrapper from config.
• Direct URL to a non-monetized vendor → passes through unchanged either way.

Both stored shapes resolve to the identical wire destination, so: the backfill can run partially, be interrupted, resumed, or reverted (a converted row can simply be left canonical — both shapes resolve identically — or the wrapper can be restored from the backfill's own before/after change log) without breaking any consumer mid-migration.

Existing helpers reused (nothing reimplemented)

• datamachine_unwrap_affiliate_url() (event-dates-sync.php) — extraction, unchanged.
• data_machine_events_is_affiliate_ticket_url() / data_machine_events_affiliate_ticket_hosts() (affiliate-links.php) — wrapper-shape detection, unchanged.
• ResolveTicketDestinationAbilities::normalizeResolvedUrl()'s entity-decoding logic — moved verbatim (not rewritten) into the new shared data_machine_events_normalize_ticket_url_entities().
• EventDateQueryAbilities::executeQueryEvents() — the backfill's event traversal, same query surface TicketUrlResyncAbilities uses (scope/status/per_page/order), not a new query path.

Backfill usage

wp data-machine-events backfill-canonical-ticket-urls                          # dry run (default)
wp data-machine-events backfill-canonical-ticket-urls --future-only --execute  # recommended first pass
wp data-machine-events backfill-canonical-ticket-urls --limit=500 --execute    # chunked
wp data-machine-events backfill-canonical-ticket-urls --format=json           # scripting

Dry-run is the default (dry_run defaults to true in both the ability input schema and the CLI). Recommended rollout: dry-run first, then --future-only --execute over the ~24,941 live, revenue-bearing events (future-dated as of issue filing), then the archival tail in slower --limit-chunked passes. Idempotent/resumable: a converted row is no longer wrapper-shaped, so re-running (or resuming after an interruption) reports it skipped_not_wrapper and moves on.

#816 compliance guarantee confirmed intact

The affiliate ID appears in zero rendered HTML and zero JS bundles, before and after this change:

• No JS files were touched by this PR (confirmed by git diff --stat — only PHP and test files changed).
• render.php's gated branch still emits no href at all for a gated ticket (data-ticket-ref only); the affiliate wrapper (or the config that would assemble it) never reaches the HTML response.
• The resolve-ticket-destination ability (which does carry the assembled wrapper) has show_in_rest: false and is deliberately not the generic REST-visible ability surface — it backs the first-party redirect endpoint server-side only.

Follow-up issue: datamachine_unwrap_affiliate_url() double-decode

Design ruling from this task: datamachine_unwrap_affiliate_url() double-decodes (parse_str + urldecode), so wrappers with nested percent-encoding (the SeatGeek shape, dd_referrer=https%253A%252F%252F...) don't round-trip. That's out of scope here — its output feeds duplicate-detection identity, and changing decode behavior could silently alter dedup matching across the corpus. I initially filed this as #827, but it turned out to duplicate #824, filed earlier by a prior session on this same task before a respawn. #827 is now closed and consolidated into #824 — that's the tracking issue this PR links to.

Merge-order dependency: #826 — rebased and resolved

Sibling PR #826 (issue #823, corrupted-redirect detector) merged to main (squash 0308076) while this PR was in flight, also touching inc/Abilities/ResolveTicketDestinationAbilities.php. It extracted the \u0026 + wp_specialchars_decode() pair out of normalizeResolvedUrl() into a new shared helper, datamachine_decode_stored_url_artifacts() (event-dates-sync.php), so that logic isn't duplicated a third time.

I rebased this branch onto post-#826 main and resolved the conflict by consuming that helper rather than keeping this PR's own copy:

• Removed data_machine_events_normalize_ticket_url_entities() entirely from ticket-destination.php (it was a verbatim duplicate of the same two-step decode datamachine_decode_stored_url_artifacts() now owns).
• ResolveTicketDestinationAbilities::normalizeResolvedUrl() and TicketUrlCanonicalBackfillAbilities's entity-normalization step both now call datamachine_decode_stored_url_artifacts() directly.
• grep -rn "datamachine_decode_stored_url_artifacts" inc/" confirms every consumer in this PR's diff calls the shared helper; there is no duplicated wp_specialchars_decode + \u0026 pair anywhere this PR touches.
• Verified post-rebase: data_machine_events_is_gated_ticket_url() (the predicate swap that prevents silent revenue loss on migrated rows — see the note at the top of this PR) survived intact in render.php, DisplayVars, and Calendar.php. The Ticketmaster compliance: JS-gate affiliate ticket links so they are not in raw HTML #816 compliance guards (data-ticket-ref with no href on the gated path, the <noscript> permalink fallback, entity normalization on both resolveTicketUrl() return paths) are all still present and unchanged.
• Re-ran the full local PHPUnit suite against a fresh, isolated origin/main worktree (post-fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826) as the baseline and compared against this rebased branch: 1232 tests on the branch, 1177 on baseline (the +55 are this PR's and fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826's own new tests), identical set of 62 pre-existing failing test names on both sides — zero new failures.
• Re-ran homeboy review lint --changed-since origin/main: zero findings.

Verification status

Confirmed:

• Read-only production queries (blog 7, events.extrachill.com) verify: 74,643 published events with an evyy.net ticketUrl; 40,560 (54.35%) HTML-entity-encoded; the exact affiliate ID (1191134) / campaign ID (264167) / ad ID (4272) baked into real stored rows match the default config exactly; ticketweb.com is 24,620 of those 74,643 rows.
• A standalone, throwaway (non-committed) PHP verification harness confirmed byte-identical round-trip against 800 random real production wrapper values, isolating only the pure functions under test (no WordPress writes, read-only wp db query + wp eval-file only).
• CI (homeboy review test, GitHub Actions, managed WP Codebox sandbox): passing — confirmed on the commit before the final rebase; will re-confirm on the rebased HEAD once this update's CI run completes.
• CI (homeboy review lint): passing — zero findings across phpcs, eslint, phpstan (level 7).
• CI (homeboy review audit): initially failed with one genuine finding — an intra-method-duplication in the new backfill ability's registerAbility() (the changes/report output_schema item shapes shared an identical 6-line post_id/title prefix before diverging). Fixed by extracting the shared prefix into a local variable and array_merge()-ing the divergent fields, rather than repeating the literal block. Re-verified locally with homeboy review lint --changed-since origin/main (unaffected — zero findings) and the affected tests (47/47 pass); the fix is pushed and awaiting this update's CI run.
• Full local PHPUnit run via a native bootstrap (not the WP Codebox managed sandbox, mirroring it closely) against a real local MariaDB test database, run twice — once pre-rebase against origin/main at ee52bd5, once post-rebase against origin/main at 0308076 (post-fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826) — with a matching from-scratch baseline built in an isolated git worktree both times: zero new failing/erroring test names in either comparison (diff of the sorted failure-name lists is empty both times). Post-rebase: 1232 tests on the branch vs 1177 on baseline (the +55 are this PR's and fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826's own new tests), identical 62 pre-existing failure names on both sides.
• php -l clean on every touched file.
• No JS/CSS files touched by this diff (git diff --stat confirms PHP + test files only).
• The fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826 merge-order dependency: rebased and resolved by consuming datamachine_decode_stored_url_artifacts() — see the dedicated section above for detail and verification.

All three CI gates are green on the current HEAD (homeboy review test, homeboy review lint, homeboy review audit — run). The audit gate required two follow-up commits after the initial submission:

1. An intra-method-duplication finding in the new backfill ability's registerAbility() — the changes/report output_schema item shapes shared an identical 6-line post_id/title prefix. First fix only deduped the shared values, leaving the surrounding array-wrapper structure still duplicated; audit caught that too, on the next run. Second fix extracted a real buildPostKeyedItemSchema() helper so no literal block is repeated at all (not just no duplicate values) — that's the one CI is now green against.
2. The fix: detect and repair affiliate redirects with punctuation-stripped destinations (#823) #826 merge-order rebase described above.

Marking ready for review.

AI disclosure

Authored by Extra Chill Bot (AI agent) via Kimaki session.

Closes #818

[homeboy-ci]

Homeboy Results — data-machine-events

Review audit

✅ review audit — passed

Deep dive: homeboy review audit data-machine-events --changed-since 0308076

Artifacts and drill-down

• CI results artifact: homeboy-ci-results-data-machine-events-review-audit-homeboy-Linux-php8.2-node24 contains immediate command JSON for this action invocation.
• Observation artifact: homeboy-observations-data-machine-events-review-audit-homeboy-Linux-php8.2-node24 contains exported Homeboy run history for deeper queries.
• Drill-down: download the observation artifact, then run homeboy runs import <dir>, homeboy runs list, and homeboy runs findings <run-id>.
• Artifacts are attached to the workflow run: https://github.com/Extra-Chill/data-machine-events/actions/runs/34551383324

Review lint

✅ review lint — passed

ℹ️ Full options: homeboy self docs commands/lint
Deep dive: homeboy review lint data-machine-events --changed-since 0308076

Artifacts and drill-down

• CI results artifact: homeboy-ci-results-data-machine-events-review-lint-homeboy-Linux-php8.2-node24 contains immediate command JSON for this action invocation.
• Observation artifact: homeboy-observations-data-machine-events-review-lint-homeboy-Linux-php8.2-node24 contains exported Homeboy run history for deeper queries.
• Drill-down: download the observation artifact, then run homeboy runs import <dir>, homeboy runs list, and homeboy runs findings <run-id>.
• Artifacts are attached to the workflow run: https://github.com/Extra-Chill/data-machine-events/actions/runs/34551383324

Tooling versions

• Homeboy CLI: homeboy 0.371.8+0535d130a6ca161b2a851e8ebf05f7fd2e600fee
• Extension: wordpress from https://github.com/Extra-Chill/homeboy-extensions
• Extension revision: e03732de
• Action: Extra-Chill/homeboy-action@v2

[chubes4]

All three gates green — merging

audit pass, lint pass, test pass (run 34551383324).

Rebase verification

I checked the conflict resolution rather than trusting it, since this branch and #826 both rewrote ResolveTicketDestinationAbilities.php:

Check	Result
data-ticket-ref gated path (no href in markup)	✅ intact
<noscript> permalink fallback	✅ intact
is_gated_ticket_url() revenue guard	✅ survived
Consumes datamachine_decode_stored_url_artifacts()	✅ 4 references
No duplicated decode logic	✅ only a docblock mention remains

You correctly consumed #826's extracted helper instead of restoring an inline copy. That was the specific thing I was worried about in the conflict.

The important part

is_gated_ticket_url() is the subtlest change in this PR and worth restating for anyone reading the history later. Once the backfill migrates a row to canonical storage, the stored value is a plain ticketmaster.com URL — so an is_affiliate_ticket_url() check returns false, the button ungates, and it links straight to Ticketmaster with no affiliate wrapper at all. Silent revenue loss on every migrated row, caused by the very migration meant to make affiliate config editable.

Gating on "routes through the first-party redirect" instead of "is stored as an affiliate URL" is the correct invariant, and it is what makes the migration safe.

Design that held up

The byte-identity gate is the right call: only convert a row when unwrap → assemble reproduces the stored wrapper byte-for-byte, otherwise skipped_roundtrip_mismatch and leave it wrapper-stored. That makes attribution drift structurally impossible rather than merely unlikely, and it bounds the migration to rows we can prove are safe. Rows that cannot round-trip stay in the shape already proven in production — still monetized, still compliant.

Follow-up correctly filed and linked rather than fixed in-scope: #824 (the datamachine_unwrap_affiliate_url() double-decode). Changing that would alter duplicate-detection identity across the whole corpus, which is a far larger blast radius than this PR.

That is now three tracked instances of the same pattern — #821, #824, and the organizerUrl corruption in #829 — all comparison-oriented primitives being reused for redirect-oriented purposes. Worth naming as a class.