Harden TanStack performance, streaming, feeds, and Convex caching

.github/workflows/ci.yml

@@ -11,9 +11,18 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: "24.21.0"
       - uses: oven-sh/setup-bun@v2
         with:
-          bun-version: 1.3.8
-      - run: bun install
+          bun-version: 1.4.3
+      - run: bun install --frozen-lockfile
       - run: bun run check
       - run: bun run typecheck
+      - run: bun test
+      - run: bun run build
+        env:
+          NODE_ENV: production
+          CONVEX_URL: https://good-elk-277.convex.cloud
+          VITE_CONVEX_URL: https://good-elk-277.convex.cloud

.gitignore

@@ -1,4 +1,5 @@
 node_modules
+.bundle-stats/
 dist
 .output
 .cache

HARDENING.md

@@ -0,0 +1,48 @@
+# Convex hardening and rollout
+
+## Deploy order and compatibility
+
+Deploy the Convex functions and schema before deploying the frontend to enable cursor pagination. New SSR responses carry `paginationVersion: 1`; only then does the frontend subscribe through `getEpisodesPage`. Older deployments omit that marker, so the frontend keeps using the existing bounded `getPodcastWithEpisodes` subscription and never calls an API that has not been deployed.
+
+Transfer mutations (`upsertTransfer`, `deleteTransfer`, and `clearTransferPhoto`) are internal-only. Trusted administrators can use authenticated Convex CLI or dashboard tools with the internal function reference. Public transfer reads remain available to the site.
+
+## Refresh leases and cache behavior
+
+Football resource TTLs, RSS's five-minute TTL, the cron schedule, RSS ingestion, and snapshot merge behavior are unchanged. Soccer snapshot and RSS refreshes use atomic leases in `refreshLocks`, separate from cache entries. A lease lasts 120 seconds; failed owners leave a 30-second retry cooldown and do not update the cache. The token fences cache completion and lease release, preventing stale owners from overwriting cache entries or releasing a successor's lease. RSS episode ingestion retains its existing independent upserts and is not fenced by this cache lease.
+
+Cache payload freshness is parsed at action time. Soccer treats a future expiry as fresh only when its JSON payload parses to a non-null value; the atomic claim mutation repeats this check. RSS metadata does not require a payload. Stale soccer data remains available during refresh and failure.
+
+## Episode pagination
+
+`getPodcastWithEpisodes` is retained for SSR and old clients, defaults to ten rows, accepts safe integer limits from 1 to 1,000, and fetches one extra row to report `hasMore`. `getEpisodesPage` uses descending `by_podcastId_and_number` cursor pages of 1–100 items. New responses include only the podcast name on each episode, not the full parent document; they no longer calculate a full episode count. The latest-episode fallback uses descending `by_podcastId_and_publishedAt`.
+
+## Remaining collection reads
+
+Episode feeds and latest-episode reads use bounded indexed queries. The remaining `.collect()` calls are limited to the curated transfer table (public full/team reads), the small cache table during expiry cleanup, and explicit internal episode administration (clear-all, count, duplicate audit). The admin operations are not invoked by normal refreshes or tests; their dataset-wide scope is intentional and separate from hot feed queries.
+
+## Dependencies and verification
+
+The unused `next-themes` and direct router-plugin dependencies were removed; the three TanStack devtools packages are development-only. `convex-test` is pinned to `0.0.61` for offline function tests; existing `happy-dom@20.14.6` supports DOM regressions without adding Vitest.
+
+`bun run typecheck` checks the root application and test files, then checks deployable Convex code with `tsc -p convex/tsconfig.json`. The Convex project excludes `*.test.ts`; those Bun tests remain covered by the root TypeScript project and `bun test`.
+
+From the repository root, with Bun 1.4.3 and Node 24.21.0:
+
+```powershell
+$env:NODE_ENV = "development"
+bun test convex/hardening.test.ts
+bun test src/features/podcast/feed-dom.test.tsx src/lib/html.test.ts src/server/chat-stream.test.ts
+bun run typecheck
+bun run lint
+bun run check
+bun test
+
+$env:NODE_ENV = "production"
+$env:CONVEX_URL = "https://good-elk-277.convex.cloud"
+$env:VITE_CONVEX_URL = "https://good-elk-277.convex.cloud"
+bun run build
+```
+
+The tests use the in-memory `convex-test` backend and local fixtures. They do not deploy Convex or contact football-data.
+
+Native schema, query, and mutation verification was also run against an anonymous local Convex deployment in a separate copy of commit `ce5d7386456509e3f0f421bccb4cbc089849c86d`. Only that validation copy omitted `convex/crons.ts` to prevent scheduled actions; fixture calls used the loopback backend and did not invoke external services.

README.md

@@ -2,16 +2,45 @@
 
 [![E2E Tests](https://github.com/ItamarShDev/hakapit/actions/workflows/e2e-tests.yml/badge.svg?branch=main)](https://github.com/ItamarShDev/hakapit/actions/workflows/e2e-tests.yml)
 
-To install dependencies:
+## Local development
 
-```bash
-bun install
+Install the locked dependencies and start the Vite development server:
+
+```powershell
+$env:NODE_ENV = "development"
+bun install --frozen-lockfile
+bun run dev
+```
+
+## Comparison checks
+
+Run the repository checks, typecheck, and test suite:
+
+```powershell
+$env:NODE_ENV = "development"
+bun run check
+bun run typecheck
+bun test
 ```
 
-To run:
+Build and start the local production server with the existing public Convex backend:
 
-```bash
-bun run
+```powershell
+$env:NODE_ENV = "production"
+$env:CONVEX_URL = "https://good-elk-277.convex.cloud"
+$env:VITE_CONVEX_URL = "https://good-elk-277.convex.cloud"
+bun run build
+node .output/server/index.mjs
 ```
 
-This project was created using `bun init` in bun v1.2.18. [Bun](https://bun.sh) is a fast all-in-one JavaScript runtime.
+The build writes the server entry point to `.output/server/index.mjs` and static assets to `.output/public`. The endpoints use the existing backend and may refresh normal feed data.
+
+## Optimized app behavior
+
+- Chat replies stream incrementally over server-sent events. The server forwards cancellation to the provider, reports safe errors (including rate limits and incomplete streams), and does not automatically retry after sending output. Retry is an explicit user action.
+- Podcast feeds retain their last resolved snapshot if a live Convex query errors. The feed exposes a retry action to resubscribe, keeps the snapshot visible during recovery, and distinguishes loading from empty feeds.
+- Podcast images use responsive size hints and promote only the initial or viewport-visible feed cards to high priority. Feed and latest-episode playback controls expose descriptive labels, and HTML descriptions are sanitized at render time.
+
+See [TESTING.md](./TESTING.md) for comparison revisions, commands, and pending browser/performance evidence. Benchmark and scoring work is maintained outside this repository.
+
+See [HARDENING.md](./HARDENING.md) for Convex rollout order, cursor pagination, refresh leases, and offline verification.

TESTING.md

@@ -1,55 +1,69 @@
-# Testing with Playwright
+# Testing and comparison builds
 
-This project uses Playwright for end-to-end testing with comprehensive coverage of the Hebrew RTL web application.
+This TanStack lab uses Bun's test runner. It does not include browser automation.
 
-## Quick Start
+## Runtime requirements
 
-```bash
-# Install dependencies
-bun install
+- Node.js 24.3 or newer (CI and local comparison builds use 24.21.0)
+- Bun 1.4.3
 
-# Install Playwright browsers
-bun run test:e2e:install
+## Revisions
 
-# Run all tests
-bun run test:e2e
+- Baseline: `202cc44f0a89942f99295bc19e6f4b9371ab7458`
+- Optimized implementation: `2fcb83727d2e7d3d7b9bbca6a21418b770e5f639`
 
-# Run tests in UI mode (interactive)
-bun run test:e2e:ui
+The baseline is an untouched worktree. Install each revision from its own
+frozen lockfile and use the same Node, Bun, and public existing Convex backend
+for comparison builds.
 
-# Run tests in headed mode (visible browser)
-bun run test:e2e:headed
+## Install and verify
 
-# Run tests on specific browser
-bun run test:e2e:chrome
-bun run test:e2e:firefox
-bun run test:e2e:safari
+Run these commands from the root of either checkout:
 
-# Run responsive tests
-bun run test:e2e:responsive
+```powershell
+$env:NODE_ENV = "development"
+bun install --frozen-lockfile
+bun run check
+bun run typecheck
+bun test
+```
 
-# Run Hebrew RTL tests
-bun run test:e2e:hebrew
+`bun run check` runs Oxfmt and Oxlint. Its formatter currently reports
+pre-existing repository-wide issues in the untouched baseline; this work
+formats changed files only. `bun run lint` runs Oxlint separately.
+Focused coverage for the comparison changes:
 
-# View test report
-bun run test:e2e:report
+```powershell
+bun test src/lib/home-snapshot.test.ts src/lib/html.test.ts src/features/chat/floating-chat.test.ts src/layouts/header/menu.test.ts src/features/podcast/feed.test.ts
 ```
 
-## Test Structure
+## Build and start the production server
 
-- **`tests/page-objects/`** - Page Object Models for maintainable test code
-- **`tests/mocks/`** - API mocking infrastructure for deterministic testing
-- **`tests/homepage.spec.ts`** - Core homepage functionality tests
-- **`tests/responsive.spec.ts`** - Mobile/tablet/desktop responsive design tests
-- **`tests/hebrew-rtl.spec.ts`** - Hebrew language and RTL direction tests
+Use the same public existing Convex backend URL for the server and browser
+bundles:
 
-## CI/CD
+```powershell
+$env:NODE_ENV = "production"
+$env:CONVEX_URL = "https://good-elk-277.convex.cloud"
+$env:VITE_CONVEX_URL = "https://good-elk-277.convex.cloud"
+bun run build
+node .output/server/index.mjs
+```
 
-Tests automatically run on every GitHub commit across multiple browsers and devices. See `.github/workflows/e2e-tests.yml` for configuration.
+The build emits static assets to `.output/public` and the Node server entry to
+`.output/server/index.mjs`. Run the start command from the same checkout root.
 
-## Current Status
+## Optimization rationale
 
-✅ **36 core tests passing** - Critical user flows validated  
-🔄 **32 tests skipped** - Optimization opportunities for future iteration
+- The SSR-visible chat launcher loads the heavier chat UI only on interaction
+  intent, keeping its Markdown and chat dependencies out of the initial client
+  entry.
+- Podcast and transfer reads cross validated server-function boundaries; the
+  homepage requests remain concurrent with independent fallbacks, and live
+  Convex subscriptions remain client-side.
+- Font subsets and image priorities are narrowed to the rendered surfaces;
+  podcast HTML is sanitized at its render sinks, and player/chat placement and
+  controls account for mobile layout and accessibility.
 
-The foundation is production-ready with solid coverage of page loading, Hebrew text rendering, component visibility, and responsive design.
+Browser and visual verification, as well as performance evidence, are pending.
+This repository contains no benchmark harness or scoring claims.