Skip to content

lessons.md

Latest commit

 

History

History
69 lines (66 loc) · 28.8 KB

lessons.md

File metadata and controls

69 lines (66 loc) · 28.8 KB
Raw

Development Lessons

  • HTML2Canvas Visibility Quirk: Never trust html2canvas to correctly paint elements positioned at -9999px out of the browser viewport. However, never physically move the hidden element into the active viewport using hacks like position: fixed and z-index: 9999 before capturing, as this causes the enormous element to flash/spill visually across the user's screen during the 100ms it takes to snapshot. Instead, leave the real element strictly behind -9999px and use the html2canvas onclone Hook to safely target the cloned iframe sandbox element (clonedDoc.getElementById), modifying the inner cloned style.position without touching the real user DOM.

  • Balancing Quality vs File Size in jsPDF: Increasing html2canvas scale (e.g., from 2 to 3 or 4) sharply enhances HD rendering for text, but bloats file sizes organically. To offset this penalty when writing the PDF via jsPDF, never use image/png if lossless quality isn't strictly required. Use image/jpeg with .toDataURL('image/jpeg', 0.7) and inject the image natively to the PDF using the 'FAST' alias (pdf.addImage(imgData, 'JPEG', 0, 0, pdfWidth, pdfHeight, undefined, 'FAST')). This results in a much smaller output byte-count but retains the sharpness of the original 3x scaled text.

  • jsPDF White Borders (Sub-pixel scaling): Browsers round element dimensions natively. This creates cases where a generated PDF will exhibit a microscopic 1px white border at the bottom/side. To completely eliminate this, lock the html2canvas render state to scrollY: 0 and scrollX: 0, and explicitly add a +2 offset mapping to the target jsPDF rendering height/width (pdf.addImage(..., pdfWidth + 2, pdfHeight + 2)).

  • Dual Auth Flow Awareness: KlikForm supports TWO Google integration methods — OAuth ("Connect with Google") and Manual Service Account keys. Before modifying any Google-related UI, always check which flow the feature belongs to. The Google Sheet URL + Copy Service Email button in the builder is specifically for manual key users only, not OAuth users.

  • Next.js 16 Proxy Migration: middleware.ts is deprecated in Next.js 16 to distinguish route interception from traditional Express middleware. The file must be renamed to proxy.ts and the main exported function must be export async function proxy(...).

  • Handling Third-Party Hydration Mismatches: Browser extensions frequently inject arbitrary attributes (e.g., data-jetski-tab-id) into the <html> or <body> tags before React finishes hydrating. This causes a hard hydration mismatch error in strict mode. Always apply suppressHydrationWarning to the root <html> and <body> tags in app/layout.tsx to safely ignore these uncontrollable third-party mutations.

  • Radix UI Hydration with Next.js App Router: When using UI components that rely heavily on useId() for internal accessibility linkages (like shadcn's NavigationMenu), placing them directly inside deeply nested Server Components (like app/page.tsx) can trigger random "Hydration Mismatch" errors because the server-generated IDs may not align perfectly with the client's initial render tree. To resolve this, always extract complex interactive UI components into a dedicated "use client" component boundary to isolate their hydration lifecycle.

  • bg-clip-text Descender Clipping: CSS background-clip: text with text-transparent clips the gradient to the text paint area, which on large headings (5xl+) cuts off descender strokes of letters like "g", "y", "p". Always add explicit pb-4 (padding-bottom) on gradient text headings to extend the clipping box past the descender line.

  • Deceptive 42P01 PostgREST Errors (search_path mutation): If Supabase suddenly throws an Error 42P01: relation "table_name" does not exist on a table that you know exists and has correct RLS policies, it is highly likely that a Postgres Trigger function attached to that table has a broken search_path. Using Supabase's Security Advisor recommendation to patch functions with ALTER FUNCTION ... SET search_path = '' will intentionally "blind" the function from finding tables in the public schema unless they are explicitly prefixed (e.g., public.forms). If the trigger function is built without schema prefixes, the search_path = '' restriction causes the entire INSERT/UPDATE operation to fail with a misleading 42P01 error indicating the primary table (not the trigger's internal reference) is missing. Revert with ALTER FUNCTION ... RESET search_path; and NOTIFY pgrst, 'reload schema' to restore functionality.

  • PostgREST Schema Caching: When adding new columns directly via SQL migrations to a live Supabase database, the changes are not immediately visible to the PostgREST API layer. Any API upserts containing the new columns will fail with 42P01 relation does not exist. Always execute NOTIFY pgrst, 'reload schema' immediately after schema alterations to flush the stale API cache.

  • Always sync feature lists across all surfaces. When updating pricing features, remember there are TWO places to update: app/pricing/page.tsx (public pricing page) AND components/pricing-modal.tsx (in-dashboard upgrade modal). Missing one causes inconsistency between what the public sees and what logged-in users see.

  • CSS flex-1 placement matters in card layouts. When pricing/plan cards have different content lengths, putting flex-1 on the top section (header/price area) creates ugly empty gaps. Always apply flex-1 to the bottom section (feature list + button) so buttons align at the bottom and the price section stays compact.

  • Turbopack cache can cause 404s unexpectedly. If a route suddenly returns 404 even though page.tsx exists, restart the dev server first. If that doesn't work, delete the .next folder and restart. This is a known Turbopack caching issue with Next.js 16.

  • KlikForm complete feature set (marketing reference): Online Forms (Google Sheets integration), Dynamic QR Code Generator, URL Shortener, E-Certificate Generation (auto-generated & emailed to participants), Certificate Verification System (unique serial number + QR code for authenticity).

  • Public-facing queries must bypass RLS: Any Supabase query that serves public/unauthenticated users (e.g., getCertificateTemplatePublic) must use createAdminClient() (service role), NOT createClient() (auth-aware SSR). The auth-aware client respects RLS policies gated on auth.uid(), which is null for unauthenticated visitors — silently returning empty results. This causes features to appear broken on mobile/logged-out devices while working perfectly for the logged-in owner on desktop.

  • **Use COUNT queries for limit checks, not SELECT ***: When checking if a user has exceeded a resource limit (e.g., max certificates), never do getCertificateTemplates().length which fetches ALL rows including heavy JSON columns. Instead, use a dedicated count function with select('*', { count: 'exact', head: true }) which returns only the count without transferring any row data. This is especially important when rows contain large payloads like certificate element arrays.

  • Google Sheets Header Syncing (Data Dropping): When writing data to Google Sheets using google-spreadsheet, sheet.addRow(data) relies entirely on the pre-existing header columns to map the object keys. If a user adds a new field to their dynamic form, the payload will contain new keys, but the sheet will quietly ignore them if the new headers don't exist in row 1. To solve "missing data" upon form updates, always explicitly diff the incoming keys against the current headers, and update the sheet with sheet.setHeaderRow([...headers, ...missingHeaders]) before appending the row.

  • react-joyride V3 Stacking Contexts: When using react-joyride V3, the portal mounts directly to document.body by default. Attempting to force the overlay zIndex higher than a sticky header by overriding Joyride's internal styles will often cause the overlay to cover its own tooltip because they share the same stacking container. Instead of battling internal portal z-indexes, dynamically downgrade the sticky header's z-index (e.g. runTour ? "z-0" : "z-10") while the tour is active, allowing Joyride's default z-100 portal to naturally cover it.

  • react-joyride V3 ScrollOffsets: The scrollOffset property is no longer accepted as a top-level prop on the <Joyride> component in V3. Passing it there will be silently ignored, causing the browser to scroll targets tight to the top of the viewport (offset 20px), causing them to get trapped under sticky headers. You must explicitly map the scrollOffset property onto the individual items inside the steps array (e.g. steps.map(step => ({ ...step, scrollOffset: 150 }))).

  • Stale lint/build artifacts mislead: When committed, files like lint_output.txt or build_output.txt look authoritative but are snapshots from a different machine/branch state. They can show problems that no longer exist (or hide problems that do). Always re-run npm run lint / npm run build directly before drawing conclusions. Add *_output.txt, build-log.txt, tsconfig.tsbuildinfo to .gitignore and untrack them via git rm --cached.

  • Optional dependencies via dynamic import need runtime string syntax: Writing await import('@upstash/redis').catch(() => null) still causes TypeScript to fail the build with "Cannot find module" because the literal string is statically analyzed. Workaround: assign the module path to a variable first (const m = '@upstash/redis'; await import(m).catch(...)). The module stays optional at runtime and the TS check passes.

  • Next.js 16 proxy.ts IS the path forward: Earlier internal lessons (now revised) suggested proxy.ts was "ignored" by Next.js 16. That was wrong — the migration failed for an unrelated reason. The official codemod (npx @next/codemod@canary middleware-to-proxy .) is just a file rename + function rename. After migrating, the build's deprecation warning disappears.

  • Sentry should be conditional in build config: Hard-wrapping with withSentryConfig causes failures on PR previews / contributors who don't have SENTRY_AUTH_TOKEN. Wrap conditionally: export default (DSN && AUTH_TOKEN) ? sentryConfig : nextConfig. Inside sentry.*.config.ts, gate Sentry.init() on the DSN being present.

  • Production trace sampling: tracesSampleRate: 1 (100%) in production explodes Sentry costs. Use process.env.NODE_ENV === 'production' ? 0.1 : 1 so dev gets full visibility while prod samples 10%.

  • Three places to update when changing rate limiting: lib/rate-limit.ts (core), actions/forms.ts (form submission), actions/certificates.ts (cert verification). Don't introduce new ad-hoc Map<string, ...>() instances per file — use the shared checkRateLimit(ip, RATE_LIMITS.<bucket>, 'name') API. Each bucket is namespaced via the third arg so different features don't share counters.

  • Schema-qualify in SET search_path = '' functions: When applying Supabase Security Advisor's search_path = '' to trigger/SECURITY DEFINER functions, every table reference in the function body MUST be schema-qualified (e.g. public.forms, auth.users). Without the prefix, the empty search_path causes 42P01 relation does not exist. The proper fix is to update function bodies, not to RESET search_path (which silences the warning by reintroducing the vulnerability).

  • Server Action modules: every export must be async AND types can't be re-exported: Files marked 'use server' are treated as Server Action modules by Next.js 16 + Turbopack. Synchronous helpers (function aggregate(...)) cause "Server Actions must be async functions". Re-exporting types via export type { Foo } from a server-action file silently builds in some cases but fails under Turbopack with "Export Foo doesn't exist in target module". Solution: keep pure helpers + types in a separate lib/<feature>/ file; server-action wrappers import from there. Never re-export types from a 'use server' module — import them directly from the lib path.

  • Privacy-first analytics: When storing visitor analytics in a SaaS multi-tenant app, hash IPs with a daily-rotating salt (SHA256(ip + YYYY-MM-DD + secret)). This gives accurate per-day unique-visitor counts without ever storing raw PII or enabling long-term cross-day tracking. Coarse-grain User-Agent into device family (mobile/desktop/tablet/bot) rather than storing the full string. Insert via service role from the public client (since visitors are anonymous), but force RLS owner-only SELECT and denormalise user_id onto each event row so the auth predicate is cheap.

  • server-only breaks Vitest: The real server-only package throws on import outside an RSC, which is correct in production but blocks Vitest (Node) from loading any storage/action helper marked with it. Map 'server-only' in vitest.config.ts resolve.alias to an empty stub file (tests/__mocks__/server-only.ts with export {}). This keeps the production guard and lets unit tests import server-tagged modules to test their pure helpers.

  • Zod v4 uses .issues, not .errors: When catching z.ZodError, read err.issues[0]?.message. The old .errors field was renamed and the v4 type doesn't expose it. Build-time TS error: Property 'errors' does not exist on type 'ZodError<unknown>'.

  • Add new types to BOTH lib/types/<file>.ts AND lib/types/index.ts: Components import from @/lib/types (the barrel), so adding a type only to the source file passes lint but fails next build typecheck for any consumer. When introducing ConditionOperator, ConditionRule, EditLinkSettings etc., re-export them from the barrel in the same commit.

  • Reusable secrets need encryption + masking discipline: For per-tenant secrets (webhook signing keys, API tokens), keep two read paths in storage: a listForOwner(...) that returns masked dots so the secret never leaves the server in plaintext, and a listForDispatch(...) that decrypts only at the moment of use. UI's "rotate" button should regenerate + immediately reveal once (then dots again on next render). This way a leaked DB dump is useless and the UI never has to re-fetch the plaintext.

  • Bulk client-side rendering needs two requestAnimationFrame waits: When looping React state changes through a hidden renderer for capture, one await nextFrame() lets React commit but the browser may not have painted the new computed styles yet. Two consecutive requestAnimationFrame waits gives one frame for commit and one for paint, after which html2canvas reads accurate dimensions and fonts.

  • CSV "empty" check must trim: A CSV string of ' ' is non-empty but produces a single empty header row when split. Guard with if (!src.trim()) not if (!src) so whitespace-only inputs return clean empty headers + rows.

  • Re-key snapshot from label to field id when prefilling: Google Sheets are keyed by field label (the human-readable header), but PublicFormClient's state is keyed by field id. When loading a snapshot for the edit-link flow, walk form.fields and rebuild initialValues[field.id] = snapshot[field.label]. Failing to re-key shows a blank form even though the snapshot is correct.

  • Robots-noindex magic-link routes: Edit-link and other token-based public URLs must set metadata.robots: { index: false, follow: false } so search engines don't crawl, archive, or expose them in cached results. RLS protects the data but a leaked URL in a search index is a separate, real problem.

  • Triggers and Database Constraints (NOT NULL/Incorrect columns): When writing or updating PostgreSQL trigger functions that automatically seed other tables (e.g. public.usage on user signup), verify that:

    1. The column names in the INSERT match the actual table schema exactly (e.g. forms_created vs total_forms).
    2. All NOT NULL columns that lack default values (e.g. month DATE NOT NULL) are populated with valid values in the trigger's query (e.g. date_trunc('month', current_date)::date). Failing to do so will cause the signup database transaction to abort, showing a generic "database error" to the user and preventing registration.

  • Dual-Method Integration Validation (OAuth vs Service Account): When implementing features that gate resource creation or modification on configured integrations (such as Google Sheets integration in createFormAction), always ensure that you check for either method. Checking only for manual Service Account fields (googleClientEmail + googlePrivateKey) will inadvertently block users who chose the recommended OAuth path (googleAccessToken), forcing them to configure settings they do not need.

  • Shared PostgreSQL Trigger Functions (Trigger Target vs Column Fields): If you repurpose a trigger function (such as generate_short_code()) to target a different table (e.g. from public.forms to public.short_links), and change the column fields it references (e.g. from NEW.short_code to NEW.slug), you must explicitly drop/recreate any old triggers executing this function on the old tables. Otherwise, the old table inserts/updates will trigger the updated function, fail on the missing columns (throwing record "new" has no field "slug"), and abort the transaction. Keep trigger functions scoped to their respective tables (e.g., generate_form_short_code() for forms vs generate_short_code() for short_links).

  • Server-side auth checks block Static Site Generation (SSG): Reading cookies() or executing server-side Supabase auth checks (like supabase.auth.getUser()) inside page components forces Next.js to render them dynamically (SSR) at runtime. For marketing pages (landing, pricing, about, product specs), this causes serverless cold starts and latency. Instead, use a client-side component (like LandingHeaderAuth) that queries auth state on mount, leaving the static page body to build statically and be cached on the Edge CDN.

  • Vercel Serverless Function Regions: Vercel deploys serverless functions to Washington D.C. (iad1) by default. If the database is hosted in Singapore (ap-southeast-1), this adds ~250ms latency per query. Explicitly configure "regions": ["sin1"] in vercel.json to place serverless execution close to the database.

  • Escape HTML for any respondent-controlled value injected into emails: Confirmation/notification email templates that embed user-submitted values (answers, custom messages, form title) into the HTML body must HTML-escape them (&, <, >, ", '). Without escaping, a respondent can inject markup/<script>/broken table cells into the email — a stored-content injection vector. Use a small escapeHtml() helper and apply it to every interpolated untrusted value. (Note: getNewSubmissionEmail owner-notification template predates this rule and still interpolates raw values — a separate fix.)

  • Respondent email confirmation reuses the edit-link pattern exactly: When adding per-form settings gated on an email field (respondent confirmation, edit-link), follow the established recipe: jsonb column on forms (avoid column proliferation) + type in lib/types/forms.ts re-exported from the barrel + 2× fromRow mapping (getFormById AND getFormByShortCode — they're duplicated) + 1× toRow in saveForm + fire-and-forget dispatch block in submitFormAction (try/catch, never fail submission) + builder card mirroring EditLinkCard + email-value resolved via dbData[field.label] (dbData is keyed by label, not id).

  • Editing the type barrel: don't drop sibling exports: lib/types/index.ts mixes export type {...} blocks with a value export (export { TIER_LIMITS }). When adding a new type export, append it — don't replace a block in a way that removes the trailing TIER_LIMITS value export, or every TIER_LIMITS consumer breaks at build time. (Nearly did this adding audit types.)

  • Two-layer enforcement for consent/gating (PDPA): A consent checkbox in the client is UX only — it can be bypassed by calling the Server Action directly. Always enforce the same gate server-side in submitFormAction (reject when pdpaSettings.enabled and the consent flag isn't exactly 'true'). Keep the predicate in a pure module (lib/forms/pdpa.ts) so both sides and the tests share one source of truth.

  • Layout-only field types must be excluded from data paths: A new non-input field type (pagebreak, like separator/image) is still returned by form.fields. If it leaks into visibleFields it gets validated, appended to FormData, and creates a junk column in Google Sheets. Filter it out of visibleFields (f.type !== 'pagebreak' && isFieldVisible(f)) while still letting splitIntoPages use it as the page delimiter.

  • Multi-page without a schema change: Implement pages by inserting a pagebreak delimiter field into the existing fields array rather than adding a new DB structure. splitIntoPages drops the markers and yields page buckets; forms with zero pagebreaks naturally collapse to a single page (backward compatible, no migration). Skip pages whose every field is hidden by conditional logic via findAdjacentNonEmptyPage so navigation never lands on a blank page.

  • Guard Enter-key submit on multi-page forms: A form with onSubmit will fire on Enter even when the visible primary button is a type="button" Next. In handleSubmit, short-circuit if (multiPage && !isLastPage) { goToNextPage(); return; } so Enter advances the page instead of submitting partial data.

  • Audit tables are append-only from the client: Give audit_logs an owner-only SELECT RLS policy and NO insert/update/delete policy. Writes go exclusively through the service-role admin client inside logAudit(). This makes the trail tamper-proof from the browser while still readable by its owner. Log meaningful, low-frequency events (create/delete) — never per-keystroke autosave (updateFormAction), which would flood the log.

  • Never read layout (offsetWidth/getBoundingClientRect) inside a render .map(): In the certificate builder, const scale = canvasRef.current.offsetWidth / template.width lived inside template.elements.map(...), so it read layout once PER element PER render. During drag/resize (rapid setState) this triggers repeated synchronous "forced reflow" / layout thrashing (Chrome [Violation] Forced reflow … took NNms). Fix: track the container size in state via a ResizeObserver (its callback delivers contentRect without forcing reflow) and compute the derived value ONCE per render, not per item. Bonus: the value (font scale) now also reacts to window/container resize, which the render-time offsetWidth read silently failed to do. Reading layout inside a discrete event handler (e.g. getBoundingClientRect in onMouseMove) is fine — it's one read per event, not per render.

  • Deploying via vercel --prod from the working dir bypasses git entirely: It uploads local files, so production can silently drift ahead of the master/production branch (no commits, no history, no rollback). If a Vercel project is git-connected, prefer git push to the production branch to auto-deploy — that keeps git == production and gives preview deploys + an audit trail. Periodically check git log master..HEAD to catch drift.

  • Unit-testing Supabase storage modules without a live DB: mock @/utils/supabase/server (createClient, async) and @/utils/supabase/admin (createAdminClient, sync) with vi.hoisted + vi.mock. Build a chainable query-builder mock where each chain method returns the builder, .single() returns a configurable promise, and the builder is thenable (then(resolve){resolve(result)}) so await from().select().eq().order() resolves too. Mock @/lib/encryption to assert encrypt-on-write / decrypt-on-read without a real key. Lets you verify query scoping, row mapping, secret masking, and ownership checks deterministically.

  • Label-control association is the most common form a11y gap: A visible <Label> that only sits near an input (no htmlFor/id) is not programmatically associated — screen readers don't announce it (WCAG 1.3.1/4.1.2). For simple inputs use htmlFor/id; for composite widgets (radio/checkbox groups, custom selects) give the label an id and point the group/trigger at it via aria-labelledby + role="group". Put required indicators behind aria-hidden and add an sr-only word so the asterisk isn't read as punctuation.

  • "Bug still there after deploy" is often a stale browser cache, not the code: A user reported multi-page "Next auto-submits" persisting after fixes. The code was correct AND triple-protected (Next is type="button", handleSubmit returns early on non-final pages, and preventDefault on the Next click) — submit on a non-final page was provably impossible. Root cause was the browser serving the previous JS bundle. Before chasing a phantom code bug across multiple deploys, first confirm the user has hard-refreshed (Ctrl+Shift+R) / tested in Incognito, and verify the latest production deployment is Ready and matches the latest commit. After a real hard refresh the multi-page nav (Next/Back + "Page X / Y" + Submit only on the last page) worked correctly.

  • Robust Column/Header Matching & Guarding Fallbacks: When parsing dynamic spreadsheets (e.g. Google Sheets), use word-boundary regexes like /\bkp\b/ or /\bic\b/ rather than strict matches or broad substrings. This accommodates compound headers (e.g., IC/Passport, No. KP/Pasport) without matching letters inside unrelated fields (e.g., office, timestamp). Additionally, never fall back to using the user's search query (like their email input) in templates if the actual target field (like ic) was missing/undefined, as this leads to rendering email addresses on documents where the IC number is expected. Use a conditional check to guard fallbacks.- Write external-API data locally FIRST, sync remotely after: Any submission that must end up in a slow/flaky third-party system (Google Sheets, CRM, webhook) should be persisted to your own DB first, then synced asynchronously (after() in Next.js 15+/16, plus a retry cron). The old pattern (remote write on the critical path) loses data on remote failure AND blocks the user on remote latency. Track sync state on the row (pending/synced/failed + partial index) so retries are cheap and idempotent — the unique submission_id doubles as an idempotency key that swallows double-submits.

  • after() snapshots: capture request context before the callback: Anything read from headers()/cookies() must be snapshotted into plain variables BEFORE scheduling after(async () => ...). Inside the after-callback the request context is finalizing and reads can fail or return stale values. Same for any object you let-rebind later (e.g. accessToken) — copy to a *Snapshot const and close over that.

  • Webhook handlers must use the service-role client: A provider webhook (BCL, Stripe...) arrives with NO user cookies. createClient() (anon) under owner-only RLS silently finds nothing — the handler 404s on data that exists. All webhook DB reads/writes go through the admin client; authenticity comes from the HMAC signature, not from session identity.

  • Payment idempotency: mark processed in the same write as the grant: Set processed_at in the SAME update that flips status to completed, BEFORE upserting the subscription/emails. Checking status === 'completed' alone is unsafe (pre-existing completed rows from partial failures must remain re-processable); checking after the grant is unsafe (crash between grant and marker = double grant). Replay response: 200 {duplicate:true} — never 4xx, or the provider keeps retrying a permanently "failing" endpoint.

  • Postgres unique-violation code is 23505: supabase-js surfaces it as error.code. Use it to distinguish "duplicate idempotency key — swallow" from "real insert failure — surface to user". Pattern: insert() returns {error:{code:'23505'}} → treat as already-done success.

  • Backfill idempotency markers when adding them to live tables: A new processed_at column starts NULL on all existing rows. Without UPDATE ... SET processed_at = now() WHERE status='completed' in the migration, the first replayed webhook after deploy double-processes every historical payment.

  • One pricing constant file, everywhere: price/amount/description strings drift when hardcoded in the checkout route, the marketing page, the in-app modal, AND a constants file. Keep a single PRO_PRICE object (amount as number for gateways + display strings) and import it from all four. Test asserts display === RM ${amount}`` so a mismatch fails CI.

  • Re-using the client's conditional evaluator server-side closes the required-field loophole: extracting submission validation into a pure module (validateSubmission) that calls the SAME evaluateConditional as the public form means "required but hidden by logic" is allowed server-side exactly when the client hides it. Re-key the label-keyed payload to field ids (byId) before evaluating — conditions reference field ids, but FormData arrives keyed by label.