From 10445e0a3523460c45dadfb90508681928619cf2 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 27 Aug 2026 10:57:43 -0500 Subject: [PATCH 01/13] [316] Set tiered Cache-Control on published dashboard objects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A republish has to show up immediately through a fronting cache we cannot invalidate — ours or a customer's CloudFront — so every object now carries a Cache-Control chosen by its key: the entry page and the baked config revalidate before every use, the keys that are content- addressed by construction are immutable, and everything else gets a short 5-minute TTL. Two key classes earn the immutable tier. The webpack output under build/static/(js|css|media) is content-hashed — a premise now pinned by tests/unit/webpackHashedOutput.spec.js, which fails if a production naming option loses its hash. The plugin uploads under assets///uploads/ are named crypto.randomUUID() by the upload router and never overwritten, so their keys are content-addressed too. The S3 copy paths (copyPrefix, copyObjectIfExists) also switch to MetadataDirective REPLACE, since CopyObject's default COPY preserves the source's metadata verbatim and cannot set a header the source object never had. REPLACE means supplying ContentType too, taken from the same extension map the uploads use. --- .../serving-a-dashboard-from-your-domain.md | 5 +- scripts/lib/aws-provision.js | 55 +++++- tests/unit/awsProvision.spec.js | 162 ++++++++++++++++++ tests/unit/webpackHashedOutput.spec.js | 64 +++++++ 4 files changed, 281 insertions(+), 5 deletions(-) create mode 100644 tests/unit/webpackHashedOutput.spec.js diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index 22e714fb7..1ca8e16da 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -13,10 +13,11 @@ CloudFront forwards the full path as the visitor typed it — it never removes t Configure these things on the cache behavior (and its origin) that forwards to the dashboard: 1. **Declare the prefix.** Add a custom origin header `X-Forwarded-Prefix` whose value is the path your rule matches, starting with `/` and without a trailing slash — for the example above, `/disasters/data-visualization`. Write the value exactly as the path appears, un-encoded — `/my path`, not `/my%20path` — and keep the path itself ASCII: letters, digits, and the common URL-safe punctuation (`-`, `_`, `.`, `~`, spaces). Non-ASCII characters in the prefix are not supported. If the header is missing or doesn't match the rule's pattern, every request under the path comes back as a 403 immediately — a loud failure, on purpose, rather than quietly serving the wrong files. (It's a 403, not a 404: our origin is a private S3 bucket behind CloudFront's Origin Access Control, and S3 answers a rejected request with AccessDenied — there are no custom error pages standing in to make it look like a 404.) -2. **Build a custom cache policy — never a managed one.** Two things have to be true of it, and no managed policy gets both right. This applies whether or not the dashboard's password is on. +2. **Build a custom cache policy — never a managed one.** Four things have to be true of it, and no managed policy gets all four right. This applies whether or not the dashboard's password is on. - **Put the `Authorization` header in the cache key**, not just on the wire. Dashboards are password-protected with HTTP Basic auth, and your CloudFront caches whatever our origin returns. A policy that forwards `Authorization` to us but doesn't vary the cache key on it will serve one visitor's authenticated response to the next visitor who hasn't entered a password at all — our password check runs on our distribution, once, and your edge then caches the result. Putting the header in the cache key both forwards it (satisfying the auth requirement) and partitions your cache per credential. - - **Set Minimum TTL to 0** (a Default TTL of 0 is sensible too). The managed policies you'd otherwise reach for, CachingOptimized and friends, set a minimum TTL of at least 1 second, which overrides the response headers our origin returns regardless of what they say. The redirect that sends a slash-less entry link to its trailing-slash form is marked uncacheable for a reason — it carries the visitor's query string — so without an explicit 0, your edge caches one visitor's redirect and replays it to the next. + - **Set Minimum TTL to 0** (a Default TTL of 0 is sensible too). The managed policies you'd otherwise reach for, CachingOptimized and friends, set a minimum TTL of at least 1 second, which overrides our `Cache-Control` directives regardless of what they say — so without an explicit 0, both the slash-less-entry redirect and the republish freshness below silently degrade. That redirect, which sends a slash-less entry link to its trailing-slash form, is marked uncacheable for a reason — it carries the visitor's query string — so your edge would cache one visitor's redirect and replay it to the next. - **Include all query strings in the cache key** (equivalently, forward all query strings). A policy that drops query strings never sends them to our origin, so a slash-less deep link like `/disasters/data-visualization?view=2` reaches our redirect stripped of its `?view=2`, and the trailing-slash URL we send the visitor to loses it for good. Keeping every query string in the cache key both forwards it and keeps distinct query strings from sharing one cached entry. + - **Honor our `Cache-Control` headers.** They have three tiers: the entry page and configuration revalidate before every use (`no-cache`, not `no-store` — so a republish shows up on your domain immediately, with no purge on your side), the files whose names are content-fingerprinted — the application's own bundles, and the images and files uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and everything else (the mosaic CSV and other supporting files that can change in place) gets a short 5-minute lifetime. CloudFront has no single "obey the origin" switch: what actually delivers all three is the Minimum TTL of 0 above, which stops the policy raising our floor, plus a Maximum TTL of at least 31536000 (a year), which stops it capping the immutable tier. 3. **Set the origin protocol policy to HTTPS-only.** The dashboard's shared Basic-auth password rides on the `Authorization` header of every request you forward to us. A "HTTP only" or "match viewer" origin setting will send that header — and the password inside it — to our origin in cleartext on any request that reaches your edge over plain HTTP. Force HTTPS to the origin so the credential is never on the wire unencrypted. 4. **Do not forward the viewer's `Host` header.** Our distribution answers only to its own `*.cloudfront.net` name; a request carrying your hostname is rejected by AWS with a 403 before any of our code runs. With no origin request policy attached at all, CloudFront already omits the viewer's `Host` header by default, so the hazard here is specifically choosing the managed `AllViewer` origin request policy, which forwards every viewer header including `Host`. If you do want an origin request policy — to forward everything else CloudFront doesn't send by default — use the managed `AllViewerExceptHostHeader` policy, which forwards the rest of the viewer's headers while still excluding `Host`. diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index 9c70374b6..00cdddfb0 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -353,6 +353,35 @@ function contentTypeForFile(filePath) { ); } +// Cache-Control tier for a published-dashboard object key. The entry page and +// baked config must revalidate on every request (a fronting cache we cannot +// invalidate may otherwise pin an old release for a day); two classes are +// immutable because their names are content-addressed by construction — the +// webpack output, whose filenames carry a content hash (pinned by +// tests/unit/webpackHashedOutput.spec.js), and plugin uploads under +// assets///uploads/, which the upload router names +// crypto.randomUUID() and never overwrites (API/Backend/Upload/uploadRouter.js). +// Everything else — the keys that really do change in place on republish — +// falls back to a short TTL. +function cacheControlForKey(key) { + if ( + key === "index.html" || + key === "build/index.html" || + /^Missions\/[^/]+\/config\.json$/.test(key) + ) + return "no-cache"; + // The uploads shape mirrors ASSETS_UPLOAD_KEY in + // src/essence/Tools/Card/adapters/buildCardData.ts: exactly two segments + // between "assets/" and "/uploads/", so a lookalike such as + // "assets/uploads/x.png" is not mistaken for the writer's shape. + if ( + /^build\/static\/(js|css|media)\//.test(key) || + /^assets\/[^/]+\/[^/]+\/uploads\//.test(key) + ) + return "public, max-age=31536000, immutable"; + return "public, max-age=300"; +} + function walkDirectory(dir, baseDir) { baseDir = baseDir || dir; let files = []; @@ -377,16 +406,18 @@ async function uploadDirectory({ bucket, dir, prefix = "", concurrency = 8 }) { async function worker() { while (index < files.length) { const file = files[index++]; + const key = `${prefix}${file.key}`; await s3.send( new PutObjectCommand({ Bucket: bucket, - Key: `${prefix}${file.key}`, + Key: key, Body: fs.createReadStream(file.absolute), // An explicit length keeps the streaming PUT retryable by the // SDK (an unknown-length stream is sent unsigned/non-retryable, // so one network blip would fail the whole publish). ContentLength: fs.statSync(file.absolute).size, ContentType: contentTypeForFile(file.absolute), + CacheControl: cacheControlForKey(key), }) ); } @@ -407,13 +438,16 @@ async function uploadFile({ bucket, key, filePath }) { Body: fs.createReadStream(filePath), ContentLength: fs.statSync(filePath).size, ContentType: contentTypeForFile(filePath), + CacheControl: cacheControlForKey(key), }) ); } // Invalidates CloudFront paths so an updated dashboard is served -// immediately (the distribution caches aggressively; hashed bundle -// names dodge it but index.html, config.json, and assets do not). +// immediately. Our own Cache-Control tiers already cover most of it — +// index.html and config.json revalidate every request, and hashed bundles +// arrive under new names — so this is what closes the gap for the short-TTL +// tier and for any edge that ignores those headers. async function createInvalidation({ distributionId, paths = ["/*"] }) { const { cloudfront } = getClients(); await cloudfront.send( @@ -457,6 +491,17 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { Bucket: destBucket, Key: obj.Key, CopySource: buildCopySource(sourceBucket, obj.Key), + // COPY (the default) cannot set new headers on the copy, so + // REPLACE is required to add a Cache-Control the source object + // never had — and REPLACE means supplying ContentType too. + // REPLACE drops the source's entire metadata set, not just its + // Content-Type: Content-Encoding, Content-Disposition and any + // x-amz-meta-* are lost unless restated here. Nothing sets those + // today (the upload router writes ContentType alone), but a future + // gzipped object would have to carry its Content-Encoding across. + MetadataDirective: "REPLACE", + ContentType: contentTypeForFile(obj.Key), + CacheControl: cacheControlForKey(obj.Key), }) ); copied++; @@ -478,6 +523,9 @@ async function copyObjectIfExists({ sourceBucket, destBucket, key }) { Bucket: destBucket, Key: key, CopySource: buildCopySource(sourceBucket, key), + MetadataDirective: "REPLACE", + ContentType: contentTypeForFile(key), + CacheControl: cacheControlForKey(key), }) ); return true; @@ -607,6 +655,7 @@ module.exports = { getStackOutputs, deleteStack, contentTypeForFile, + cacheControlForKey, uploadDirectory, uploadFile, createInvalidation, diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 5712d4949..462cfcad9 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -1,4 +1,7 @@ import { test, expect } from 'vitest' +import fs from 'fs' +import os from 'os' +import path from 'path' // Tests for scripts/lib/aws-provision.js using injected mock clients — // no test here (or anywhere) ever calls real AWS. @@ -676,6 +679,137 @@ test.describe('emptyBucket', () => { }) }) +test.describe('contentTypeForFile', () => { + test('maps a known extension', () => { + expect(provision.contentTypeForFile('a/b/c.png')).toBe('image/png') + }) + + test('matches extensions case-insensitively', () => { + expect(provision.contentTypeForFile('a/b/C.PNG')).toBe('image/png') + }) + + // Load-bearing under CopyObject's MetadataDirective: REPLACE, which drops + // the source's Content-Type and takes whatever this returns instead. + test('falls back to octet-stream for an unmapped extension', () => { + expect(provision.contentTypeForFile('a/b/c.xyz')).toBe( + 'application/octet-stream' + ) + }) +}) + +test.describe('cacheControlForKey', () => { + // [key, expected Cache-Control]. Three tiers: revalidate-always for the + // entry page and the baked config, immutable for the content-addressed + // keys (hashed webpack output and the never-overwritten plugin uploads), + // a short TTL for everything else. + const TIERS = [ + ['index.html', 'no-cache'], + ['build/index.html', 'no-cache'], + ['Missions/M/config.json', 'no-cache'], + [ + 'build/static/js/main.abc123.js', + 'public, max-age=31536000, immutable', + ], + ['build/static/css/x.css', 'public, max-age=31536000, immutable'], + ['build/static/media/a.png', 'public, max-age=31536000, immutable'], + ['Missions/M/Data/mosaic_parameters.csv', 'public, max-age=300'], + // Under build/static but not content-hashed, so explicitly NOT + // immutable. + ['build/static/cesium/Cesium.js', 'public, max-age=300'], + ['public/workers/pdf.worker.min.mjs', 'public, max-age=300'], + // The upload router names every object crypto.randomUUID(). and + // never overwrites, so the key is content-addressed in practice. + [ + 'assets/M/CardPlugin/uploads/a.png', + 'public, max-age=31536000, immutable', + ], + // Under assets/ but not the writer's shape (no /uploads/ segment two + // levels down), so it stays on the fallback tier. + ['assets/M/CardPlugin/icon.png', 'public, max-age=300'], + // A lookalike: "uploads" here is the mission segment, not the + // router's directory, so it is not the content-addressed shape. + ['assets/uploads/a.png', 'public, max-age=300'], + ] + + TIERS.forEach(([key, expected]) => { + test(`'${key}' -> '${expected}'`, () => { + expect(provision.cacheControlForKey(key)).toBe(expected) + }) + }) +}) + +// Runs fn(dir, puts) against a fresh temp directory with an injected S3 +// client that records every command input into `puts`, then resets the client +// and removes the directory. The mock never reads the body, so the stream's +// deferred fs.open() is swallowed here — otherwise the cleanup below can race +// it into an unhandled 'error' event. +async function withUploadFixture(fn) { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mmgis-upload-')) + const puts = [] + provision.setClients({ + s3: mockClient((command) => { + puts.push(command.input) + if (command.input.Body) { + command.input.Body.on('error', () => {}) + command.input.Body.destroy() + } + return {} + }), + }) + try { + await fn(dir, puts) + } finally { + provision.setClients(null) + fs.rmSync(dir, { recursive: true, force: true }) + } +} + +test.describe('uploadDirectory', () => { + test('uploads every file with the tiered Cache-Control for its key', async () => { + await withUploadFixture(async (dir, puts) => { + fs.mkdirSync(path.join(dir, 'static', 'js'), { recursive: true }) + fs.writeFileSync(path.join(dir, 'index.html'), '') + fs.writeFileSync( + path.join(dir, 'static', 'js', 'main.abc123.js'), + 'console.log(1)' + ) + const count = await provision.uploadDirectory({ + bucket: 'dash', + dir, + prefix: 'build/', + }) + expect(count).toBe(2) + const byKey = Object.fromEntries( + puts.map((input) => [input.Key, input]) + ) + // Only the prefixed key earns the immutable tier — the relative + // path 'static/js/main.abc123.js' falls through to max-age=300 — + // so this fails if the tier is read off anything but the key. + expect(byKey['build/static/js/main.abc123.js'].CacheControl).toBe( + 'public, max-age=31536000, immutable' + ) + expect(byKey['build/index.html'].CacheControl).toBe('no-cache') + }) + }) +}) + +test.describe('uploadFile', () => { + test('sets CacheControl for the tier of the target key', async () => { + await withUploadFixture(async (dir, puts) => { + const filePath = path.join(dir, 'mosaic_parameters.csv') + fs.writeFileSync(filePath, 'a,b,c\n') + await provision.uploadFile({ + bucket: 'dash', + key: 'Missions/M/Data/mosaic_parameters.csv', + filePath, + }) + // Literal, not cacheControlForKey(key): that form would pass even + // if the tiering broke. + expect(puts[0].CacheControl).toBe('public, max-age=300') + }) + }) +}) + test.describe('copyPrefix', () => { test.afterEach(() => provision.setClients(null)) @@ -721,6 +855,13 @@ test.describe('copyPrefix', () => { expect(copies[2].CopySource).toBe( 'shared/assets/TestMission/with%20space.png' ) + // CopyObject's default (COPY) keeps the source's metadata and cannot + // add the Cache-Control the source never had; REPLACE can, and in turn + // obliges the copy to restate its Content-Type. Tier coverage lives in + // the cacheControlForKey table — this pins the wiring at this site. + expect(copies[0].MetadataDirective).toBe('REPLACE') + expect(copies[0].ContentType).toBe('image/png') + expect(copies[0].CacheControl).toBe('public, max-age=300') }) }) @@ -754,6 +895,27 @@ test.describe('copyObjectIfExists', () => { }) ).toBe(true) }) + + // The same wiring as copyPrefix, pinned at this second call site. + test('replaces metadata and sets ContentType + CacheControl on the copy', async () => { + let input + provision.setClients({ + s3: mockClient((command) => { + input = command.input + return {} + }), + }) + await provision.copyObjectIfExists({ + sourceBucket: 'shared', + destBucket: 'dash', + key: 'Missions/Test/Data/mosaic_parameters.csv', + }) + expect(input.MetadataDirective).toBe('REPLACE') + // REPLACE drops the source's own Content-Type, so the copy supplies + // one — '.csv' resolves through the extension map. + expect(input.ContentType).toBe('text/csv') + expect(input.CacheControl).toBe('public, max-age=300') + }) }) test.describe('runPublishTask', () => { diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js new file mode 100644 index 000000000..bce2522ba --- /dev/null +++ b/tests/unit/webpackHashedOutput.spec.js @@ -0,0 +1,64 @@ +import { test, expect } from 'vitest' +import fs from 'fs' +import path from 'path' + +// Pins the premise of the immutable Cache-Control tier in +// scripts/lib/aws-provision.js: every production filename webpack writes under +// build/static/(js|css|media) carries a content hash. Drop the hash from one of +// these options (or rename it) and a stable filename lands in the immutable +// tier, where customers' CloudFront edges would pin it for a year. +const CONFIG = fs.readFileSync( + path.join(__dirname, '..', '..', 'configuration', 'webpack.config.js'), + 'utf8' +) + +const HASH_TOKEN = /\[contenthash|\[hash/ + +// The literal `re` captures, failing the test when it matches nothing — a +// renamed or restructured option must break the suite, not skip its assertion. +function capture(label, re, text = CONFIG) { + const match = text.match(re) + expect(match, `no match for ${label}`).not.toBeNull() + return match[1] +} + +test.describe('webpack production output is content-hashed', () => { + test('output.filename and output.chunkFilename carry a hash', () => { + // Leading [^A-Za-z] so `filename:` does not match `chunkFilename:`. + expect( + capture( + 'output.filename', + /[^A-Za-z]filename:\s*isEnvProduction\s*\?\s*"([^"]+)"/ + ) + ).toMatch(HASH_TOKEN) + expect( + capture( + 'output.chunkFilename', + /chunkFilename:\s*isEnvProduction\s*\?\s*"([^"]+)"/ + ) + ).toMatch(HASH_TOKEN) + }) + + test("MiniCssExtractPlugin's filenames carry a hash", () => { + const options = capture( + 'MiniCssExtractPlugin options', + /new MiniCssExtractPlugin\(\{([\s\S]*?)\}\)/ + ) + expect( + capture('css filename', /[^A-Za-z]filename:\s*"([^"]+)"/, options) + ).toMatch(HASH_TOKEN) + expect( + capture('css chunkFilename', /chunkFilename:\s*"([^"]+)"/, options) + ).toMatch(HASH_TOKEN) + }) + + test('the media loaders name files with a hash', () => { + // The url-loader (small images, inlined above a size limit) and the + // catch-all file-loader both emit into static/media. + const names = [ + ...CONFIG.matchAll(/name:\s*"(static\/media\/[^"]+)"/g), + ].map((m) => m[1]) + expect(names.length).toBeGreaterThanOrEqual(2) + names.forEach((name) => expect(name).toMatch(HASH_TOKEN)) + }) +}) From 2ca175fb8d867ae5cf5d935c55e3b089769af268 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 16:28:55 -0500 Subject: [PATCH 02/13] [316] Name the upload-key tier regex, test the real webpack config, and skip the raw template --- scripts/lib/aws-provision.js | 48 +++++++++----- scripts/lib/cfn-template.js | 5 +- scripts/publish-static.js | 13 ++-- src/pre/uploadKey.ts | 7 +- tests/unit/awsProvision.spec.js | 68 +++++++++++++++----- tests/unit/uploadKeyClassifier.spec.js | 30 ++++++--- tests/unit/webpackHashedOutput.spec.js | 88 ++++++++++++++------------ 7 files changed, 167 insertions(+), 92 deletions(-) diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index f88aec23e..2778d6b3c 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -376,6 +376,12 @@ function contentTypeForFile(filePath) { ); } +// The object keys API/Backend/Upload/uploadRouter.js writes for plugin +// uploads: "assets/", exactly two path segments, then "/uploads/". The same +// classifier lives in src/pre/uploadKey.ts and configure/src/core/upload.js; +// tests/unit/uploadKeyClassifier.spec.js runs one table through all three. +const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; + // Cache-Control tier for a published-dashboard object key. The entry page and // baked config must revalidate on every request (a fronting cache we cannot // invalidate may otherwise pin an old release for a day); two classes are @@ -385,7 +391,11 @@ function contentTypeForFile(filePath) { // assets///uploads/, which the upload router names // crypto.randomUUID() and never overwrites (API/Backend/Upload/uploadRouter.js). // Everything else — the keys that really do change in place on republish — -// falls back to a short TTL. +// falls back to a short TTL. Two runtime families sit in that fallback under +// stable names, the pdf.js worker under public/workers and the Cesium tree +// under build/static/cesium, so a republish that bumps the MMGIS version can +// leave a customer's edge pairing a new bundle with a copy of those up to five +// minutes old. function cacheControlForKey(key) { if ( key === "index.html" || @@ -393,13 +403,9 @@ function cacheControlForKey(key) { /^Missions\/[^/]+\/config\.json$/.test(key) ) return "no-cache"; - // The uploads shape mirrors ASSETS_UPLOAD_KEY in - // src/essence/Tools/Card/adapters/buildCardData.ts: exactly two segments - // between "assets/" and "/uploads/", so a lookalike such as - // "assets/uploads/x.png" is not mistaken for the writer's shape. if ( /^build\/static\/(js|css|media)\//.test(key) || - /^assets\/[^/]+\/[^/]+\/uploads\//.test(key) + ASSETS_UPLOAD_KEY.test(key) ) return "public, max-age=31536000, immutable"; return "public, max-age=300"; @@ -421,10 +427,19 @@ function walkDirectory(dir, baseDir) { } // Uploads every file under `dir` to `bucket`, keys relative to `dir` -// (optionally prefixed). Returns the number of files uploaded. -async function uploadDirectory({ bucket, dir, prefix = "", concurrency = 8 }) { +// (optionally prefixed). `filter` receives that relative key and keeps the +// file when it returns true. Returns the number of files uploaded. +async function uploadDirectory({ + bucket, + dir, + prefix = "", + concurrency = 8, + filter, +}) { const { s3 } = getClients(); - const files = walkDirectory(dir); + const files = filter + ? walkDirectory(dir).filter((file) => filter(file.key)) + : walkDirectory(dir); let index = 0; async function worker() { while (index < files.length) { @@ -466,11 +481,11 @@ async function uploadFile({ bucket, key, filePath }) { ); } -// Invalidates CloudFront paths so an updated dashboard is served -// immediately. Our own Cache-Control tiers already cover most of it — -// index.html and config.json revalidate every request, and hashed bundles -// arrive under new names — so this is what closes the gap for the short-TTL -// tier and for any edge that ignores those headers. +// Invalidates paths on our own distribution, the only one this reaches. The +// Cache-Control tiers already cover most of it there — index.html and +// config.json revalidate every request, and hashed bundles arrive under new +// names — so this is what closes the gap for the five-minute tier. Any other +// edge in front of the dashboard is governed by those headers alone. async function createInvalidation({ distributionId, paths = ["/*"] }) { const { cloudfront } = getClients(); await cloudfront.send( @@ -519,9 +534,8 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { // never had — and REPLACE means supplying ContentType too. // REPLACE drops the source's entire metadata set, not just its // Content-Type: Content-Encoding, Content-Disposition and any - // x-amz-meta-* are lost unless restated here. Nothing sets those - // today (the upload router writes ContentType alone), but a future - // gzipped object would have to carry its Content-Encoding across. + // x-amz-meta-* are lost unless restated here. Nothing sets those: + // the upload router writes ContentType alone. MetadataDirective: "REPLACE", ContentType: contentTypeForFile(obj.Key), CacheControl: cacheControlForKey(obj.Key), diff --git a/scripts/lib/cfn-template.js b/scripts/lib/cfn-template.js index 1aef65a13..84408f087 100644 --- a/scripts/lib/cfn-template.js +++ b/scripts/lib/cfn-template.js @@ -227,7 +227,10 @@ function renderCfnTemplate({ password } = {}) { DefaultCacheBehavior: { TargetOriginId: "DashboardBucketOrigin", ViewerProtocolPolicy: "redirect-to-https", - // AWS managed policy: CachingOptimized + // AWS managed policy: CachingOptimized — minimum TTL 1 s, + // default 86400 s, maximum 31536000 s. The Cache-Control tiers + // in scripts/lib/aws-provision.js rely on that maximum being at + // least a year, or the edge would cap the immutable tier. CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6", FunctionAssociations: [ { diff --git a/scripts/publish-static.js b/scripts/publish-static.js index 971954910..03943e451 100644 --- a/scripts/publish-static.js +++ b/scripts/publish-static.js @@ -288,6 +288,8 @@ async function main() { bucket, dir: path.join(rootDir, "public"), prefix: "public/", + // public/index.html is the un-rendered Pug template. + filter: (key) => key !== "index.html", }); await provision.uploadFile({ bucket, @@ -312,11 +314,12 @@ async function main() { `Uploaded ${uploadedBuild} build and ${uploadedPublic} public file(s) to ${bucket}.` ); - // 5.5 Bust the CDN so the refreshed bundle/config/assets serve - // immediately — the distribution caches aggressively, and only the - // hashed bundle filenames are naturally cache-safe. A brand-new - // distribution has nothing cached, so doing this unconditionally - // keeps publish and update on one path. + // 5.5 Invalidate our own distribution so the five-minute tier serves the + // new release now. Customers' edges are governed by the Cache-Control + // tiers instead — see + // docs/infrastructure/serving-a-dashboard-from-your-domain.md. A brand-new + // distribution has nothing cached, so doing this unconditionally keeps + // publish and update on one path. if (outputs.DistributionId) { await provision.createInvalidation({ distributionId: outputs.DistributionId, diff --git a/src/pre/uploadKey.ts b/src/pre/uploadKey.ts index 892f758e0..0a4b1549b 100644 --- a/src/pre/uploadKey.ts +++ b/src/pre/uploadKey.ts @@ -16,10 +16,11 @@ // "assets/uploads/x.png", and a looser test ("starts with assets/") would // grab those too and resolve them against the wrong root. // -// configure/src/core/upload.js carries its own copy of this regex — the CMS -// is a separate bundle with no import path into this one. +// configure/src/core/upload.js and scripts/lib/aws-provision.js carry their +// own copies of this regex — the CMS is a separate bundle and the publish +// scripts are CommonJS run by Node, neither with an import path into this one. // tests/unit/uploadKeyClassifier.spec.js runs one table of values through -// both and fails if they classify any of them differently. +// all three and fails if they classify any of them differently. export const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\// // What URL should the page request for a stored value? Four cases, checked diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 2728c7199..67b50fcd3 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -7,6 +7,7 @@ import path from 'path' // no test here (or anywhere) ever calls real AWS. const provision = require('../../scripts/lib/aws-provision') +const { IMAGE_MIME_TO_EXT } = require('../../API/Backend/Upload/validate') function mockClient(handler) { return { send: async (command) => handler(command) } @@ -774,16 +775,21 @@ test.describe('emptyBucket', () => { }) test.describe('contentTypeForFile', () => { - test('maps a known extension', () => { - expect(provision.contentTypeForFile('a/b/c.png')).toBe('image/png') - }) + // CopyObject's MetadataDirective: REPLACE drops the source's Content-Type + // and takes this one, so every type the upload router can write has to + // round-trip back to itself through the extension it was stored under. + test.each(Object.entries(IMAGE_MIME_TO_EXT))( + "round-trips the upload router's %s", + (mime, ext) => { + expect(provision.contentTypeForFile('x.' + ext)).toBe(mime) + } + ) test('matches extensions case-insensitively', () => { expect(provision.contentTypeForFile('a/b/C.PNG')).toBe('image/png') }) - // Load-bearing under CopyObject's MetadataDirective: REPLACE, which drops - // the source's Content-Type and takes whatever this returns instead. + // The catch-all a copy or upload gets when nothing maps the extension. test('falls back to octet-stream for an unmapped extension', () => { expect(provision.contentTypeForFile('a/b/c.xyz')).toBe( 'application/octet-stream' @@ -806,7 +812,7 @@ test.describe('cacheControlForKey', () => { ], ['build/static/css/x.css', 'public, max-age=31536000, immutable'], ['build/static/media/a.png', 'public, max-age=31536000, immutable'], - ['Missions/M/Data/mosaic_parameters.csv', 'public, max-age=300'], + ['Missions/M/Data/waypoints.csv', 'public, max-age=300'], // Under build/static but not content-hashed, so explicitly NOT // immutable. ['build/static/cesium/Cesium.js', 'public, max-age=300'], @@ -885,26 +891,47 @@ test.describe('uploadDirectory', () => { expect(byKey['build/index.html'].CacheControl).toBe('no-cache') }) }) + + test('skips the files filter rejects, by their unprefixed key', async () => { + await withUploadFixture(async (dir, puts) => { + fs.writeFileSync(path.join(dir, 'index.html'), '') + fs.writeFileSync(path.join(dir, 'keep.txt'), 'keep') + const count = await provision.uploadDirectory({ + bucket: 'dash', + dir, + prefix: 'public/', + filter: (key) => key !== 'index.html', + }) + expect(count).toBe(1) + expect(puts.map((input) => input.Key)).toEqual(['public/keep.txt']) + }) + }) }) test.describe('uploadFile', () => { test('sets CacheControl for the tier of the target key', async () => { await withUploadFixture(async (dir, puts) => { - const filePath = path.join(dir, 'mosaic_parameters.csv') - fs.writeFileSync(filePath, 'a,b,c\n') + // The local file's own name sits on a different tier from the key + // it is uploaded under, so this fails if the tier is read off the + // path instead of the key. + const filePath = path.join(dir, 'payload.txt') + fs.writeFileSync(filePath, '\n') await provision.uploadFile({ bucket: 'dash', - key: 'Missions/M/Data/mosaic_parameters.csv', + key: 'index.html', filePath, }) // Literal, not cacheControlForKey(key): that form would pass even // if the tiering broke. - expect(puts[0].CacheControl).toBe('public, max-age=300') + expect(puts[0].CacheControl).toBe('no-cache') }) }) }) test.describe('copyPrefix', () => { + // The upload router names every file crypto.randomUUID() + the extension. + const UPLOAD_UUID = '6f1e2a3c-4b5d-4e6f-8a9b-0c1d2e3f4a5b' + test.afterEach(() => provision.setClients(null)) test('same-key copies every object under the prefix', async () => { @@ -916,8 +943,11 @@ test.describe('copyPrefix', () => { expect(command.input.Prefix).toBe('assets/TestMission/') return { Contents: [ + // The shape the upload router writes. + { + Key: `assets/TestMission/CardPlugin/uploads/${UPLOAD_UUID}.png`, + }, { Key: 'assets/TestMission/icon.png' }, - { Key: 'assets/TestMission/photo.jpg' }, { Key: 'assets/TestMission/with space.png' }, ], IsTruncated: false, @@ -937,12 +967,12 @@ test.describe('copyPrefix', () => { }) expect(count).toBe(3) // Same keys in the destination bucket - expect(copies[0].Bucket).toBe('dash') - expect(copies[0].Key).toBe('assets/TestMission/icon.png') + expect(copies[1].Bucket).toBe('dash') + expect(copies[1].Key).toBe('assets/TestMission/icon.png') // CopySource is "bucket/key" with the separators left intact — // NOT encodeURIComponent of the whole string (that would turn the // slashes into %2F and break the copy). - expect(copies[0].CopySource).toBe('shared/assets/TestMission/icon.png') + expect(copies[1].CopySource).toBe('shared/assets/TestMission/icon.png') // Special chars inside a segment are encoded; the "/" separators // and the bucket/key boundary are preserved. expect(copies[2].Key).toBe('assets/TestMission/with space.png') @@ -952,10 +982,16 @@ test.describe('copyPrefix', () => { // CopyObject's default (COPY) keeps the source's metadata and cannot // add the Cache-Control the source never had; REPLACE can, and in turn // obliges the copy to restate its Content-Type. Tier coverage lives in - // the cacheControlForKey table — this pins the wiring at this site. + // the cacheControlForKey table — this pins the wiring at this site, + // across both tiers a copied object can land on. expect(copies[0].MetadataDirective).toBe('REPLACE') expect(copies[0].ContentType).toBe('image/png') - expect(copies[0].CacheControl).toBe('public, max-age=300') + expect(copies[0].CacheControl).toBe( + 'public, max-age=31536000, immutable' + ) + expect(copies[1].MetadataDirective).toBe('REPLACE') + expect(copies[1].ContentType).toBe('image/png') + expect(copies[1].CacheControl).toBe('public, max-age=300') }) }) diff --git a/tests/unit/uploadKeyClassifier.spec.js b/tests/unit/uploadKeyClassifier.spec.js index ed322fdaf..954a31a15 100644 --- a/tests/unit/uploadKeyClassifier.spec.js +++ b/tests/unit/uploadKeyClassifier.spec.js @@ -2,16 +2,16 @@ import { test, expect } from 'vitest' import { resolveMissionAssetUrl } from '../../src/pre/uploadKey.ts' import { buildPreviewSrc } from '../../configure/src/core/upload.js' +const { cacheControlForKey } = require('../../scripts/lib/aws-provision') + // One table of stored values, run through every copy of the upload-key -// classifier. The app bundle and the Configure SPA cannot share a module, so -// each carries its own regex; what they must agree on is which values are -// upload keys written by API/Backend/Upload/uploadRouter.js, not the bytes of -// the regex. A value one treats as an upload key and another as a -// mission-relative path renders a broken image only at runtime. What each -// consumer then does with a matched key differs by design and is asserted -// per classifier below. The third classifier, cacheControlForKey in -// scripts/lib/aws-provision.js, is added to this table by the PR that -// introduces it. +// classifier: the app bundle, the Configure SPA and the publish scripts. None +// of the three can share a module with the others, so each carries its own +// regex; what they must agree on is which values are upload keys written by +// API/Backend/Upload/uploadRouter.js, not the bytes of the regex. A value one +// treats as an upload key and another as a mission-relative path renders a +// broken image only at runtime. What each consumer then does with a matched +// key differs by design and is asserted per classifier below. const MISSION = 'M' const MISSION_PATH = 'Missions/M/' @@ -47,4 +47,16 @@ test.describe('upload-key classification', () => { key === null ? `${BASE}Missions/${MISSION}/${value}` : BASE + key, ) }) + + // The publish gives a matched key the immutable tier, because the upload + // router names those files crypto.randomUUID() and never overwrites one. + // The rooted row is not a key the publish can ever see — S3 object keys + // have no leading slash — so it lands on the short tier here. + test.each(VALUES)('cacheControlForKey: %s', (value, key) => { + expect(cacheControlForKey(value)).toBe( + key !== null && !value.startsWith('/') + ? 'public, max-age=31536000, immutable' + : 'public, max-age=300', + ) + }) }) diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index bce2522ba..a33816885 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -1,64 +1,70 @@ import { test, expect } from 'vitest' -import fs from 'fs' -import path from 'path' // Pins the premise of the immutable Cache-Control tier in // scripts/lib/aws-provision.js: every production filename webpack writes under // build/static/(js|css|media) carries a content hash. Drop the hash from one of -// these options (or rename it) and a stable filename lands in the immutable -// tier, where customers' CloudFront edges would pin it for a year. -const CONFIG = fs.readFileSync( - path.join(__dirname, '..', '..', 'configuration', 'webpack.config.js'), - 'utf8' -) +// these options, or send a verbatim copy to one of those prefixes, and a stable +// filename lands in the immutable tier, where customers' CloudFront edges would +// pin it for a year. +// +// configuration/env.js throws without NODE_ENV, and building the config +// installs its own crypto.createHash, so the variable is set here and the +// require stays inside this one spec file. +process.env.NODE_ENV = 'production' +const CONFIG = require('../../configuration/webpack.config.js')('production') const HASH_TOKEN = /\[contenthash|\[hash/ -// The literal `re` captures, failing the test when it matches nothing — a -// renamed or restructured option must break the suite, not skip its assertion. -function capture(label, re, text = CONFIG) { - const match = text.match(re) - expect(match, `no match for ${label}`).not.toBeNull() - return match[1] +// Every loader `options.name` that emits into static/media, however the rules +// are nested — the production asset loaders sit inside a `oneOf`. +function mediaNames(rules, found = []) { + const list = rules || [] + list.forEach((rule) => { + if (!rule) return + mediaNames(rule.oneOf, found) + mediaNames(rule.rules, found) + const name = rule.options && rule.options.name + if (typeof name === 'string' && name.startsWith('static/media/')) + found.push(name) + }) + return found +} + +function pluginNamed(name) { + const plugin = CONFIG.plugins.find( + (p) => p && p.constructor && p.constructor.name === name + ) + expect(plugin, `no ${name} in the production config`).toBeDefined() + return plugin } test.describe('webpack production output is content-hashed', () => { test('output.filename and output.chunkFilename carry a hash', () => { - // Leading [^A-Za-z] so `filename:` does not match `chunkFilename:`. - expect( - capture( - 'output.filename', - /[^A-Za-z]filename:\s*isEnvProduction\s*\?\s*"([^"]+)"/ - ) - ).toMatch(HASH_TOKEN) - expect( - capture( - 'output.chunkFilename', - /chunkFilename:\s*isEnvProduction\s*\?\s*"([^"]+)"/ - ) - ).toMatch(HASH_TOKEN) + expect(CONFIG.output.filename).toMatch(HASH_TOKEN) + expect(CONFIG.output.chunkFilename).toMatch(HASH_TOKEN) }) test("MiniCssExtractPlugin's filenames carry a hash", () => { - const options = capture( - 'MiniCssExtractPlugin options', - /new MiniCssExtractPlugin\(\{([\s\S]*?)\}\)/ - ) - expect( - capture('css filename', /[^A-Za-z]filename:\s*"([^"]+)"/, options) - ).toMatch(HASH_TOKEN) - expect( - capture('css chunkFilename', /chunkFilename:\s*"([^"]+)"/, options) - ).toMatch(HASH_TOKEN) + const options = pluginNamed('MiniCssExtractPlugin').options + expect(options.filename).toMatch(HASH_TOKEN) + expect(options.chunkFilename).toMatch(HASH_TOKEN) }) test('the media loaders name files with a hash', () => { - // The url-loader (small images, inlined above a size limit) and the + // The url-loader (small images, inlined below a size limit) and the // catch-all file-loader both emit into static/media. - const names = [ - ...CONFIG.matchAll(/name:\s*"(static\/media\/[^"]+)"/g), - ].map((m) => m[1]) + const names = mediaNames(CONFIG.module.rules) expect(names.length).toBeGreaterThanOrEqual(2) names.forEach((name) => expect(name).toMatch(HASH_TOKEN)) }) + + test('no copied file lands under a hashed prefix', () => { + // CopyPlugin passes files through under their own names, so a + // destination under static/js, static/css or static/media would drop a + // stable name into the immutable tier. Cesium's copies go to + // static/cesium, which is on the five-minute tier. + pluginNamed('CopyPlugin').patterns.forEach((pattern) => { + expect(pattern.to).not.toMatch(/^static\/(js|css|media)\//) + }) + }) }) From 382dc361b3809240aa04e7c47eef19bf782dcddb Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 16:28:55 -0500 Subject: [PATCH 03/13] [316] Scope the customer doc's cache promises to what each tier delivers --- docs/infrastructure/serving-a-dashboard-from-your-domain.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index 0040360d9..b2e635d3e 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -52,7 +52,7 @@ Every dashboard is password-protected, so every row also sits behind the passwor **Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the pages we mark for revalidation, so a republish would not reach your visitors until that floor expired. -**Maximum TTL a year:** our responses carry three cache tiers. The entry page and the mission configuration revalidate before every use (`no-cache`, not `no-store` — a republish shows up on your domain immediately, with no purge on your side). The files whose names are content-fingerprinted — the application's own bundles, and the images and files uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`). Everything else, the supporting files that can change in place, gets five minutes. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. +**Maximum TTL a year:** our responses carry three cache tiers. The entry page and the mission configuration revalidate before every use (`no-cache`, not `no-store` — a republish reaches your domain with no purge on your side: the entry page and the mission configuration immediately, the supporting files within five minutes). The files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images and files uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`). Everything else, the supporting files that can change in place, gets five minutes. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. **HTTPS only to the origin:** the dashboard's password rides on the `Authorization` header of every request you forward. Over plain HTTP it would cross the internet unencrypted. From cab8324e741f1accde47d82398f390336dc58ba9 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 17:28:13 -0500 Subject: [PATCH 04/13] [316] Say why the tiers are public, guard every copy target, and skip both raw templates --- configure/src/core/upload.js | 6 ++-- scripts/lib/aws-provision.js | 44 ++++++++++++++++---------- scripts/lib/cfn-template.js | 5 ++- scripts/publish-static.js | 7 ++-- tests/unit/awsProvision.spec.js | 30 ++++++------------ tests/unit/webpackHashedOutput.spec.js | 39 +++++++++++++++++------ 6 files changed, 80 insertions(+), 51 deletions(-) diff --git a/configure/src/core/upload.js b/configure/src/core/upload.js index 7c13cd349..0f226a580 100644 --- a/configure/src/core/upload.js +++ b/configure/src/core/upload.js @@ -34,8 +34,10 @@ export async function uploadImage(file, mission, subdir) { // The CMS is a separate bundle with no import path into the app's, so this // is a copy of ASSETS_UPLOAD_KEY in src/pre/uploadKey.ts, where the shape it -// matches is explained. tests/unit/uploadKeyClassifier.spec.js runs one table -// of values through both and fails if they classify any of them differently. +// matches is explained. scripts/lib/aws-provision.js carries a third copy, a +// CommonJS module run by Node. tests/unit/uploadKeyClassifier.spec.js runs one +// table of values through all three and fails if they classify any of them +// differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // Turns a stored upload-field value into the URL the CMS's preview diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index f81107fe5..af71ebf66 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -424,6 +424,15 @@ const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // under build/static/cesium, so a republish that bumps the MMGIS version can // leave a customer's edge pairing a new bundle with a copy of those up to five // minutes old. +// +// The two cacheable tiers say "public" because every response sits behind +// Basic auth, and RFC 7234 lets a shared cache store a response to an +// Authorization-bearing request only when it is marked that way — the +// customer's own CloudFront is a shared cache keyed on Authorization, so +// "public" is what lets it cache at all. The cost is that any other shared +// cache on a visitor's path, a corporate proxy say, may also store the +// immutable and five-minute tiers and serve them to someone who never +// authenticated; the no-cache tier revalidates and so 401s there. function cacheControlForKey(key) { if ( key === "index.html" || @@ -455,8 +464,9 @@ function walkDirectory(dir, baseDir) { } // Uploads every file under `dir` to `bucket`, keys relative to `dir` -// (optionally prefixed). `filter` receives that relative key and keeps the -// file when it returns true. Returns the number of files uploaded. +// (optionally prefixed). `filter` receives the prefixed key — the same string +// cacheControlForKey is given — and keeps the file when it returns true. +// Returns the number of files uploaded. async function uploadDirectory({ bucket, dir, @@ -466,7 +476,7 @@ async function uploadDirectory({ }) { const { s3 } = getClients(); const files = filter - ? walkDirectory(dir).filter((file) => filter(file.key)) + ? walkDirectory(dir).filter((file) => filter(`${prefix}${file.key}`)) : walkDirectory(dir); let index = 0; async function worker() { @@ -509,11 +519,13 @@ async function uploadFile({ bucket, key, filePath }) { ); } -// Invalidates paths on our own distribution, the only one this reaches. The -// Cache-Control tiers already cover most of it there — index.html and -// config.json revalidate every request, and hashed bundles arrive under new -// names — so this is what closes the gap for the five-minute tier. Any other -// edge in front of the dashboard is governed by those headers alone. +// Invalidates paths on our own distribution, the only one this reaches. Its +// cache policy is the managed CachingOptimized, whose Minimum TTL of 1 s +// overrides the no-cache tier at this edge, so index.html and config.json are +// bounded at a second stale rather than revalidated; the fallback tier is +// bounded at five minutes. Hashed bundles need nothing, arriving under new +// names, so the "/*" invalidation is what clears those two tiers here. Any +// other edge in front of the dashboard is governed by the headers alone. async function createInvalidation({ distributionId, paths = ["/*"] }) { const { cloudfront } = getClients(); await cloudfront.send( @@ -538,7 +550,12 @@ function buildCopySource(bucket, key) { } // Same-key copies every object under `prefix` from sourceBucket into -// destBucket. Returns the number of objects copied. +// destBucket, giving each copy the Cache-Control tier for its key. COPY (the +// default) cannot set headers the source object never had, so that takes +// MetadataDirective: REPLACE, which drops the source's entire metadata set — +// Content-Encoding, Content-Disposition and any x-amz-meta-* are lost unless +// restated alongside Content-Type. Nothing sets those: the upload router +// writes ContentType alone. Returns the number of objects copied. async function copyPrefix({ sourceBucket, destBucket, prefix }) { const { s3 } = getClients(); let copied = 0; @@ -557,13 +574,8 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { Bucket: destBucket, Key: obj.Key, CopySource: buildCopySource(sourceBucket, obj.Key), - // COPY (the default) cannot set new headers on the copy, so - // REPLACE is required to add a Cache-Control the source object - // never had — and REPLACE means supplying ContentType too. - // REPLACE drops the source's entire metadata set, not just its - // Content-Type: Content-Encoding, Content-Disposition and any - // x-amz-meta-* are lost unless restated here. Nothing sets those: - // the upload router writes ContentType alone. + // REPLACE so the copy carries the Cache-Control and Content-Type + // set here rather than the source's metadata. MetadataDirective: "REPLACE", ContentType: contentTypeForFile(obj.Key), CacheControl: cacheControlForKey(obj.Key), diff --git a/scripts/lib/cfn-template.js b/scripts/lib/cfn-template.js index 84408f087..6c87258c2 100644 --- a/scripts/lib/cfn-template.js +++ b/scripts/lib/cfn-template.js @@ -230,7 +230,10 @@ function renderCfnTemplate({ password } = {}) { // AWS managed policy: CachingOptimized — minimum TTL 1 s, // default 86400 s, maximum 31536000 s. The Cache-Control tiers // in scripts/lib/aws-provision.js rely on that maximum being at - // least a year, or the edge would cap the immutable tier. + // least a year, or the edge would cap the immutable tier. The + // minimum overrides the no-cache tier here, so the entry page + // and the baked config are at most a second stale at this edge + // rather than revalidated on every request. CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6", FunctionAssociations: [ { diff --git a/scripts/publish-static.js b/scripts/publish-static.js index f2ac80f3e..2b050a000 100644 --- a/scripts/publish-static.js +++ b/scripts/publish-static.js @@ -271,17 +271,20 @@ async function main() { // bucket must mirror that layout: the webpack output under build/, // the repo's public/ assets under public/, and index.html at the // root (the distribution's default root object). + // Both filters skip a copy of the un-rendered Pug template: scripts/build.js + // writes one into build/ beside the page, and public/index.html is the + // source the build renders from. The rendered page is uploaded below. const uploadedBuild = await provision.uploadDirectory({ bucket, dir: path.join(rootDir, "build"), prefix: "build/", + filter: (key) => key !== "build/index.pug", }); const uploadedPublic = await provision.uploadDirectory({ bucket, dir: path.join(rootDir, "public"), prefix: "public/", - // public/index.html is the un-rendered Pug template. - filter: (key) => key !== "index.html", + filter: (key) => key !== "public/index.html", }); await provision.uploadFile({ bucket, diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 6104f0e0a..2971b44c3 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -904,7 +904,9 @@ test.describe('cacheControlForKey', () => { // [key, expected Cache-Control]. Three tiers: revalidate-always for the // entry page and the baked config, immutable for the content-addressed // keys (hashed webpack output and the never-overwritten plugin uploads), - // a short TTL for everything else. + // a short TTL for everything else. The upload-key half of the immutable + // tier belongs to tests/unit/uploadKeyClassifier.spec.js, which checks it + // against the other two copies of that classifier. const TIERS = [ ['index.html', 'no-cache'], ['build/index.html', 'no-cache'], @@ -915,23 +917,11 @@ test.describe('cacheControlForKey', () => { ], ['build/static/css/x.css', 'public, max-age=31536000, immutable'], ['build/static/media/a.png', 'public, max-age=31536000, immutable'], - ['Missions/M/Data/waypoints.csv', 'public, max-age=300'], + ['build/asset-manifest.json', 'public, max-age=300'], // Under build/static but not content-hashed, so explicitly NOT // immutable. ['build/static/cesium/Cesium.js', 'public, max-age=300'], ['public/workers/pdf.worker.min.mjs', 'public, max-age=300'], - // The upload router names every object crypto.randomUUID(). and - // never overwrites, so the key is content-addressed in practice. - [ - 'assets/M/CardPlugin/uploads/a.png', - 'public, max-age=31536000, immutable', - ], - // Under assets/ but not the writer's shape (no /uploads/ segment two - // levels down), so it stays on the fallback tier. - ['assets/M/CardPlugin/icon.png', 'public, max-age=300'], - // A lookalike: "uploads" here is the mission segment, not the - // router's directory, so it is not the content-addressed shape. - ['assets/uploads/a.png', 'public, max-age=300'], ] TIERS.forEach(([key, expected]) => { @@ -995,15 +985,17 @@ test.describe('uploadDirectory', () => { }) }) - test('skips the files filter rejects, by their unprefixed key', async () => { + test('skips the files filter rejects, by their prefixed key', async () => { await withUploadFixture(async (dir, puts) => { + // The filter names the prefixed key, so it fails to match — and + // nothing is skipped — if the filter is handed the relative one. fs.writeFileSync(path.join(dir, 'index.html'), '') fs.writeFileSync(path.join(dir, 'keep.txt'), 'keep') const count = await provision.uploadDirectory({ bucket: 'dash', dir, prefix: 'public/', - filter: (key) => key !== 'index.html', + filter: (key) => key !== 'public/index.html', }) expect(count).toBe(1) expect(puts.map((input) => input.Key)).toEqual(['public/keep.txt']) @@ -1082,11 +1074,7 @@ test.describe('copyPrefix', () => { expect(copies[2].CopySource).toBe( 'shared/assets/TestMission/with%20space.png' ) - // CopyObject's default (COPY) keeps the source's metadata and cannot - // add the Cache-Control the source never had; REPLACE can, and in turn - // obliges the copy to restate its Content-Type. Tier coverage lives in - // the cacheControlForKey table — this pins the wiring at this site, - // across both tiers a copied object can land on. + // REPLACE lets the copy carry its own Cache-Control and Content-Type. expect(copies[0].MetadataDirective).toBe('REPLACE') expect(copies[0].ContentType).toBe('image/png') expect(copies[0].CacheControl).toBe( diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index a33816885..b8f531a03 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -1,4 +1,5 @@ -import { test, expect } from 'vitest' +import { afterAll, test, expect } from 'vitest' +import path from 'path' // Pins the premise of the immutable Cache-Control tier in // scripts/lib/aws-provision.js: every production filename webpack writes under @@ -7,12 +8,19 @@ import { test, expect } from 'vitest' // filename lands in the immutable tier, where customers' CloudFront edges would // pin it for a year. // -// configuration/env.js throws without NODE_ENV, and building the config -// installs its own crypto.createHash, so the variable is set here and the -// require stays inside this one spec file. +// configuration/env.js throws without NODE_ENV, so the variable is set here +// and put back afterwards. Building the config also patches +// crypto.createHash process-wide, which anything else in the same worker would +// then inherit, so the require stays inside this one spec file. +const NODE_ENV_BEFORE = process.env.NODE_ENV process.env.NODE_ENV = 'production' const CONFIG = require('../../configuration/webpack.config.js')('production') +afterAll(() => { + if (NODE_ENV_BEFORE === undefined) delete process.env.NODE_ENV + else process.env.NODE_ENV = NODE_ENV_BEFORE +}) + const HASH_TOKEN = /\[contenthash|\[hash/ // Every loader `options.name` that emits into static/media, however the rules @@ -30,10 +38,14 @@ function mediaNames(rules, found = []) { return found } -function pluginNamed(name) { - const plugin = CONFIG.plugins.find( +function pluginsNamed(name) { + return CONFIG.plugins.filter( (p) => p && p.constructor && p.constructor.name === name ) +} + +function pluginNamed(name) { + const plugin = pluginsNamed(name)[0] expect(plugin, `no ${name} in the production config`).toBeDefined() return plugin } @@ -62,9 +74,18 @@ test.describe('webpack production output is content-hashed', () => { // CopyPlugin passes files through under their own names, so a // destination under static/js, static/css or static/media would drop a // stable name into the immutable tier. Cesium's copies go to - // static/cesium, which is on the five-minute tier. - pluginNamed('CopyPlugin').patterns.forEach((pattern) => { - expect(pattern.to).not.toMatch(/^static\/(js|css|media)\//) + // static/cesium, which is on the five-minute tier. Destinations are + // built with path.join, so they carry the platform's separator. + const copiers = pluginsNamed('CopyPlugin') + expect( + copiers.length, + 'no CopyPlugin in the production config' + ).toBeGreaterThan(0) + copiers.forEach((copier) => { + copier.patterns.forEach((pattern) => { + const to = String(pattern.to).split(path.sep).join('/') + expect(to).not.toMatch(/^static\/(js|css|media)(\/|$)/) + }) }) }) }) From 6baba7bf1cb0994c86ba7c21a58e87ba8915e201 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 17:28:13 -0500 Subject: [PATCH 05/13] [316] Give each TTL bullet one setting and name the shared-cache mark --- docs/infrastructure/serving-a-dashboard-from-your-domain.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index b2e635d3e..4f3603a13 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -46,13 +46,13 @@ Every dashboard is password-protected, so every row also sits behind the passwor **The header** tells us how much of the forwarded path is yours. CloudFront forwards the full path exactly as the visitor typed it, so our side receives `/tools/dashboard/index.html` and needs to know that `/tools/dashboard` is prefix, not content. We remove exactly what the header declares and serve the file. If the header is missing or doesn't match the path, every request under the path fails with a 403 immediately — a loud failure on purpose, instead of quietly serving the wrong files. (It's a 403 rather than a 404 because the rejection comes from our storage layer, which answers "access denied.") -**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. The password is shared by every dashboard published from the same MMGIS environment, so it is an internal credential, not something to hand to your visitors. +**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Our cacheable responses are marked `public` — the mark a shared cache needs before it may store a response to a request that carried an `Authorization` header — precisely so your CloudFront can hold them under that per-credential key. The password is shared by every dashboard published from the same MMGIS environment, so it is an internal credential, not something to hand to your visitors. **All query strings in the cache key:** a policy that drops query strings never sends them to us. A deep link like `/tools/dashboard?view=2` then reaches us stripped of its `?view=2`, and the address we redirect the visitor to has lost it for good. -**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the pages we mark for revalidation, so a republish would not reach your visitors until that floor expired. +**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files within five minutes. -**Maximum TTL a year:** our responses carry three cache tiers. The entry page and the mission configuration revalidate before every use (`no-cache`, not `no-store` — a republish reaches your domain with no purge on your side: the entry page and the mission configuration immediately, the supporting files within five minutes). The files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images and files uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`). Everything else, the supporting files that can change in place, gets five minutes. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. +**Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. **HTTPS only to the origin:** the dashboard's password rides on the `Authorization` header of every request you forward. Over plain HTTP it would cross the internet unencrypted. From 4a9b93b4fa7c0bfd6bd231430382f0c8c70b9eec Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 17:47:19 -0500 Subject: [PATCH 06/13] [316] Keep a copied object's own Content-Type when its extension is unmapped, and pin the cache policy --- scripts/lib/aws-provision.js | 54 ++++++++++++++++---------- scripts/publish-static.js | 8 ++-- tests/unit/awsProvision.spec.js | 21 ++++++++-- tests/unit/cfnTemplate.spec.js | 5 +++ tests/unit/infrastructure.spec.js | 14 +++++++ tests/unit/uploadKeyClassifier.spec.js | 21 +++++----- tests/unit/webpackHashedOutput.spec.js | 44 ++++++++++++++------- 7 files changed, 116 insertions(+), 51 deletions(-) diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index af71ebf66..6e03601fc 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -23,6 +23,7 @@ const { S3Client, PutObjectCommand, CopyObjectCommand, + HeadObjectCommand, ListObjectsV2Command, DeleteObjectsCommand, } = require("@aws-sdk/client-s3"); @@ -426,13 +427,11 @@ const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // minutes old. // // The two cacheable tiers say "public" because every response sits behind -// Basic auth, and RFC 7234 lets a shared cache store a response to an -// Authorization-bearing request only when it is marked that way — the -// customer's own CloudFront is a shared cache keyed on Authorization, so -// "public" is what lets it cache at all. The cost is that any other shared -// cache on a visitor's path, a corporate proxy say, may also store the -// immutable and five-minute tiers and serve them to someone who never -// authenticated; the no-cache tier revalidates and so 401s there. +// Basic auth, and RFC 7234 §3.2 lets a shared cache reuse a response to an +// Authorization-bearing request only when it carries must-revalidate, public +// or s-maxage, so "public" removes any conformance question. What a customer +// fronting the dashboard gets from that is spelled out in +// docs/infrastructure/serving-a-dashboard-from-your-domain.md. function cacheControlForKey(key) { if ( key === "index.html" || @@ -520,12 +519,11 @@ async function uploadFile({ bucket, key, filePath }) { } // Invalidates paths on our own distribution, the only one this reaches. Its -// cache policy is the managed CachingOptimized, whose Minimum TTL of 1 s -// overrides the no-cache tier at this edge, so index.html and config.json are -// bounded at a second stale rather than revalidated; the fallback tier is -// bounded at five minutes. Hashed bundles need nothing, arriving under new -// names, so the "/*" invalidation is what clears those two tiers here. Any -// other edge in front of the dashboard is governed by the headers alone. +// cache policy (see the CachePolicyId in scripts/lib/cfn-template.js) leaves +// the entry page, the baked config and the fallback tier holdable at this +// edge; hashed bundles need nothing, arriving under new names, so the "/*" +// invalidation is what clears those tiers here. Any other edge in front of +// the dashboard is governed by the headers alone. async function createInvalidation({ distributionId, paths = ["/*"] }) { const { cloudfront } = getClients(); await cloudfront.send( @@ -549,13 +547,26 @@ function buildCopySource(bucket, key) { return `${bucket}/${encodedKey}`; } +// Content-Type for a copied object: the CONTENT_TYPES mapping when the +// extension is one it names, otherwise the source object's own header read +// with HeadObject, and octet-stream when the source carries none either. +async function copiedContentType({ s3, sourceBucket, key }) { + if (CONTENT_TYPES[path.extname(key).toLowerCase()] != null) + return contentTypeForFile(key); + const head = await s3.send( + new HeadObjectCommand({ Bucket: sourceBucket, Key: key }) + ); + return head.ContentType || "application/octet-stream"; +} + // Same-key copies every object under `prefix` from sourceBucket into // destBucket, giving each copy the Cache-Control tier for its key. COPY (the // default) cannot set headers the source object never had, so that takes -// MetadataDirective: REPLACE, which drops the source's entire metadata set — -// Content-Encoding, Content-Disposition and any x-amz-meta-* are lost unless -// restated alongside Content-Type. Nothing sets those: the upload router -// writes ContentType alone. Returns the number of objects copied. +// MetadataDirective: REPLACE, which rewrites the metadata of every key under +// the prefix — Content-Encoding, Content-Disposition and any x-amz-meta-* are +// dropped unless restated alongside Content-Type. Nothing sets those: the +// upload router writes ContentType alone. Returns the number of objects +// copied. async function copyPrefix({ sourceBucket, destBucket, prefix }) { const { s3 } = getClients(); let copied = 0; @@ -569,15 +580,18 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { }) ); for (const obj of list.Contents || []) { + const contentType = await copiedContentType({ + s3, + sourceBucket, + key: obj.Key, + }); await s3.send( new CopyObjectCommand({ Bucket: destBucket, Key: obj.Key, CopySource: buildCopySource(sourceBucket, obj.Key), - // REPLACE so the copy carries the Cache-Control and Content-Type - // set here rather than the source's metadata. MetadataDirective: "REPLACE", - ContentType: contentTypeForFile(obj.Key), + ContentType: contentType, CacheControl: cacheControlForKey(obj.Key), }) ); diff --git a/scripts/publish-static.js b/scripts/publish-static.js index 2b050a000..495071772 100644 --- a/scripts/publish-static.js +++ b/scripts/publish-static.js @@ -271,9 +271,11 @@ async function main() { // bucket must mirror that layout: the webpack output under build/, // the repo's public/ assets under public/, and index.html at the // root (the distribution's default root object). - // Both filters skip a copy of the un-rendered Pug template: scripts/build.js - // writes one into build/ beside the page, and public/index.html is the - // source the build renders from. The rendered page is uploaded below. + // Both filters skip a template that still carries the raw #{...} + // placeholders: build/index.pug is the Pug conversion scripts/build.js + // writes into build/, and public/index.html is the HTML template the + // build renders from. Either one served as-is would hand a visitor the + // placeholders un-interpolated. The rendered page is uploaded below. const uploadedBuild = await provision.uploadDirectory({ bucket, dir: path.join(rootDir, "build"), diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 2971b44c3..e46bfd340 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -1031,6 +1031,7 @@ test.describe('copyPrefix', () => { test('same-key copies every object under the prefix', async () => { const copies = [] + const heads = [] provision.setClients({ s3: mockClient((command) => { const name = command.constructor.name @@ -1044,10 +1045,17 @@ test.describe('copyPrefix', () => { }, { Key: 'assets/TestMission/icon.png' }, { Key: 'assets/TestMission/with space.png' }, + { Key: 'assets/TestMission/photo.jpg' }, + // An extension the Content-Type table does not name. + { Key: 'assets/TestMission/scan.tif' }, ], IsTruncated: false, } } + if (name === 'HeadObjectCommand') { + heads.push(command.input.Key) + return { ContentType: 'image/tiff' } + } if (name === 'CopyObjectCommand') { copies.push(command.input) return {} @@ -1060,7 +1068,7 @@ test.describe('copyPrefix', () => { destBucket: 'dash', prefix: 'assets/TestMission/', }) - expect(count).toBe(3) + expect(count).toBe(5) // Same keys in the destination bucket expect(copies[1].Bucket).toBe('dash') expect(copies[1].Key).toBe('assets/TestMission/icon.png') @@ -1076,13 +1084,18 @@ test.describe('copyPrefix', () => { ) // REPLACE lets the copy carry its own Cache-Control and Content-Type. expect(copies[0].MetadataDirective).toBe('REPLACE') - expect(copies[0].ContentType).toBe('image/png') + // An upload key gets the immutable tier, everything else the short one. expect(copies[0].CacheControl).toBe( 'public, max-age=31536000, immutable' ) - expect(copies[1].MetadataDirective).toBe('REPLACE') - expect(copies[1].ContentType).toBe('image/png') expect(copies[1].CacheControl).toBe('public, max-age=300') + // A mapped extension is typed from the key alone... + expect(copies[1].ContentType).toBe('image/png') + expect(copies[3].ContentType).toBe('image/jpeg') + // ...and only an unmapped one costs a HeadObject, which is what keeps + // the source's own type instead of downgrading it to octet-stream. + expect(heads).toEqual(['assets/TestMission/scan.tif']) + expect(copies[4].ContentType).toBe('image/tiff') }) }) diff --git a/tests/unit/cfnTemplate.spec.js b/tests/unit/cfnTemplate.spec.js index 385c3ff59..f3b0cdc1b 100644 --- a/tests/unit/cfnTemplate.spec.js +++ b/tests/unit/cfnTemplate.spec.js @@ -136,6 +136,11 @@ test.describe('renderCfnTemplate', () => { template.Resources.DashboardDistribution.Properties .DistributionConfig expect(dist.DefaultRootObject).toBe('index.html') + // The managed CachingOptimized policy, whose maximum TTL of a year is + // what lets the immutable Cache-Control tier survive this edge. + expect(dist.DefaultCacheBehavior.CachePolicyId).toBe( + '658327ea-f89d-4fab-a63d-7e88639e58f6' + ) const associations = dist.DefaultCacheBehavior.FunctionAssociations expect(associations).toHaveLength(1) expect(associations[0].EventType).toBe('viewer-request') diff --git a/tests/unit/infrastructure.spec.js b/tests/unit/infrastructure.spec.js index ef05eb305..d0eea78ed 100644 --- a/tests/unit/infrastructure.spec.js +++ b/tests/unit/infrastructure.spec.js @@ -354,6 +354,20 @@ test.describe('infrastructure/ recipes (JSON and Terraform)', () => { expect(adminEnv.DISABLE_FIRST_SIGNUP).toBe('true') }) + test('the publish skips both un-interpolated index templates', () => { + // build/index.pug and public/index.html still carry the raw #{...} + // placeholders, so either one in the bucket is a page a visitor can + // reach with the placeholders showing. The upload filters name the two + // keys literally; pinning them here catches a rename on one side that + // never reaches the other. + const source = fs.readFileSync( + path.join(ROOT, 'scripts', 'publish-static.js'), + 'utf8' + ) + expect(source).toContain('key !== "build/index.pug"') + expect(source).toContain('key !== "public/index.html"') + }) + test('publish task role omits rds-db:connect (password auth only)', () => { for (const file of IAM_FILES) { const actions = statementsOf(readJson(file)).flatMap((s) => diff --git a/tests/unit/uploadKeyClassifier.spec.js b/tests/unit/uploadKeyClassifier.spec.js index 954a31a15..6ae367365 100644 --- a/tests/unit/uploadKeyClassifier.spec.js +++ b/tests/unit/uploadKeyClassifier.spec.js @@ -50,13 +50,16 @@ test.describe('upload-key classification', () => { // The publish gives a matched key the immutable tier, because the upload // router names those files crypto.randomUUID() and never overwrites one. - // The rooted row is not a key the publish can ever see — S3 object keys - // have no leading slash — so it lands on the short tier here. - test.each(VALUES)('cacheControlForKey: %s', (value, key) => { - expect(cacheControlForKey(value)).toBe( - key !== null && !value.startsWith('/') - ? 'public, max-age=31536000, immutable' - : 'public, max-age=300', - ) - }) + // The rooted row sits out: S3 object keys have no leading slash, so it is + // not a value this classifier is ever handed. + test.each(VALUES.filter(([value]) => !value.startsWith('/')))( + 'cacheControlForKey: %s', + (value, key) => { + expect(cacheControlForKey(value)).toBe( + key !== null + ? 'public, max-age=31536000, immutable' + : 'public, max-age=300', + ) + }, + ) }) diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index b8f531a03..eb5fc6f22 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -1,4 +1,5 @@ import { afterAll, test, expect } from 'vitest' +import crypto from 'crypto' import path from 'path' // Pins the premise of the immutable Cache-Control tier in @@ -11,29 +12,33 @@ import path from 'path' // configuration/env.js throws without NODE_ENV, so the variable is set here // and put back afterwards. Building the config also patches // crypto.createHash process-wide, which anything else in the same worker would -// then inherit, so the require stays inside this one spec file. +// then inherit, so the original is captured here and restored too. const NODE_ENV_BEFORE = process.env.NODE_ENV +const CREATE_HASH_BEFORE = crypto.createHash process.env.NODE_ENV = 'production' const CONFIG = require('../../configuration/webpack.config.js')('production') afterAll(() => { if (NODE_ENV_BEFORE === undefined) delete process.env.NODE_ENV else process.env.NODE_ENV = NODE_ENV_BEFORE + crypto.createHash = CREATE_HASH_BEFORE }) const HASH_TOKEN = /\[contenthash|\[hash/ -// Every loader `options.name` that emits into static/media, however the rules -// are nested — the production asset loaders sit inside a `oneOf`. -function mediaNames(rules, found = []) { +// Every emitted-file name pattern in the rules, however they are nested — the +// production asset loaders sit inside a `oneOf`. Loaders name their output +// with `options.name`; webpack 5 asset modules use `generator.filename`. +function emittedNames(rules, found = []) { const list = rules || [] list.forEach((rule) => { if (!rule) return - mediaNames(rule.oneOf, found) - mediaNames(rule.rules, found) + emittedNames(rule.oneOf, found) + emittedNames(rule.rules, found) const name = rule.options && rule.options.name - if (typeof name === 'string' && name.startsWith('static/media/')) - found.push(name) + if (typeof name === 'string') found.push(name) + const generated = rule.generator && rule.generator.filename + if (typeof generated === 'string') found.push(generated) }) return found } @@ -51,23 +56,32 @@ function pluginNamed(name) { } test.describe('webpack production output is content-hashed', () => { - test('output.filename and output.chunkFilename carry a hash', () => { + test('output.filename and output.chunkFilename land hashed under static/js', () => { expect(CONFIG.output.filename).toMatch(HASH_TOKEN) + expect(CONFIG.output.filename).toMatch(/^static\/js\//) expect(CONFIG.output.chunkFilename).toMatch(HASH_TOKEN) + expect(CONFIG.output.chunkFilename).toMatch(/^static\/js\//) }) - test("MiniCssExtractPlugin's filenames carry a hash", () => { + test("MiniCssExtractPlugin's filenames land hashed under static/css", () => { const options = pluginNamed('MiniCssExtractPlugin').options expect(options.filename).toMatch(HASH_TOKEN) + expect(options.filename).toMatch(/^static\/css\//) expect(options.chunkFilename).toMatch(HASH_TOKEN) + expect(options.chunkFilename).toMatch(/^static\/css\//) }) - test('the media loaders name files with a hash', () => { + test('every rule names its emitted files hashed under static/media', () => { // The url-loader (small images, inlined below a size limit) and the - // catch-all file-loader both emit into static/media. - const names = mediaNames(CONFIG.module.rules) - expect(names.length).toBeGreaterThanOrEqual(2) - names.forEach((name) => expect(name).toMatch(HASH_TOKEN)) + // catch-all file-loader both emit into static/media; every prefix in + // the immutable tier has to be hashed, so a rule emitting elsewhere + // under a hashless name is caught here too. + const names = emittedNames(CONFIG.module.rules) + expect(names.length).toBeGreaterThan(0) + names.forEach((name) => { + expect(name).toMatch(/^static\/media\//) + expect(name).toMatch(HASH_TOKEN) + }) }) test('no copied file lands under a hashed prefix', () => { From d5b9f984f92fa5cebf6c1ba5ab41664379916bf9 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 17:47:19 -0500 Subject: [PATCH 07/13] [316] State the shared-cache rule precisely and add the Default TTL bullet --- .../infrastructure/serving-a-dashboard-from-your-domain.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index 4f3603a13..b8f736b79 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -12,7 +12,8 @@ The examples below use the path `/tools/dashboard`; substitute your own everywhe - the `Authorization` header in the cache key, - all query strings in the cache key, - Minimum TTL 0, - - Maximum TTL 31536000 (one year) or more. + - Maximum TTL 31536000 (one year) or more, + - Default TTL — any value; every response carries its own `Cache-Control`. 3. **Add two cache behaviors** pointing at that origin, both using that cache policy: - path pattern `/tools/dashboard` — exact, no wildcard, @@ -46,11 +47,11 @@ Every dashboard is password-protected, so every row also sits behind the passwor **The header** tells us how much of the forwarded path is yours. CloudFront forwards the full path exactly as the visitor typed it, so our side receives `/tools/dashboard/index.html` and needs to know that `/tools/dashboard` is prefix, not content. We remove exactly what the header declares and serve the file. If the header is missing or doesn't match the path, every request under the path fails with a 403 immediately — a loud failure on purpose, instead of quietly serving the wrong files. (It's a 403 rather than a 404 because the rejection comes from our storage layer, which answers "access denied.") -**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Our cacheable responses are marked `public` — the mark a shared cache needs before it may store a response to a request that carried an `Authorization` header — precisely so your CloudFront can hold them under that per-credential key. The password is shared by every dashboard published from the same MMGIS environment, so it is an internal credential, not something to hand to your visitors. +**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Our cacheable responses are marked `public` — RFC 7234 §3.2 lets a shared cache reuse a response to a request that carried an `Authorization` header only when it is marked `must-revalidate`, `public` or `s-maxage`, so the mark leaves your CloudFront no conformance question about holding them under that per-credential key. The mark travels with the response, so any other shared cache between the visitor and you may store the long-lived tiers too — the bundles and uploaded images, and the supporting files for five minutes. The password is shared by every dashboard published from the same MMGIS environment, so it is an internal credential, not something to hand to your visitors. **All query strings in the cache key:** a policy that drops query strings never sends them to us. A deep link like `/tools/dashboard?view=2` then reaches us stripped of its `?view=2`, and the address we redirect the visitor to has lost it for good. -**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files within five minutes. +**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files within about five minutes of the publish completing. **Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. From d99cec9f4c4aeb8a2016376c1fb73bb4707d08da Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:15:32 -0500 Subject: [PATCH 08/13] [316] Drop public from the cache tiers and read the upload-key shape from the router's own module --- API/Backend/Upload/validate.js | 9 ++ configure/src/core/upload.js | 8 +- scripts/lib/aws-provision.js | 39 +++--- src/pre/uploadKey.ts | 9 +- tests/unit/awsProvision.spec.js | 145 ++++++++++++++++------- tests/unit/infrastructure.spec.js | 14 --- tests/unit/publicHasNoHashedDirs.spec.js | 32 +++++ tests/unit/uploadKeyClassifier.spec.js | 14 ++- tests/unit/webpackHashedOutput.spec.js | 16 ++- 9 files changed, 191 insertions(+), 95 deletions(-) create mode 100644 tests/unit/publicHasNoHashedDirs.spec.js diff --git a/API/Backend/Upload/validate.js b/API/Backend/Upload/validate.js index 70b3b3afd..b67b72203 100644 --- a/API/Backend/Upload/validate.js +++ b/API/Backend/Upload/validate.js @@ -11,6 +11,14 @@ const IMAGE_MIME_TO_EXT = { 'image/svg+xml': 'svg', }; +// The object keys ./uploadRouter.js writes for plugin uploads when the S3 +// asset bucket is configured: "assets/", exactly two path segments, then +// "/uploads/". src/pre/uploadKey.ts and configure/src/core/upload.js each +// carry a copy — separate frontend bundles with no import path into this +// CommonJS module; tests/unit/uploadKeyClassifier.spec.js runs one table of +// values through all three and fails if they classify any of them differently. +const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; + // Map an upload mimetype to a safe file extension using the given allow-list, // or null if the type is not allowed. function extensionForMime(mimeType, allowedMimeToExt = IMAGE_MIME_TO_EXT) { @@ -38,6 +46,7 @@ const isValidSubdir = isSafePathSegment; module.exports = { IMAGE_MIME_TO_EXT, + ASSETS_UPLOAD_KEY, extensionForMime, isSafePathSegment, isValidMission, diff --git a/configure/src/core/upload.js b/configure/src/core/upload.js index 0f226a580..70fe5f8c2 100644 --- a/configure/src/core/upload.js +++ b/configure/src/core/upload.js @@ -34,10 +34,10 @@ export async function uploadImage(file, mission, subdir) { // The CMS is a separate bundle with no import path into the app's, so this // is a copy of ASSETS_UPLOAD_KEY in src/pre/uploadKey.ts, where the shape it -// matches is explained. scripts/lib/aws-provision.js carries a third copy, a -// CommonJS module run by Node. tests/unit/uploadKeyClassifier.spec.js runs one -// table of values through all three and fails if they classify any of them -// differently. +// matches is explained. API/Backend/Upload/validate.js holds the third, beside +// the router that writes the keys, in CommonJS for the publish scripts. +// tests/unit/uploadKeyClassifier.spec.js runs one table of values through all +// three and fails if they classify any of them differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // Turns a stored upload-field value into the URL the CMS's preview diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index e8dae88d5..f11581319 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -33,6 +33,10 @@ const { CreateInvalidationCommand, } = require("@aws-sdk/client-cloudfront"); +// The classifier for plugin-upload object keys, taken from the router that +// writes them. +const { ASSETS_UPLOAD_KEY } = require("../../API/Backend/Upload/validate"); + let _clients = null; function getClients() { @@ -431,12 +435,6 @@ function contentTypeForFile(filePath) { ); } -// The object keys API/Backend/Upload/uploadRouter.js writes for plugin -// uploads: "assets/", exactly two path segments, then "/uploads/". The same -// classifier lives in src/pre/uploadKey.ts and configure/src/core/upload.js; -// tests/unit/uploadKeyClassifier.spec.js runs one table through all three. -const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; - // Cache-Control tier for a published-dashboard object key. The entry page and // baked config must revalidate on every request (a fronting cache we cannot // invalidate may otherwise pin an old release for a day); two classes are @@ -446,17 +444,18 @@ const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // assets///uploads/, which the upload router names // crypto.randomUUID() and never overwrites (API/Backend/Upload/uploadRouter.js). // Everything else — the keys that really do change in place on republish — -// falls back to a short TTL. Two runtime families sit in that fallback under -// stable names, the pdf.js worker under public/workers and the Cesium tree -// under build/static/cesium, so a republish that bumps the MMGIS version can -// leave a customer's edge pairing a new bundle with a copy of those up to five -// minutes old. +// falls back to a short TTL. Every stable-named runtime asset outside those two +// immutable prefixes sits in that fallback: the pdf.js worker, the Cesium tree, +// the fonts, the ffmpeg core. scripts/build.js copyPublicFolder copies public/ +// into build/ verbatim, so nothing arriving that way is content-hashed either +// (tests/unit/publicHasNoHashedDirs.spec.js). A republish that bumps the MMGIS +// version can therefore leave a customer's edge pairing a new bundle with a copy +// of those roughly five minutes old, up to about ten while our own invalidation +// propagates. // -// The two cacheable tiers say "public" because every response sits behind -// Basic auth, and RFC 7234 §3.2 lets a shared cache reuse a response to an -// Authorization-bearing request only when it carries must-revalidate, public -// or s-maxage, so "public" removes any conformance question. What a customer -// fronting the dashboard gets from that is spelled out in +// Neither cacheable tier says "public": CloudFront caches on max-age alone, and +// omitting it keeps a conforming shared cache from storing these password-gated +// responses at all (RFC 9111 §3.5) — see // docs/infrastructure/serving-a-dashboard-from-your-domain.md. function cacheControlForKey(key) { if ( @@ -469,8 +468,8 @@ function cacheControlForKey(key) { /^build\/static\/(js|css|media)\//.test(key) || ASSETS_UPLOAD_KEY.test(key) ) - return "public, max-age=31536000, immutable"; - return "public, max-age=300"; + return "max-age=31536000, immutable"; + return "max-age=300"; } function walkDirectory(dir, baseDir) { @@ -577,8 +576,8 @@ function buildCopySource(bucket, key) { // extension is one it names, otherwise the source object's own header read // with HeadObject, and octet-stream when the source carries none either. async function copiedContentType({ s3, sourceBucket, key }) { - if (CONTENT_TYPES[path.extname(key).toLowerCase()] != null) - return contentTypeForFile(key); + const mapped = CONTENT_TYPES[path.extname(key).toLowerCase()]; + if (mapped != null) return mapped; const head = await s3.send( new HeadObjectCommand({ Bucket: sourceBucket, Key: key }) ); diff --git a/src/pre/uploadKey.ts b/src/pre/uploadKey.ts index 7c2a3b76f..a9a2741f3 100644 --- a/src/pre/uploadKey.ts +++ b/src/pre/uploadKey.ts @@ -16,10 +16,11 @@ // "assets/uploads/x.png", and a looser test ("starts with assets/") would // grab those too and resolve them against the wrong root. // -// configure/src/core/upload.js and scripts/lib/aws-provision.js carry their -// own copies of this regex — the CMS is a separate bundle and the publish -// scripts are CommonJS run by Node, neither with an import path into this one. -// tests/unit/uploadKeyClassifier.spec.js runs one table of values through +// This is a copy of the regex in API/Backend/Upload/validate.js, the CommonJS +// home beside the router that writes the keys and the copy the publish scripts +// require. configure/src/core/upload.js carries the third, for the CMS bundle. +// Neither frontend bundle has an import path into a CommonJS module run by +// Node. tests/unit/uploadKeyClassifier.spec.js runs one table of values through // all three and fails if they classify any of them differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\// diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 62bca2d56..1a53ce8cf 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -990,23 +990,20 @@ test.describe('cacheControlForKey', () => { ['index.html', 'no-cache'], ['build/index.html', 'no-cache'], ['Missions/M/config.json', 'no-cache'], - [ - 'build/static/js/main.abc123.js', - 'public, max-age=31536000, immutable', - ], - ['build/static/css/x.css', 'public, max-age=31536000, immutable'], - ['build/static/media/a.png', 'public, max-age=31536000, immutable'], - ['build/asset-manifest.json', 'public, max-age=300'], + ['build/static/js/main.abc123.js', 'max-age=31536000, immutable'], + ['build/static/css/x.css', 'max-age=31536000, immutable'], + ['build/static/media/a.png', 'max-age=31536000, immutable'], + ['build/asset-manifest.json', 'max-age=300'], // Under build/static but not content-hashed, so explicitly NOT // immutable. - ['build/static/cesium/Cesium.js', 'public, max-age=300'], - ['public/workers/pdf.worker.min.mjs', 'public, max-age=300'], + ['build/static/cesium/Cesium.js', 'max-age=300'], + ['public/workers/pdf.worker.min.mjs', 'max-age=300'], ] - TIERS.forEach(([key, expected]) => { - test(`'${key}' -> '${expected}'`, () => { - expect(provision.cacheControlForKey(key)).toBe(expected) - }) + // No tier says "public": the responses are password-gated, and CloudFront + // caches on max-age alone. + test.each(TIERS)("'%s' -> '%s'", (key, expected) => { + expect(provision.cacheControlForKey(key)).toBe(expected) }) }) @@ -1058,7 +1055,7 @@ test.describe('uploadDirectory', () => { // path 'static/js/main.abc123.js' falls through to max-age=300 — // so this fails if the tier is read off anything but the key. expect(byKey['build/static/js/main.abc123.js'].CacheControl).toBe( - 'public, max-age=31536000, immutable' + 'max-age=31536000, immutable' ) expect(byKey['build/index.html'].CacheControl).toBe('no-cache') }) @@ -1105,6 +1102,26 @@ test.describe('uploadFile', () => { test.describe('copyPrefix', () => { // The upload router names every file crypto.randomUUID() + the extension. const UPLOAD_UUID = '6f1e2a3c-4b5d-4e6f-8a9b-0c1d2e3f4a5b' + const UPLOAD_KEY = `assets/TestMission/CardPlugin/uploads/${UPLOAD_UUID}.png` + + // What each mocked source object carries as its own Content-Type, for the + // extensions the table does not name. An empty head is a source with none. + const SOURCE_HEADS = { + 'assets/TestMission/scan.tif': { ContentType: 'image/tiff' }, + 'assets/TestMission/untyped.bin': {}, + } + + const SOURCE_KEYS = [ + // The shape the upload router writes. + UPLOAD_KEY, + 'assets/TestMission/icon.png', + 'assets/TestMission/with space.png', + 'assets/TestMission/photo.jpg', + ...Object.keys(SOURCE_HEADS), + ] + + const byKey = (copies) => + Object.fromEntries(copies.map((input) => [input.Key, input])) test.afterEach(() => provision.setClients(null)) @@ -1117,23 +1134,13 @@ test.describe('copyPrefix', () => { if (name === 'ListObjectsV2Command') { expect(command.input.Prefix).toBe('assets/TestMission/') return { - Contents: [ - // The shape the upload router writes. - { - Key: `assets/TestMission/CardPlugin/uploads/${UPLOAD_UUID}.png`, - }, - { Key: 'assets/TestMission/icon.png' }, - { Key: 'assets/TestMission/with space.png' }, - { Key: 'assets/TestMission/photo.jpg' }, - // An extension the Content-Type table does not name. - { Key: 'assets/TestMission/scan.tif' }, - ], + Contents: SOURCE_KEYS.map((Key) => ({ Key })), IsTruncated: false, } } if (name === 'HeadObjectCommand') { heads.push(command.input.Key) - return { ContentType: 'image/tiff' } + return SOURCE_HEADS[command.input.Key] } if (name === 'CopyObjectCommand') { copies.push(command.input) @@ -1147,34 +1154,90 @@ test.describe('copyPrefix', () => { destBucket: 'dash', prefix: 'assets/TestMission/', }) - expect(count).toBe(5) + expect(count).toBe(SOURCE_KEYS.length) + const copied = byKey(copies) // Same keys in the destination bucket - expect(copies[1].Bucket).toBe('dash') - expect(copies[1].Key).toBe('assets/TestMission/icon.png') + expect(Object.keys(copied).sort()).toEqual([...SOURCE_KEYS].sort()) + const icon = copied['assets/TestMission/icon.png'] + expect(icon.Bucket).toBe('dash') // CopySource is "bucket/key" with the separators left intact — // NOT encodeURIComponent of the whole string (that would turn the // slashes into %2F and break the copy). - expect(copies[1].CopySource).toBe('shared/assets/TestMission/icon.png') + expect(icon.CopySource).toBe('shared/assets/TestMission/icon.png') // Special chars inside a segment are encoded; the "/" separators // and the bucket/key boundary are preserved. - expect(copies[2].Key).toBe('assets/TestMission/with space.png') - expect(copies[2].CopySource).toBe( + expect(copied['assets/TestMission/with space.png'].CopySource).toBe( 'shared/assets/TestMission/with%20space.png' ) // REPLACE lets the copy carry its own Cache-Control and Content-Type. - expect(copies[0].MetadataDirective).toBe('REPLACE') + expect(copied[UPLOAD_KEY].MetadataDirective).toBe('REPLACE') // An upload key gets the immutable tier, everything else the short one. - expect(copies[0].CacheControl).toBe( - 'public, max-age=31536000, immutable' + expect(copied[UPLOAD_KEY].CacheControl).toBe( + 'max-age=31536000, immutable' ) - expect(copies[1].CacheControl).toBe('public, max-age=300') + expect(icon.CacheControl).toBe('max-age=300') // A mapped extension is typed from the key alone... - expect(copies[1].ContentType).toBe('image/png') - expect(copies[3].ContentType).toBe('image/jpeg') + expect(icon.ContentType).toBe('image/png') + expect(copied['assets/TestMission/photo.jpg'].ContentType).toBe( + 'image/jpeg' + ) // ...and only an unmapped one costs a HeadObject, which is what keeps - // the source's own type instead of downgrading it to octet-stream. - expect(heads).toEqual(['assets/TestMission/scan.tif']) - expect(copies[4].ContentType).toBe('image/tiff') + // the source's own type instead of downgrading it to octet-stream... + expect(heads).toEqual(Object.keys(SOURCE_HEADS)) + expect(copied['assets/TestMission/scan.tif'].ContentType).toBe( + 'image/tiff' + ) + // ...which is where a source with no Content-Type of its own lands. + expect(copied['assets/TestMission/untyped.bin'].ContentType).toBe( + 'application/octet-stream' + ) + }) + + test('copies the pages after the first the same way', async () => { + // A prefix holding more than a page of objects lists truncated, and + // every later page has to be copied under the same rules — a loop that + // stopped after page one would leave those objects behind in the + // shared bucket, missing from the published dashboard. + const copies = [] + const tokens = [] + provision.setClients({ + s3: mockClient((command) => { + const name = command.constructor.name + if (name === 'ListObjectsV2Command') { + tokens.push(command.input.ContinuationToken) + if (command.input.ContinuationToken == null) + return { + Contents: [{ Key: 'assets/TestMission/one.png' }], + IsTruncated: true, + NextContinuationToken: 'page-two', + } + return { + Contents: [{ Key: 'assets/TestMission/two.png' }], + // A last page can still name a token; IsTruncated is + // what ends the loop. + IsTruncated: false, + NextContinuationToken: 'page-three', + } + } + if (name === 'CopyObjectCommand') { + copies.push(command.input) + return {} + } + throw new Error(`Unexpected command ${name}`) + }), + }) + const count = await provision.copyPrefix({ + sourceBucket: 'shared', + destBucket: 'dash', + prefix: 'assets/TestMission/', + }) + expect(count).toBe(2) + expect(tokens).toEqual([undefined, 'page-two']) + const pageTwo = byKey(copies)['assets/TestMission/two.png'] + expect(pageTwo.Bucket).toBe('dash') + expect(pageTwo.MetadataDirective).toBe('REPLACE') + expect(pageTwo.CacheControl).toBe('max-age=300') + expect(pageTwo.ContentType).toBe('image/png') }) }) diff --git a/tests/unit/infrastructure.spec.js b/tests/unit/infrastructure.spec.js index 9ab625c73..15569f622 100644 --- a/tests/unit/infrastructure.spec.js +++ b/tests/unit/infrastructure.spec.js @@ -351,20 +351,6 @@ test.describe('infrastructure/ recipes (JSON and Terraform)', () => { expect(adminEnv.DISABLE_FIRST_SIGNUP).toBe('true') }) - test('the publish skips both un-interpolated index templates', () => { - // build/index.pug and public/index.html still carry the raw #{...} - // placeholders, so either one in the bucket is a page a visitor can - // reach with the placeholders showing. The upload filters name the two - // keys literally; pinning them here catches a rename on one side that - // never reaches the other. - const source = fs.readFileSync( - path.join(ROOT, 'scripts', 'publish-static.js'), - 'utf8' - ) - expect(source).toContain('key !== "build/index.pug"') - expect(source).toContain('key !== "public/index.html"') - }) - test('publish task role omits rds-db:connect (password auth only)', () => { for (const file of IAM_FILES) { const actions = statementsOf(readJson(file)).flatMap((s) => diff --git a/tests/unit/publicHasNoHashedDirs.spec.js b/tests/unit/publicHasNoHashedDirs.spec.js new file mode 100644 index 000000000..027e78b90 --- /dev/null +++ b/tests/unit/publicHasNoHashedDirs.spec.js @@ -0,0 +1,32 @@ +import { test, expect } from 'vitest' +import fs from 'fs' +import path from 'path' + +const { cacheControlForKey } = require('../../scripts/lib/aws-provision') + +// scripts/build.js copyPublicFolder copies public/ into build/ verbatim, under +// the names the files already have. Nothing in there is content-hashed, so a +// static/(js|css|media) subtree in public/ would put a stable name into the +// immutable Cache-Control tier of scripts/lib/aws-provision.js, where a +// customer's CloudFront edge pins it for a year and a republish can never +// dislodge it. + +const PUBLIC = path.join(__dirname, '..', '..', 'public') + +function keysUnder(dir, baseDir = dir) { + return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { + const full = path.join(dir, entry.name) + if (entry.isDirectory()) return keysUnder(full, baseDir) + if (!entry.isFile()) return [] + return [path.relative(baseDir, full).split(path.sep).join('/')] + }) +} + +test('no file copied out of public/ lands in the immutable tier', () => { + const keys = keysUnder(PUBLIC) + expect(keys.length).toBeGreaterThan(0) + const immutable = keys.filter((key) => + cacheControlForKey(`build/${key}`).includes('immutable'), + ) + expect(immutable).toEqual([]) +}) diff --git a/tests/unit/uploadKeyClassifier.spec.js b/tests/unit/uploadKeyClassifier.spec.js index 6ae367365..a3a5d9103 100644 --- a/tests/unit/uploadKeyClassifier.spec.js +++ b/tests/unit/uploadKeyClassifier.spec.js @@ -5,10 +5,12 @@ import { buildPreviewSrc } from '../../configure/src/core/upload.js' const { cacheControlForKey } = require('../../scripts/lib/aws-provision') // One table of stored values, run through every copy of the upload-key -// classifier: the app bundle, the Configure SPA and the publish scripts. None -// of the three can share a module with the others, so each carries its own -// regex; what they must agree on is which values are upload keys written by -// API/Backend/Upload/uploadRouter.js, not the bytes of the regex. A value one +// classifier: the app bundle, the Configure SPA and API/Backend/Upload/ +// validate.js — the CommonJS copy beside the router that writes the keys, and +// the one the publish scripts require. Neither frontend bundle can import that +// module, so each carries its own regex; what they must agree on is which +// values are upload keys written by API/Backend/Upload/uploadRouter.js, not the +// bytes of the regex. A value one // treats as an upload key and another as a mission-relative path renders a // broken image only at runtime. What each consumer then does with a matched // key differs by design and is asserted per classifier below. @@ -57,8 +59,8 @@ test.describe('upload-key classification', () => { (value, key) => { expect(cacheControlForKey(value)).toBe( key !== null - ? 'public, max-age=31536000, immutable' - : 'public, max-age=300', + ? 'max-age=31536000, immutable' + : 'max-age=300', ) }, ) diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index eb5fc6f22..ed0504555 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -71,16 +71,20 @@ test.describe('webpack production output is content-hashed', () => { expect(options.chunkFilename).toMatch(/^static\/css\//) }) - test('every rule names its emitted files hashed under static/media', () => { - // The url-loader (small images, inlined below a size limit) and the - // catch-all file-loader both emit into static/media; every prefix in - // the immutable tier has to be hashed, so a rule emitting elsewhere - // under a hashless name is caught here too. + test('every rule names its emitted files with a content hash', () => { + // Every rule has to hash, wherever it emits: a hashless name lands in + // the immutable tier if it goes to one of those three prefixes, and + // collides with the previous release's file if it does not. The + // url-loader (small images, inlined below a size limit) and the + // catch-all file-loader are the rules that reach an immutable prefix, + // and static/media is the only one they may use — js and css belong to + // the compiler's own output. const names = emittedNames(CONFIG.module.rules) expect(names.length).toBeGreaterThan(0) names.forEach((name) => { - expect(name).toMatch(/^static\/media\//) expect(name).toMatch(HASH_TOKEN) + if (/^static\/(js|css|media)\//.test(name)) + expect(name).toMatch(/^static\/media\//) }) }) From 5f57f7f52f690a05edecdf4a9684ca7057f4286c Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:15:32 -0500 Subject: [PATCH 09/13] [316] Tell the customer nothing is marked for shared caches --- .../serving-a-dashboard-from-your-domain.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index e6a7a5ad8..3df6a1a28 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -13,7 +13,7 @@ The examples below use the path `/tools/dashboard`; substitute your own everywhe - all query strings in the cache key, - Minimum TTL 0, - Maximum TTL 31536000 (one year) or more, - - Default TTL — any value; every response carries its own `Cache-Control`. + - Default TTL — any value. 3. **Add two cache behaviors** pointing at that origin, both using that cache policy: - path pattern `/tools/dashboard` — exact, no wildcard, @@ -47,14 +47,18 @@ Every dashboard is password-protected, so every row also sits behind the passwor **The header** tells us how much of the forwarded path is yours. CloudFront forwards the full path exactly as the visitor typed it, so our side receives `/tools/dashboard/index.html` and needs to know that `/tools/dashboard` is prefix, not content. We remove exactly what the header declares and serve the file. If the header is missing or doesn't match the path, every request under the path fails with a 403 immediately — a loud failure on purpose, instead of quietly serving the wrong files. (It's a 403 rather than a 404 because the rejection comes from our storage layer, which answers "access denied.") -**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Our cacheable responses are marked `public`, which is what lets a shared cache hold a response to a request that carried an `Authorization` header at all (RFC 7234 §3.2) — so your CloudFront keeps them under that per-credential key, and any other shared cache between you and the visitor may hold the long-lived tiers too: the bundles and uploaded images, and the supporting files for five minutes. Every visitor needs that password — there is no unauthenticated mode today — and the same password opens every dashboard published from the same MMGIS environment, so give it only to an audience you would hand every one of those dashboards to. Never publish it on a page. +**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Every visitor needs that password — there is no unauthenticated mode today — and the same password opens every dashboard published from the same MMGIS environment, so give it only to an audience you would hand every one of those dashboards to. Never publish it on a page. + +**Nothing is marked for shared caches.** Our responses carry no `public` in their `Cache-Control`. Your CloudFront caches them anyway, on `max-age` alone, under the per-credential key you just configured — that is how CloudFront behaves. An intermediary proxy that follows the standard to the letter will not store them at all, because the requests carry an `Authorization` header (RFC 9111 §3.5). Password-gated files stay out of caches we know nothing about. **All query strings in the cache key:** a policy that drops query strings never sends them to us. A deep link like `/tools/dashboard?view=2` then reaches us stripped of its `?view=2`, and the address we redirect the visitor to has lost it for good. -**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files within about five minutes of the publish completing. +**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files roughly five minutes after the publish completes, up to about ten while our own invalidation propagates. **Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. +**Default TTL:** every file we serve carries its own `Cache-Control`, so the policy default never decides how long one is held. + **HTTPS only to the origin:** the dashboard's password rides on the `Authorization` header of every request you forward. Over plain HTTP it would cross the internet unencrypted. **No viewer `Host` header:** our distribution answers only to its own `*.cloudfront.net` name; a request carrying your hostname is rejected by AWS with a 403 before anything of ours runs. CloudFront omits the viewer's `Host` by default — the hazard is specifically the managed `AllViewer` origin request policy, which forwards it. `AllViewerExceptHostHeader` forwards everything else while excluding it. From 20424973dd907ae4803279f956668666b98191ad Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:35:33 -0500 Subject: [PATCH 10/13] [316] Make the router's module the documented home of the upload-key shape and tighten the tier specs --- API/Backend/Upload/validate.js | 20 ++++++++++---- configure/src/core/upload.js | 8 +++--- scripts/lib/aws-provision.js | 20 ++++++-------- scripts/publish-static.js | 2 ++ src/pre/uploadKey.ts | 22 ++++----------- tests/unit/awsProvision.spec.js | 32 +++++++++++++++++----- tests/unit/cfnTemplate.spec.js | 9 +++--- tests/unit/uploadKeyClassifier.spec.js | 38 ++++++++++---------------- tests/unit/uploadRouterS3.spec.js | 5 ++++ tests/unit/webpackHashedOutput.spec.js | 16 +++++++++-- 10 files changed, 99 insertions(+), 73 deletions(-) diff --git a/API/Backend/Upload/validate.js b/API/Backend/Upload/validate.js index b67b72203..f1902e947 100644 --- a/API/Backend/Upload/validate.js +++ b/API/Backend/Upload/validate.js @@ -12,11 +12,21 @@ const IMAGE_MIME_TO_EXT = { }; // The object keys ./uploadRouter.js writes for plugin uploads when the S3 -// asset bucket is configured: "assets/", exactly two path segments, then -// "/uploads/". src/pre/uploadKey.ts and configure/src/core/upload.js each -// carry a copy — separate frontend bundles with no import path into this -// CommonJS module; tests/unit/uploadKeyClassifier.spec.js runs one table of -// values through all three and fails if they classify any of them differently. +// asset bucket is configured. Those look exactly like +// +// assets///uploads/ +// +// so this matches "assets/", then exactly two path segments, then "/uploads/". +// The exactness matters: a plugin whose subdir is itself named "assets" stores +// ordinary mission-relative values like "assets/uploads/x.png", and a looser +// test ("starts with assets/") would grab those too and resolve them against +// the wrong root. +// +// This is the documented home of that shape. src/pre/uploadKey.ts and +// configure/src/core/upload.js each carry a copy — separate frontend bundles +// with no import path into this CommonJS module; +// tests/unit/uploadKeyClassifier.spec.js runs one table of values through all +// three and fails if they classify any of them differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // Map an upload mimetype to a safe file extension using the given allow-list, diff --git a/configure/src/core/upload.js b/configure/src/core/upload.js index 70fe5f8c2..dcdc4513d 100644 --- a/configure/src/core/upload.js +++ b/configure/src/core/upload.js @@ -32,10 +32,10 @@ export async function uploadImage(file, mission, subdir) { return data.path; } -// The CMS is a separate bundle with no import path into the app's, so this -// is a copy of ASSETS_UPLOAD_KEY in src/pre/uploadKey.ts, where the shape it -// matches is explained. API/Backend/Upload/validate.js holds the third, beside -// the router that writes the keys, in CommonJS for the publish scripts. +// The CMS is a separate bundle with no import path into a CommonJS module run +// by Node, so this is a copy of ASSETS_UPLOAD_KEY in +// API/Backend/Upload/validate.js; see it for the shape this matches and why. +// src/pre/uploadKey.ts carries a third copy for the app bundle. // tests/unit/uploadKeyClassifier.spec.js runs one table of values through all // three and fails if they classify any of them differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index 95d9d06db..a1d479acc 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -464,19 +464,17 @@ function contentTypeForFile(filePath) { // tests/unit/webpackHashedOutput.spec.js), and plugin uploads under // assets///uploads/, which the upload router names // crypto.randomUUID() and never overwrites (API/Backend/Upload/uploadRouter.js). -// Everything else — the keys that really do change in place on republish — -// falls back to a short TTL. Every stable-named runtime asset outside those two -// immutable prefixes sits in that fallback: the pdf.js worker, the Cesium tree, -// the fonts, the ffmpeg core. scripts/build.js copyPublicFolder copies public/ -// into build/ verbatim, so nothing arriving that way is content-hashed either -// (tests/unit/publicHasNoHashedDirs.spec.js). A republish that bumps the MMGIS -// version can therefore leave a customer's edge pairing a new bundle with a copy -// of those roughly five minutes old, up to about ten while our own invalidation -// propagates. +// Everything else — the keys that really do change in place on republish, which +// is every stable-named runtime asset outside those two immutable prefixes — +// falls back to a short TTL. scripts/build.js copyPublicFolder copies public/ +// and dist/ into build/ verbatim, so nothing arriving that way is +// content-hashed either (tests/unit/publicHasNoHashedDirs.spec.js). // // Neither cacheable tier says "public": CloudFront caches on max-age alone, and -// omitting it keeps a conforming shared cache from storing these password-gated -// responses at all (RFC 9111 §3.5) — see +// omitting it keeps a conforming shared cache from ever serving these +// password-gated responses to another request (RFC 9111 §3.5; reuse would take +// public, must-revalidate or s-maxage, none of which we send). What the tiers +// mean for a customer fronting the dashboard is in // docs/infrastructure/serving-a-dashboard-from-your-domain.md. function cacheControlForKey(key) { if ( diff --git a/scripts/publish-static.js b/scripts/publish-static.js index a5e585918..3e9d906c3 100644 --- a/scripts/publish-static.js +++ b/scripts/publish-static.js @@ -293,6 +293,8 @@ async function main() { // writes into build/, and public/index.html is the HTML template the // build renders from. Either one served as-is would hand a visitor the // placeholders un-interpolated. The rendered page is uploaded below. + // A publish only writes: it deletes nothing already in the bucket, so a + // bucket that picked up either key before keeps it until emptied. const uploadedBuild = await provision.uploadDirectory({ bucket, dir: path.join(rootDir, "build"), diff --git a/src/pre/uploadKey.ts b/src/pre/uploadKey.ts index a9a2741f3..8b82a616b 100644 --- a/src/pre/uploadKey.ts +++ b/src/pre/uploadKey.ts @@ -6,22 +6,12 @@ */ // Is this stored value one of the keys API/Backend/Upload/uploadRouter.js -// writes when the S3 asset bucket is configured? Those look exactly like -// -// assets///uploads/ -// -// so this matches "assets/", then exactly two path segments, then -// "/uploads/". The exactness matters: a plugin whose subdir is itself named -// "assets" stores ordinary mission-relative values like -// "assets/uploads/x.png", and a looser test ("starts with assets/") would -// grab those too and resolve them against the wrong root. -// -// This is a copy of the regex in API/Backend/Upload/validate.js, the CommonJS -// home beside the router that writes the keys and the copy the publish scripts -// require. configure/src/core/upload.js carries the third, for the CMS bundle. -// Neither frontend bundle has an import path into a CommonJS module run by -// Node. tests/unit/uploadKeyClassifier.spec.js runs one table of values through -// all three and fails if they classify any of them differently. +// writes when the S3 asset bucket is configured? This is a copy of +// ASSETS_UPLOAD_KEY in API/Backend/Upload/validate.js; see it for the shape +// this matches and why. The Essence bundle has no import path into a CommonJS +// module run by Node, and configure/src/core/upload.js carries a third copy +// for the CMS bundle. tests/unit/uploadKeyClassifier.spec.js runs one table of +// values through all three and fails if they classify any of them differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\// // What URL should the page request for a stored value? Four cases, checked diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index 1d6c6a1ef..a8e86b7af 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -991,9 +991,12 @@ test.describe('emptyBucket', () => { }) test.describe('contentTypeForFile', () => { - // CopyObject's MetadataDirective: REPLACE drops the source's Content-Type - // and takes this one, so every type the upload router can write has to - // round-trip back to itself through the extension it was stored under. + // CopyObject's MetadataDirective: REPLACE drops the source's Content-Type, + // so copyPrefix restates one. An extension this mapping knows answers from + // the key alone and saves a HeadObject per object, so every type the upload + // router can write has to round-trip back to itself through the extension + // it was stored under. An unmapped type still copies correctly — it just + // costs the HeadObject that reads the source's own Content-Type. test.each(Object.entries(IMAGE_MIME_TO_EXT))( "round-trips the upload router's %s", (mime, ext) => { @@ -1030,7 +1033,10 @@ test.describe('cacheControlForKey', () => { ['build/asset-manifest.json', 'max-age=300'], // Under build/static but not content-hashed, so explicitly NOT // immutable. - ['build/static/cesium/Cesium.js', 'max-age=300'], + [ + 'build/static/cesium/Workers/cesiumWorkerBootstrapper.js', + 'max-age=300', + ], ['public/workers/pdf.worker.min.mjs', 'max-age=300'], ] @@ -1159,7 +1165,10 @@ test.describe('copyPrefix', () => { test.afterEach(() => provision.setClients(null)) - test('same-key copies every object under the prefix', async () => { + // One copyPrefix run over SOURCE_KEYS against a mocked source bucket: + // returns how many objects it copied, the copies by key, and the keys that + // cost a HeadObject. + async function runCopyPrefix() { const copies = [] const heads = [] provision.setClients({ @@ -1188,8 +1197,12 @@ test.describe('copyPrefix', () => { destBucket: 'dash', prefix: 'assets/TestMission/', }) + return { count, copied: byKey(copies), heads } + } + + test('same-key copies every object under the prefix', async () => { + const { count, copied } = await runCopyPrefix() expect(count).toBe(SOURCE_KEYS.length) - const copied = byKey(copies) // Same keys in the destination bucket expect(Object.keys(copied).sort()).toEqual([...SOURCE_KEYS].sort()) const icon = copied['assets/TestMission/icon.png'] @@ -1203,6 +1216,11 @@ test.describe('copyPrefix', () => { expect(copied['assets/TestMission/with space.png'].CopySource).toBe( 'shared/assets/TestMission/with%20space.png' ) + }) + + test('every copy carries its own Cache-Control and Content-Type', async () => { + const { copied, heads } = await runCopyPrefix() + const icon = copied['assets/TestMission/icon.png'] // REPLACE lets the copy carry its own Cache-Control and Content-Type. expect(copied[UPLOAD_KEY].MetadataDirective).toBe('REPLACE') // An upload key gets the immutable tier, everything else the short one. @@ -1217,7 +1235,7 @@ test.describe('copyPrefix', () => { ) // ...and only an unmapped one costs a HeadObject, which is what keeps // the source's own type instead of downgrading it to octet-stream... - expect(heads).toEqual(Object.keys(SOURCE_HEADS)) + expect([...heads].sort()).toEqual(Object.keys(SOURCE_HEADS).sort()) expect(copied['assets/TestMission/scan.tif'].ContentType).toBe( 'image/tiff' ) diff --git a/tests/unit/cfnTemplate.spec.js b/tests/unit/cfnTemplate.spec.js index 8a197f703..5d4eba32c 100644 --- a/tests/unit/cfnTemplate.spec.js +++ b/tests/unit/cfnTemplate.spec.js @@ -136,11 +136,10 @@ test.describe('renderCfnTemplate', () => { template.Resources.DashboardDistribution.Properties .DistributionConfig expect(dist.DefaultRootObject).toBe('index.html') - // The managed CachingOptimized policy, whose maximum TTL of a year is - // what lets the immutable Cache-Control tier survive this edge. - expect(dist.DefaultCacheBehavior.CachePolicyId).toBe( - '658327ea-f89d-4fab-a63d-7e88639e58f6' - ) + expect( + dist.DefaultCacheBehavior.CachePolicyId, + 'must be a policy whose maximum TTL is a year or more, or this edge caps the immutable Cache-Control tier (this id is the managed CachingOptimized)' + ).toBe('658327ea-f89d-4fab-a63d-7e88639e58f6') const associations = dist.DefaultCacheBehavior.FunctionAssociations expect(associations).toHaveLength(1) expect(associations[0].EventType).toBe('viewer-request') diff --git a/tests/unit/uploadKeyClassifier.spec.js b/tests/unit/uploadKeyClassifier.spec.js index a3a5d9103..4ac3d3b99 100644 --- a/tests/unit/uploadKeyClassifier.spec.js +++ b/tests/unit/uploadKeyClassifier.spec.js @@ -2,18 +2,17 @@ import { test, expect } from 'vitest' import { resolveMissionAssetUrl } from '../../src/pre/uploadKey.ts' import { buildPreviewSrc } from '../../configure/src/core/upload.js' -const { cacheControlForKey } = require('../../scripts/lib/aws-provision') +const { ASSETS_UPLOAD_KEY } = require('../../API/Backend/Upload/validate') // One table of stored values, run through every copy of the upload-key -// classifier: the app bundle, the Configure SPA and API/Backend/Upload/ -// validate.js — the CommonJS copy beside the router that writes the keys, and -// the one the publish scripts require. Neither frontend bundle can import that -// module, so each carries its own regex; what they must agree on is which -// values are upload keys written by API/Backend/Upload/uploadRouter.js, not the -// bytes of the regex. A value one +// classifier: API/Backend/Upload/validate.js — the CommonJS home beside the +// router that writes the keys — the app bundle and the Configure SPA. Neither +// frontend bundle can import that module, so each carries its own regex; what +// they must agree on is which values are upload keys written by +// API/Backend/Upload/uploadRouter.js, not the bytes of the regex. A value one // treats as an upload key and another as a mission-relative path renders a -// broken image only at runtime. What each consumer then does with a matched -// key differs by design and is asserted per classifier below. +// broken image only at runtime. What each consumer then does with a matched key +// differs by design and is asserted per classifier below. const MISSION = 'M' const MISSION_PATH = 'Missions/M/' @@ -50,18 +49,11 @@ test.describe('upload-key classification', () => { ) }) - // The publish gives a matched key the immutable tier, because the upload - // router names those files crypto.randomUUID() and never overwrites one. - // The rooted row sits out: S3 object keys have no leading slash, so it is - // not a value this classifier is ever handed. - test.each(VALUES.filter(([value]) => !value.startsWith('/')))( - 'cacheControlForKey: %s', - (value, key) => { - expect(cacheControlForKey(value)).toBe( - key !== null - ? 'max-age=31536000, immutable' - : 'max-age=300', - ) - }, - ) + // The regex itself, which every consumer applies to the value with any + // leading slash already stripped. + test.each(VALUES)('ASSETS_UPLOAD_KEY: %s', (value, key) => { + expect(ASSETS_UPLOAD_KEY.test(value.replace(/^\//, ''))).toBe( + key !== null, + ) + }) }) diff --git a/tests/unit/uploadRouterS3.spec.js b/tests/unit/uploadRouterS3.spec.js index 26f8244dc..8ec825731 100644 --- a/tests/unit/uploadRouterS3.spec.js +++ b/tests/unit/uploadRouterS3.spec.js @@ -25,6 +25,7 @@ import path from 'path' const ROUTER_PATH = '../../API/Backend/Upload/uploadRouter.js' const MODE_PATH = '../../API/Backend/Utils/deploymentMode.js' const MISSIONS_DIR = path.join(__dirname, '../../Missions') +const { ASSETS_UPLOAD_KEY } = require('../../API/Backend/Upload/validate') const ENV_KEYS = ['MMGIS_DEPLOYMENT_MODE', 'MMGIS_SHARED_ASSET_BUCKET'] @@ -151,6 +152,10 @@ test.describe('upload router lean-mode S3 storage', () => { `^assets/${mission}/CardPlugin/uploads/[0-9a-f-]{36}\\.png$` ) ) + // And the shape every consumer of the stored value classifies as + // an upload key: miss it and the dashboard resolves the path + // against the mission folder instead of the dashboard root. + expect(ASSETS_UPLOAD_KEY.test(body.path)).toBe(true) expect(s3.calls.length).toBe(1) const cmd = s3.calls[0] diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index ed0504555..143ed6a2a 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -28,13 +28,20 @@ const HASH_TOKEN = /\[contenthash|\[hash/ // Every emitted-file name pattern in the rules, however they are nested — the // production asset loaders sit inside a `oneOf`. Loaders name their output -// with `options.name`; webpack 5 asset modules use `generator.filename`. +// with `options.name`, whether the loader sits on the rule itself or in its +// `use` chain; webpack 5 asset modules use `generator.filename`. function emittedNames(rules, found = []) { const list = rules || [] list.forEach((rule) => { if (!rule) return emittedNames(rule.oneOf, found) emittedNames(rule.rules, found) + // `use` holds either one loader or a chain of them. + const uses = [].concat(rule.use || []) + uses.forEach((entry) => { + const used = entry && entry.options && entry.options.name + if (typeof used === 'string') found.push(used) + }) const name = rule.options && rule.options.name if (typeof name === 'string') found.push(name) const generated = rule.generator && rule.generator.filename @@ -56,11 +63,16 @@ function pluginNamed(name) { } test.describe('webpack production output is content-hashed', () => { - test('output.filename and output.chunkFilename land hashed under static/js', () => { + test('the compiler hashes the filenames it picks itself', () => { expect(CONFIG.output.filename).toMatch(HASH_TOKEN) expect(CONFIG.output.filename).toMatch(/^static\/js\//) expect(CONFIG.output.chunkFilename).toMatch(HASH_TOKEN) expect(CONFIG.output.chunkFilename).toMatch(/^static\/js\//) + // The name an asset module falls back to when no rule gives it one. + // The config leaves it to webpack's own hashed default; setting it to + // anything hashless is what this guards against. + if (CONFIG.output.assetModuleFilename) + expect(CONFIG.output.assetModuleFilename).toMatch(HASH_TOKEN) }) test("MiniCssExtractPlugin's filenames land hashed under static/css", () => { From 6bd7df7982112bd0e51aab9b4629667c987c8998 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:35:33 -0500 Subject: [PATCH 11/13] [316] Correct the shared-cache and managed-policy claims and add the rotation note --- .../serving-a-dashboard-from-your-domain.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index 3df6a1a28..88c3a7c52 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -13,7 +13,7 @@ The examples below use the path `/tools/dashboard`; substitute your own everywhe - all query strings in the cache key, - Minimum TTL 0, - Maximum TTL 31536000 (one year) or more, - - Default TTL — any value. + - Default TTL — anything between the two (it never applies). 3. **Add two cache behaviors** pointing at that origin, both using that cache policy: - path pattern `/tools/dashboard` — exact, no wildcard, @@ -47,15 +47,15 @@ Every dashboard is password-protected, so every row also sits behind the passwor **The header** tells us how much of the forwarded path is yours. CloudFront forwards the full path exactly as the visitor typed it, so our side receives `/tools/dashboard/index.html` and needs to know that `/tools/dashboard` is prefix, not content. We remove exactly what the header declares and serve the file. If the header is missing or doesn't match the path, every request under the path fails with a 403 immediately — a loud failure on purpose, instead of quietly serving the wrong files. (It's a 403 rather than a 404 because the rejection comes from our storage layer, which answers "access denied.") -**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Every visitor needs that password — there is no unauthenticated mode today — and the same password opens every dashboard published from the same MMGIS environment, so give it only to an audience you would hand every one of those dashboards to. Never publish it on a page. +**`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Nothing we return is marked for shared caches: our responses carry no `public` in their `Cache-Control`. Your CloudFront caches them anyway, on `max-age` alone, under the per-credential key you just configured — that is how CloudFront behaves. An intermediary proxy that follows the standard to the letter will never serve them to another request, because they were fetched with an `Authorization` header (RFC 9111 §3.5); reuse would take `public`, `must-revalidate` or `s-maxage`, and we send none of the three. Password-gated files stay out of caches we know nothing about. Every visitor needs that password — there is no unauthenticated mode today — and the same password opens every dashboard published from the same MMGIS environment, so give it only to an audience you would hand every one of those dashboards to. Never publish it on a page. -**Nothing is marked for shared caches.** Our responses carry no `public` in their `Cache-Control`. Your CloudFront caches them anyway, on `max-age` alone, under the per-credential key you just configured — that is how CloudFront behaves. An intermediary proxy that follows the standard to the letter will not store them at all, because the requests carry an `Authorization` header (RFC 9111 §3.5). Password-gated files stay out of caches we know nothing about. +**After a password rotation, invalidate.** A new password does not evict what your edge is already holding under the old one: a cached object lives out its `max-age` whatever the credential now is. So when the dashboards password is rotated, run an invalidation on your own distribution — otherwise the retired password keeps opening cached pages until they age out, up to a year for the fingerprinted files. **All query strings in the cache key:** a policy that drops query strings never sends them to us. A deep link like `/tools/dashboard?view=2` then reaches us stripped of its `?view=2`, and the address we redirect the visitor to has lost it for good. -**Minimum TTL 0:** managed policies impose a minimum cache time that overrides what our responses ask for. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files roughly five minutes after the publish completes, up to about ten while our own invalidation propagates. +**Minimum TTL 0:** a policy's minimum cache time overrides what our responses ask for, and most of the managed ones sit above 0. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files roughly five minutes after the publish completes, up to about ten while our own invalidation propagates. -**Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. CloudFront has no single "obey the origin" switch: Minimum TTL 0 stops the policy raising our floor, and a Maximum TTL of at least a year stops it capping the immutable tier. +**Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. AWS's managed `UseOriginCacheControlHeaders` policies (and the `-QueryStrings` variant) already pair Minimum TTL 0 with a one-year maximum, but neither puts `Authorization` in the cache key — which is why the policy has to be a custom one, with both bounds set by hand: Minimum 0 so it raises no floor over what we ask for, Maximum a year so it does not cap the immutable tier. **Default TTL:** every file we serve carries its own `Cache-Control`, so the policy default never decides how long one is held. From 2018df1491babca6aa96acc7ddcca72081e75a23 Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:55:32 -0500 Subject: [PATCH 12/13] [316] Carry a hand-placed object's storage headers across the copy and pin the router's PutObject shape --- API/Backend/Upload/validate.js | 9 +++-- configure/src/core/upload.js | 9 ++--- scripts/lib/aws-provision.js | 56 ++++++++++++++++---------- scripts/publish-static.js | 3 +- src/pre/uploadKey.ts | 6 +-- tests/unit/awsProvision.spec.js | 17 +++++--- tests/unit/cfnTemplate.spec.js | 2 +- tests/unit/uploadRouterS3.spec.js | 16 +++++--- tests/unit/webpackHashedOutput.spec.js | 5 --- 9 files changed, 69 insertions(+), 54 deletions(-) diff --git a/API/Backend/Upload/validate.js b/API/Backend/Upload/validate.js index f1902e947..7639ce3fd 100644 --- a/API/Backend/Upload/validate.js +++ b/API/Backend/Upload/validate.js @@ -23,10 +23,11 @@ const IMAGE_MIME_TO_EXT = { // the wrong root. // // This is the documented home of that shape. src/pre/uploadKey.ts and -// configure/src/core/upload.js each carry a copy — separate frontend bundles -// with no import path into this CommonJS module; -// tests/unit/uploadKeyClassifier.spec.js runs one table of values through all -// three and fails if they classify any of them differently. +// configure/src/core/upload.js each carry a copy because both frontend bundles +// restrict imports to their own src/ (webpack's ModuleScopePlugin), which puts +// this file out of reach of either. tests/unit/uploadKeyClassifier.spec.js runs +// one table of values through all three and fails if they classify any of them +// differently. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // Map an upload mimetype to a safe file extension using the given allow-list, diff --git a/configure/src/core/upload.js b/configure/src/core/upload.js index dcdc4513d..b42352991 100644 --- a/configure/src/core/upload.js +++ b/configure/src/core/upload.js @@ -32,12 +32,9 @@ export async function uploadImage(file, mission, subdir) { return data.path; } -// The CMS is a separate bundle with no import path into a CommonJS module run -// by Node, so this is a copy of ASSETS_UPLOAD_KEY in -// API/Backend/Upload/validate.js; see it for the shape this matches and why. -// src/pre/uploadKey.ts carries a third copy for the app bundle. -// tests/unit/uploadKeyClassifier.spec.js runs one table of values through all -// three and fails if they classify any of them differently. +// A copy of ASSETS_UPLOAD_KEY in API/Backend/Upload/validate.js; see it for +// the shape this matches, why, and why each bundle carries its own copy. +// src/pre/uploadKey.ts carries the third. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\//; // Turns a stored upload-field value into the URL the CMS's preview diff --git a/scripts/lib/aws-provision.js b/scripts/lib/aws-provision.js index a1d479acc..f0fd57230 100644 --- a/scripts/lib/aws-provision.js +++ b/scripts/lib/aws-provision.js @@ -464,17 +464,15 @@ function contentTypeForFile(filePath) { // tests/unit/webpackHashedOutput.spec.js), and plugin uploads under // assets///uploads/, which the upload router names // crypto.randomUUID() and never overwrites (API/Backend/Upload/uploadRouter.js). -// Everything else — the keys that really do change in place on republish, which -// is every stable-named runtime asset outside those two immutable prefixes — -// falls back to a short TTL. scripts/build.js copyPublicFolder copies public/ -// and dist/ into build/ verbatim, so nothing arriving that way is -// content-hashed either (tests/unit/publicHasNoHashedDirs.spec.js). +// Everything else falls back to a short TTL. scripts/build.js copyPublicFolder +// copies public/ into build/ verbatim, so nothing arriving that way is +// content-hashed either (tests/unit/publicHasNoHashedDirs.spec.js). The Cesium +// tree is the bulk of that, and the short tier is where it belongs: its +// filenames are stable and their contents change on a release bump. // -// Neither cacheable tier says "public": CloudFront caches on max-age alone, and -// omitting it keeps a conforming shared cache from ever serving these -// password-gated responses to another request (RFC 9111 §3.5; reuse would take -// public, must-revalidate or s-maxage, none of which we send). What the tiers -// mean for a customer fronting the dashboard is in +// Neither cacheable tier says "public", which keeps these password-gated +// responses out of shared caches. What the tiers mean for a customer fronting +// the dashboard is in // docs/infrastructure/serving-a-dashboard-from-your-domain.md. function cacheControlForKey(key) { if ( @@ -591,26 +589,40 @@ function buildCopySource(bucket, key) { return `${bucket}/${encodedKey}`; } -// Content-Type for a copied object: the CONTENT_TYPES mapping when the -// extension is one it names, otherwise the source object's own header read -// with HeadObject, and octet-stream when the source carries none either. -async function copiedContentType({ s3, sourceBucket, key }) { +// The headers a copied object should carry. Content-Type comes from the +// CONTENT_TYPES mapping when the extension is one it names; otherwise the +// source object's own headers are read with HeadObject, which gives its +// Content-Type (octet-stream when it carries none) plus whichever of +// Content-Encoding, Content-Disposition and Content-Language it was stored +// with — headers a REPLACE copy would otherwise drop. +const COPIED_STORAGE_HEADERS = [ + "ContentEncoding", + "ContentDisposition", + "ContentLanguage", +]; +async function copiedHeaders({ s3, sourceBucket, key }) { const mapped = CONTENT_TYPES[path.extname(key).toLowerCase()]; - if (mapped != null) return mapped; + if (mapped != null) return { ContentType: mapped }; const head = await s3.send( new HeadObjectCommand({ Bucket: sourceBucket, Key: key }) ); - return head.ContentType || "application/octet-stream"; + const headers = { + ContentType: head.ContentType || "application/octet-stream", + }; + COPIED_STORAGE_HEADERS.forEach((field) => { + if (head[field]) headers[field] = head[field]; + }); + return headers; } // Same-key copies every object under `prefix` from sourceBucket into // destBucket, giving each copy the Cache-Control tier for its key. COPY (the // default) cannot set headers the source object never had, so that takes // MetadataDirective: REPLACE, which rewrites the metadata of every key under -// the prefix — Content-Encoding, Content-Disposition and any x-amz-meta-* are -// dropped unless restated alongside Content-Type. Nothing sets those: the -// upload router writes ContentType alone. Returns the number of objects -// copied. +// the prefix — anything not restated alongside Content-Type is dropped, x-amz- +// meta-* included. The upload router writes Content-Type alone; a hand-placed +// object keeps the storage headers copiedHeaders restates for it. Returns the +// number of objects copied. async function copyPrefix({ sourceBucket, destBucket, prefix }) { const { s3 } = getClients(); let copied = 0; @@ -624,7 +636,7 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { }) ); for (const obj of list.Contents || []) { - const contentType = await copiedContentType({ + const headers = await copiedHeaders({ s3, sourceBucket, key: obj.Key, @@ -635,7 +647,7 @@ async function copyPrefix({ sourceBucket, destBucket, prefix }) { Key: obj.Key, CopySource: buildCopySource(sourceBucket, obj.Key), MetadataDirective: "REPLACE", - ContentType: contentType, + ...headers, CacheControl: cacheControlForKey(obj.Key), }) ); diff --git a/scripts/publish-static.js b/scripts/publish-static.js index 3e9d906c3..2d2f877af 100644 --- a/scripts/publish-static.js +++ b/scripts/publish-static.js @@ -293,8 +293,7 @@ async function main() { // writes into build/, and public/index.html is the HTML template the // build renders from. Either one served as-is would hand a visitor the // placeholders un-interpolated. The rendered page is uploaded below. - // A publish only writes: it deletes nothing already in the bucket, so a - // bucket that picked up either key before keeps it until emptied. + // A publish only writes; it deletes nothing already in the bucket. const uploadedBuild = await provision.uploadDirectory({ bucket, dir: path.join(rootDir, "build"), diff --git a/src/pre/uploadKey.ts b/src/pre/uploadKey.ts index 8b82a616b..bbe3427c5 100644 --- a/src/pre/uploadKey.ts +++ b/src/pre/uploadKey.ts @@ -8,10 +8,8 @@ // Is this stored value one of the keys API/Backend/Upload/uploadRouter.js // writes when the S3 asset bucket is configured? This is a copy of // ASSETS_UPLOAD_KEY in API/Backend/Upload/validate.js; see it for the shape -// this matches and why. The Essence bundle has no import path into a CommonJS -// module run by Node, and configure/src/core/upload.js carries a third copy -// for the CMS bundle. tests/unit/uploadKeyClassifier.spec.js runs one table of -// values through all three and fails if they classify any of them differently. +// this matches, why, and why each bundle carries its own copy. +// configure/src/core/upload.js carries the third. const ASSETS_UPLOAD_KEY = /^assets\/[^/]+\/[^/]+\/uploads\// // What URL should the page request for a stored value? Four cases, checked diff --git a/tests/unit/awsProvision.spec.js b/tests/unit/awsProvision.spec.js index a8e86b7af..52774c282 100644 --- a/tests/unit/awsProvision.spec.js +++ b/tests/unit/awsProvision.spec.js @@ -1020,9 +1020,7 @@ test.describe('cacheControlForKey', () => { // [key, expected Cache-Control]. Three tiers: revalidate-always for the // entry page and the baked config, immutable for the content-addressed // keys (hashed webpack output and the never-overwritten plugin uploads), - // a short TTL for everything else. The upload-key half of the immutable - // tier belongs to tests/unit/uploadKeyClassifier.spec.js, which checks it - // against the other two copies of that classifier. + // a short TTL for everything else. const TIERS = [ ['index.html', 'no-cache'], ['build/index.html', 'no-cache'], @@ -1030,6 +1028,7 @@ test.describe('cacheControlForKey', () => { ['build/static/js/main.abc123.js', 'max-age=31536000, immutable'], ['build/static/css/x.css', 'max-age=31536000, immutable'], ['build/static/media/a.png', 'max-age=31536000, immutable'], + ['assets/M/S/uploads/x.png', 'max-age=31536000, immutable'], ['build/asset-manifest.json', 'max-age=300'], // Under build/static but not content-hashed, so explicitly NOT // immutable. @@ -1144,10 +1143,13 @@ test.describe('copyPrefix', () => { const UPLOAD_UUID = '6f1e2a3c-4b5d-4e6f-8a9b-0c1d2e3f4a5b' const UPLOAD_KEY = `assets/TestMission/CardPlugin/uploads/${UPLOAD_UUID}.png` - // What each mocked source object carries as its own Content-Type, for the + // What each mocked source object carries as its own headers, for the // extensions the table does not name. An empty head is a source with none. const SOURCE_HEADS = { - 'assets/TestMission/scan.tif': { ContentType: 'image/tiff' }, + 'assets/TestMission/scan.tif': { + ContentType: 'image/tiff', + ContentEncoding: 'gzip', + }, 'assets/TestMission/untyped.bin': {}, } @@ -1239,6 +1241,11 @@ test.describe('copyPrefix', () => { expect(copied['assets/TestMission/scan.tif'].ContentType).toBe( 'image/tiff' ) + // The same head carries the storage headers a REPLACE copy would + // otherwise drop, so they ride along onto the copy. + expect(copied['assets/TestMission/scan.tif'].ContentEncoding).toBe( + 'gzip' + ) // ...which is where a source with no Content-Type of its own lands. expect(copied['assets/TestMission/untyped.bin'].ContentType).toBe( 'application/octet-stream' diff --git a/tests/unit/cfnTemplate.spec.js b/tests/unit/cfnTemplate.spec.js index 5d4eba32c..7b8d9940f 100644 --- a/tests/unit/cfnTemplate.spec.js +++ b/tests/unit/cfnTemplate.spec.js @@ -138,7 +138,7 @@ test.describe('renderCfnTemplate', () => { expect(dist.DefaultRootObject).toBe('index.html') expect( dist.DefaultCacheBehavior.CachePolicyId, - 'must be a policy whose maximum TTL is a year or more, or this edge caps the immutable Cache-Control tier (this id is the managed CachingOptimized)' + "changing this id means re-checking the policy's Maximum TTL by hand: below a year, this edge caps the immutable Cache-Control tier (this id is the managed CachingOptimized)" ).toBe('658327ea-f89d-4fab-a63d-7e88639e58f6') const associations = dist.DefaultCacheBehavior.FunctionAssociations expect(associations).toHaveLength(1) diff --git a/tests/unit/uploadRouterS3.spec.js b/tests/unit/uploadRouterS3.spec.js index 8ec825731..21c024126 100644 --- a/tests/unit/uploadRouterS3.spec.js +++ b/tests/unit/uploadRouterS3.spec.js @@ -25,7 +25,6 @@ import path from 'path' const ROUTER_PATH = '../../API/Backend/Upload/uploadRouter.js' const MODE_PATH = '../../API/Backend/Utils/deploymentMode.js' const MISSIONS_DIR = path.join(__dirname, '../../Missions') -const { ASSETS_UPLOAD_KEY } = require('../../API/Backend/Upload/validate') const ENV_KEYS = ['MMGIS_DEPLOYMENT_MODE', 'MMGIS_SHARED_ASSET_BUCKET'] @@ -152,10 +151,6 @@ test.describe('upload router lean-mode S3 storage', () => { `^assets/${mission}/CardPlugin/uploads/[0-9a-f-]{36}\\.png$` ) ) - // And the shape every consumer of the stored value classifies as - // an upload key: miss it and the dashboard resolves the path - // against the mission folder instead of the dashboard root. - expect(ASSETS_UPLOAD_KEY.test(body.path)).toBe(true) expect(s3.calls.length).toBe(1) const cmd = s3.calls[0] @@ -166,6 +161,17 @@ test.describe('upload router lean-mode S3 storage', () => { expect(cmd.input.ContentType).toBe('image/png') expect(Buffer.compare(cmd.input.Body, PNG_BYTES)).toBe(0) expect(cmd.input.ContentLength).toBe(PNG_BYTES.length) + // Exactly these, nothing more: publishing copies the object with + // MetadataDirective REPLACE, which restates this set and drops + // anything else, so a header added here would vanish on the copy + // (scripts/lib/aws-provision.js copyPrefix). + expect(Object.keys(cmd.input).sort()).toEqual([ + 'Body', + 'Bucket', + 'ContentLength', + 'ContentType', + 'Key', + ]) // No partial/parallel write under Missions/. expect(fs.existsSync(path.join(MISSIONS_DIR, mission))).toBe(false) diff --git a/tests/unit/webpackHashedOutput.spec.js b/tests/unit/webpackHashedOutput.spec.js index 143ed6a2a..97fc5fa51 100644 --- a/tests/unit/webpackHashedOutput.spec.js +++ b/tests/unit/webpackHashedOutput.spec.js @@ -68,11 +68,6 @@ test.describe('webpack production output is content-hashed', () => { expect(CONFIG.output.filename).toMatch(/^static\/js\//) expect(CONFIG.output.chunkFilename).toMatch(HASH_TOKEN) expect(CONFIG.output.chunkFilename).toMatch(/^static\/js\//) - // The name an asset module falls back to when no rule gives it one. - // The config leaves it to webpack's own hashed default; setting it to - // anything hashless is what this guards against. - if (CONFIG.output.assetModuleFilename) - expect(CONFIG.output.assetModuleFilename).toMatch(HASH_TOKEN) }) test("MiniCssExtractPlugin's filenames land hashed under static/css", () => { From 1bab0dd1ccc9755ae0448ac53ae0e9b09e44214e Mon Sep 17 00:00:00 2001 From: Carson Davis Date: Thu, 3 Sep 2026 18:55:32 -0500 Subject: [PATCH 13/13] [316] Set Default TTL 0 and move the rotation note into its own section --- .../serving-a-dashboard-from-your-domain.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/infrastructure/serving-a-dashboard-from-your-domain.md b/docs/infrastructure/serving-a-dashboard-from-your-domain.md index 88c3a7c52..2ead02edc 100644 --- a/docs/infrastructure/serving-a-dashboard-from-your-domain.md +++ b/docs/infrastructure/serving-a-dashboard-from-your-domain.md @@ -13,7 +13,7 @@ The examples below use the path `/tools/dashboard`; substitute your own everywhe - all query strings in the cache key, - Minimum TTL 0, - Maximum TTL 31536000 (one year) or more, - - Default TTL — anything between the two (it never applies). + - Default TTL 0. 3. **Add two cache behaviors** pointing at that origin, both using that cache policy: - path pattern `/tools/dashboard` — exact, no wildcard, @@ -49,11 +49,9 @@ Every dashboard is password-protected, so every row also sits behind the passwor **`Authorization` in the cache key:** dashboards are password-protected, and your CloudFront caches whatever we return. If the header is forwarded but not part of the cache key, one visitor's authenticated page gets cached and served to the next visitor who never entered a password. In the cache key, the header is both forwarded and kept separate per credential. Nothing we return is marked for shared caches: our responses carry no `public` in their `Cache-Control`. Your CloudFront caches them anyway, on `max-age` alone, under the per-credential key you just configured — that is how CloudFront behaves. An intermediary proxy that follows the standard to the letter will never serve them to another request, because they were fetched with an `Authorization` header (RFC 9111 §3.5); reuse would take `public`, `must-revalidate` or `s-maxage`, and we send none of the three. Password-gated files stay out of caches we know nothing about. Every visitor needs that password — there is no unauthenticated mode today — and the same password opens every dashboard published from the same MMGIS environment, so give it only to an audience you would hand every one of those dashboards to. Never publish it on a page. -**After a password rotation, invalidate.** A new password does not evict what your edge is already holding under the old one: a cached object lives out its `max-age` whatever the credential now is. So when the dashboards password is rotated, run an invalidation on your own distribution — otherwise the retired password keeps opening cached pages until they age out, up to a year for the fingerprinted files. - **All query strings in the cache key:** a policy that drops query strings never sends them to us. A deep link like `/tools/dashboard?view=2` then reaches us stripped of its `?view=2`, and the address we redirect the visitor to has lost it for good. -**Minimum TTL 0:** a policy's minimum cache time overrides what our responses ask for, and most of the managed ones sit above 0. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files roughly five minutes after the publish completes, up to about ten while our own invalidation propagates. +**Minimum TTL 0:** a policy's minimum cache time overrides what our responses ask for, and most of the managed ones sit above 0. Our slash-less-entry redirect must not be cached — it contains one visitor's query string — and without an explicit 0 your edge would replay that visitor's redirect to the next. A floor above 0 also holds the entry page and the mission configuration, which we mark to revalidate before every use (`no-cache`, not `no-store`); at 0 a republish reaches your domain with no purge on your side — those two immediately, the supporting files roughly five minutes after the publish completes, plus however long our invalidation takes to reach your region (the publish does not wait for it). **Maximum TTL a year:** the files whose names are content-fingerprinted — the JS, CSS and media bundles, and the images uploaded into the dashboard, each stored under a name that is never reused — are cacheable forever (`immutable`), and a policy maximum below a year would cap them at whatever it sets. AWS's managed `UseOriginCacheControlHeaders` policies (and the `-QueryStrings` variant) already pair Minimum TTL 0 with a one-year maximum, but neither puts `Authorization` in the cache key — which is why the policy has to be a custom one, with both bounds set by hand: Minimum 0 so it raises no floor over what we ask for, Maximum a year so it does not cap the immutable tier. @@ -62,3 +60,7 @@ Every dashboard is password-protected, so every row also sits behind the passwor **HTTPS only to the origin:** the dashboard's password rides on the `Authorization` header of every request you forward. Over plain HTTP it would cross the internet unencrypted. **No viewer `Host` header:** our distribution answers only to its own `*.cloudfront.net` name; a request carrying your hostname is rejected by AWS with a 403 before anything of ours runs. CloudFront omits the viewer's `Host` by default — the hazard is specifically the managed `AllViewer` origin request policy, which forwards it. `AllViewerExceptHostHeader` forwards everything else while excluding it. + +## After you're set up + +**After a password rotation, invalidate.** A new password does not evict what your edge is already holding under the old one: a cached object lives out its `max-age` whatever the credential now is. So when the dashboards password is rotated, run an invalidation on your own distribution — otherwise the retired password keeps opening cached pages until they age out, up to a year for the fingerprinted files.