feat(storage): resolve S3 config from S3_* env vars at runtime by UpperM · Pull Request #457 · emdash-cms/emdash

UpperM · 2026-04-11T17:21:47Z

What does this PR do?

astro.config.mjs runs at Astro build time, so values passed to s3({...}) get baked into the virtual emdash/config module. That blocks the "build the image once, inject credentials at boot" pattern.

This PR resolves any field omitted from s3({...}) from the matching S3_* env var at runtime (Node only). Existing deployments passing explicit values are unaffected, explicit always wins. s3() with no args is now valid for fully env-driven configs.

Precedence: explicit s3({...}) > S3_* env var > absent. Empty strings are treated as absent in both sources. Endpoint values are validated at both sources with the source named in the error (S3_ENDPOINT is not a valid http/https URL vs s3({ endpoint }) is not a valid http/https URL).

Scope is Node; Workers binding resolution is out of scope and documented as such.

// astro.config.mjs
storage: s3(),                                    // all fields from S3_* env
storage: s3({ publicUrl: "https://cdn.ex.com" }), // mix
storage: s3({ endpoint, bucket, accessKeyId, secretAccessKey }), // unchanged

Docs updated in storage.mdx / configuration.mdx, including a prerequisite note that users must install @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner themselves since core deliberately does not bundle the SDK (the ambient packages/core/src/aws-sdk.d.ts establishes the "bring-your-own-SDK" contract but was previously undocumented).

26 new unit tests cover precedence, empty-string coercion, URL validation, credential pairing, and the Workers typeof process guard.

Type of change

Checklist

I have read CONTRIBUTING.md
pnpm typecheck passes
pnpm --silent lint:json 0 diagnostics in files touched by this PR (3 pre-existing warnings in unrelated files left alone per scope rules)
pnpm test passes (2280 / 2280)
pnpm format has been run
Tests added (26 in tests/unit/storage/s3.test.ts)
Changeset added (.changeset/feat-s3-runtime-env-fallback.md, minor)
New features link to an approved Discussion Resolve S3 config from env vars at runtime #459

AI-generated code disclosure

This PR includes AI-generated code

changeset-bot · 2026-04-11T17:21:55Z

🦋 Changeset detected

Latest commit: 9f325d6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages

Name	Type
emdash	Major
@emdash-cms/cloudflare	Major
@emdash-cms/plugin-ai-moderation	Major
@emdash-cms/plugin-atproto	Major
@emdash-cms/plugin-audit-log	Major
@emdash-cms/plugin-color	Major
@emdash-cms/plugin-embeds	Major
@emdash-cms/plugin-forms	Major
@emdash-cms/plugin-webhook-notifier	Major
@emdash-cms/admin	Major
@emdash-cms/auth	Major
@emdash-cms/blocks	Major
@emdash-cms/gutenberg-to-portable-text	Major
@emdash-cms/x402	Major
create-emdash	Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions · 2026-04-11T17:21:59Z

Scope check

This PR changes 566 lines across 8 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

pkg-pr-new · 2026-04-11T17:23:19Z

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@457

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@457

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@457

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@457

emdash

npm i https://pkg.pr.new/emdash@457

create-emdash

npm i https://pkg.pr.new/create-emdash@457

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@457

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@457

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@457

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@457

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@457

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@457

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@457

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@457

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@457

commit: 9f325d6

astro.config.mjs is evaluated in Node at Astro build time, so values passed to s3({...}) are captured and baked into the virtual emdash/config module. Operators running the same image across multiple container deployments (per-tenant credentials injected at boot) have no way to change those values without a rebuild. Adds runtime resolution: any field omitted from s3({...}) is read from the matching S3_* env var when the container starts. Existing deployments that pass explicit values to s3({...}) are unaffected. Env var names match the existing docs convention (S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_REGION, S3_PUBLIC_URL). Precedence per field: explicit s3({...}) value > S3_* env var > absent. Empty strings are treated as absent in both sources. Endpoint values are validated as http/https URLs with a host at both sources; invalid values throw MISSING_S3_CONFIG with the source named so operators know whether to fix their env var or their astro.config.mjs. endpoint and bucket remain required. accessKeyId and secretAccessKey become optional in the type (both or neither). Relaxes s3() to accept Partial<S3StorageConfig> so s3() with no args is valid for fully env-driven deployments. Scope: Node. process.env is used for runtime reads, guarded with typeof-process for bundler safety. Workers users should continue passing explicit values to s3({...}); worker binding resolution is not in scope for this change.

@aws-sdk

The AWS SDK is a bring-your-own dependency and is not installed in CI. Stub @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner via vi.mock so loading s3.ts (which statically imports both) does not require the real packages. Verified by removing @aws-sdk from node_modules and rerunning: all 26 resolveS3Config tests pass.

UpperM · 2026-04-11T17:48:06Z

I stubbed @aws-sdk/* with vi.mock in the s3 unit tests to keep core dependency-free. Open to adding them as devDependencies instead if you'd prefer real-SDK tests. Any preference ?

ascorbic · 2026-04-11T18:10:53Z

packages/core/src/storage/s3.ts

+function parseSource(raw: unknown, source: "env" | "explicit"): Partial<S3StorageConfig> {
+	const result = s3ConfigSchema.safeParse(stripAbsent(raw));
+	if (result.success) return result.data;
+	const issue = result.error.issues[0];
+	const pathKey = issue?.path[0];
+	if (!issue || !isConfigKey(pathKey)) fail("S3 config validation failed");
+	const label = source === "env" ? ENV_KEYS[pathKey] : `s3({ ${pathKey} })`;
+	fail(`${label} ${issue.message}`);
+}
+
+function readEnv(): Partial<S3StorageConfig> {
+	if (typeof process === "undefined" || !process.env) return {};
+	const raw: Record<string, unknown> = {};
+	for (const [field, envKey] of Object.entries(ENV_KEYS)) {
+		raw[field] = process.env[envKey];
+	}
+	return parseSource(raw, "env");
+}


I don't think we should be parsing and validating all the env vars here. We should parse them individually, only if they're used. If somebody has an explicit value set in config, a malformed env var shouldn;t break the build. Instead of this eager parsing, I'd construct the config object manually, defaulting to any manually set values and falling back to the env var if not set, and only validating it after that.

You're right.
Reworked: resolveS3Config now loops through each field, takes the explicit value if present, falls back to the env var otherwise, then validates once on the merged result. A malformed env var is never seen if the caller provides that field.
Removed readEnv, parseSource, stripAbsent, it's one loop + one safeParse now.

ascorbic · 2026-04-11T18:12:12Z

packages/core/src/storage/s3.ts

 		this.publicUrl = config.publicUrl;
 		this.endpoint = config.endpoint;

+		// eslint-disable-next-line typescript-eslint(no-unsafe-type-assertion) -- SDK type declares credentials as required but accepts omission at runtime when the caller provides credentials through an alternative mechanism


Is this an error in the S3 client types?

That eslint-disable comment was not needed, my bad.

Env vars are only used as fallback when no explicit value is provided. A malformed env var no longer breaks the build if the caller overrides that field. Removes readEnv/parseSource/stripAbsent in favor of a single loop + one Zod parse on the merged result. Removes misleading eslint-disable comment on S3Client constructor.

ascorbic

Great stuff, thanks

@aws-sdk

…h-cms#457) * feat(storage): resolve S3 config from S3_* env vars at runtime astro.config.mjs is evaluated in Node at Astro build time, so values passed to s3({...}) are captured and baked into the virtual emdash/config module. Operators running the same image across multiple container deployments (per-tenant credentials injected at boot) have no way to change those values without a rebuild. Adds runtime resolution: any field omitted from s3({...}) is read from the matching S3_* env var when the container starts. Existing deployments that pass explicit values to s3({...}) are unaffected. Env var names match the existing docs convention (S3_ENDPOINT, S3_BUCKET, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY, S3_REGION, S3_PUBLIC_URL). Precedence per field: explicit s3({...}) value > S3_* env var > absent. Empty strings are treated as absent in both sources. Endpoint values are validated as http/https URLs with a host at both sources; invalid values throw MISSING_S3_CONFIG with the source named so operators know whether to fix their env var or their astro.config.mjs. endpoint and bucket remain required. accessKeyId and secretAccessKey become optional in the type (both or neither). Relaxes s3() to accept Partial<S3StorageConfig> so s3() with no args is valid for fully env-driven deployments. Scope: Node. process.env is used for runtime reads, guarded with typeof-process for bundler safety. Workers users should continue passing explicit values to s3({...}); worker binding resolution is not in scope for this change. * test(storage): mock aws-sdk in s3 unit tests The AWS SDK is a bring-your-own dependency and is not installed in CI. Stub @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner via vi.mock so loading s3.ts (which statically imports both) does not require the real packages. Verified by removing @aws-sdk from node_modules and rerunning: all 26 resolveS3Config tests pass. * refactor(storage): merge-first-validate-once for S3 config Env vars are only used as fallback when no explicit value is provided. A malformed env var no longer breaks the build if the caller overrides that field. Removes readEnv/parseSource/stripAbsent in favor of a single loop + one Zod parse on the merged result. Removes misleading eslint-disable comment on S3Client constructor.

github-actions bot added area/core area/docs size/XL labels Apr 11, 2026

github-actions bot added the cla: signed label Apr 11, 2026

UpperM added 2 commits April 11, 2026 19:36

UpperM force-pushed the fix/s3-runtime-env-fallback branch from 6302f08 to bdfbb7c Compare April 11, 2026 17:44

UpperM marked this pull request as ready for review April 11, 2026 17:52

ascorbic reviewed Apr 11, 2026

View reviewed changes

ascorbic added the Awaiting author response label Apr 11, 2026

UpperM force-pushed the fix/s3-runtime-env-fallback branch from 9fb30ba to 9f325d6 Compare April 12, 2026 12:41

ascorbic approved these changes Apr 12, 2026

View reviewed changes

ascorbic merged commit f2b3973 into emdash-cms:main Apr 12, 2026
26 checks passed

emdashbot bot mentioned this pull request Apr 12, 2026

ci: release #488

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(storage): resolve S3 config from S3_* env vars at runtime#457

feat(storage): resolve S3 config from S3_* env vars at runtime#457
ascorbic merged 3 commits intoemdash-cms:mainfrom
UpperM:fix/s3-runtime-env-fallback

UpperM commented Apr 11, 2026 •

edited

Loading

Uh oh!

changeset-bot bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 11, 2026

Uh oh!

pkg-pr-new bot commented Apr 11, 2026 •

edited

Loading

Uh oh!

UpperM commented Apr 11, 2026

Uh oh!

ascorbic Apr 11, 2026

Uh oh!

UpperM Apr 12, 2026

Uh oh!

ascorbic Apr 11, 2026

Uh oh!

UpperM Apr 12, 2026

Uh oh!

ascorbic left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

UpperM commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Type of change

Checklist

AI-generated code disclosure

Uh oh!

changeset-bot bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

github-actions bot commented Apr 11, 2026

Scope check

Uh oh!

pkg-pr-new bot commented Apr 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

UpperM commented Apr 11, 2026

Uh oh!

ascorbic Apr 11, 2026

Choose a reason for hiding this comment

Uh oh!

UpperM Apr 12, 2026

Choose a reason for hiding this comment

Uh oh!

ascorbic Apr 11, 2026

Choose a reason for hiding this comment

Uh oh!

UpperM Apr 12, 2026

Choose a reason for hiding this comment

Uh oh!

ascorbic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

UpperM commented Apr 11, 2026 •

edited

Loading

changeset-bot bot commented Apr 11, 2026 •

edited

Loading

pkg-pr-new bot commented Apr 11, 2026 •

edited

Loading