diff --git a/README.md b/README.md index ed68798..e3b886d 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,7 @@ request-local cache -> process-local cache -> Redis cache -> fallback function - Redis misses call the fallback and attempt to populate Redis and, when active, the process-local cache. Tracked invalidation may suppress both publications. - Redis cache read/write failures are logged, counted in metrics, and fail open; fallback results still return when fallback succeeds. `invalidateRemote` logs/counts Redis failures and rethrows them so callers do not assume invalidation succeeded. - Cache-key construction and config-provider failures also fail open and run the fallback uncached. -- Missing process-local/Redis TTL or ramp config disables that layer, records a disabled reason, and falls through to the next layer/fallback. +- A missing effective process-local/Redis TTL disables that layer by policy; a configured TTL with no ramp defaults to 100%. Disabled layers record a disabled reason and fall through to the next layer/fallback. Caching as a whole is only active inside an enabled context, described next. @@ -116,7 +116,7 @@ await dialcache.enable(async () => { | `keyType` | yes | The kind of id the key addresses (e.g. `"user_id"`). Together with the id, the invalidation unit for tracked entries. | | `useCase` | yes | Identifies the individual cache: part of the stored key and the metrics label. | | `cacheKey` | yes | Selector over `fn`'s parameters; returns a bare id or `{ id, args }`. | -| `defaultConfig` | no | `DialCacheKeyConfig` used when no runtime override applies (see [Runtime config](#runtime-config-and-ramp-controls)). | +| `defaultConfig` | no | `DialCacheKeyConfig` baseline policy that runtime config overlays field by field (see [Runtime config](#runtime-config-and-ramp-controls)). | | `serializer` | when the return type is not statically JSON-compatible | Per-function `Serializer` for Redis values (see [Serialization](#serialization)). | | `trackForInvalidation` | no (default `false`) | Opts this use case's Redis entries into watermark-based targeted invalidation. | | `fallbackTimeoutMs` | no (default `60_000`) | Fallback deadline in milliseconds, at most 2,147,483,647; `null` disables it (see [Fallback deadlines](#fallback-deadlines)). | @@ -170,14 +170,26 @@ Instance-wide behavior is set through the `DialCache` constructor: | `namespace` | `"urn"` | Logical cache namespace and first key component (see [Keys, ids, and extra dimensions](#keys-ids-and-extra-dimensions)). | | `redis` | none | `{ client: DialCacheRedisClient }`; enables the Redis layer (see [Redis-backed TTL cache](#redis-backed-ttl-cache)). | | `localMaxSize` | `10_000` | Global process-local entry cap; `0` disables process-local storage. Nonnegative safe integer. | -| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation; `null` falls back to the function's `defaultConfig`. | +| `cacheConfigProvider` | none | Resolves runtime config per enabled invocation as a sparse overlay on the function's `defaultConfig`; `null` applies no overrides. | | `rampSampler` | deterministic by key and layer | Percentage sampler for partial ramps; `randomRampSampler` is also exported. | | `metrics` | disabled | A `DialCacheMetricsAdapter` (see [Metrics](#metrics)). | | `logger` | `console` | Receives operational cache failures (`debug`, `warn`, `error`). | Per-invocation cache policy is a `DialCacheKeyConfig`: per-layer `ttlSec` and `ramp` maps keyed by `CacheLayer.LOCAL` (process-local) and `CacheLayer.REMOTE` (Redis), plus a `requestLocal` boolean. -Every cached function can provide a per-use-case `defaultConfig`; a `cacheConfigProvider` can override it at runtime. If the provider returns `null`, DialCache falls back to the cached function's `defaultConfig`. If neither exists, or a layer's TTL/ramp is missing or disabled, only that layer is skipped. +Every cached function can provide an optional per-use-case `defaultConfig`. It is the baseline policy, and the `cacheConfigProvider` result is a sparse field-level overlay on that baseline. Precedence is: runtime field, then `defaultConfig` field, then DialCache's disabled baseline. + +The disabled baseline sets `requestLocal` to false and leaves the process-local and Redis TTLs unset. A shared layer with no effective TTL is disabled by policy. When a shared layer has an effective TTL but no effective ramp, its ramp defaults to 100%. + +`DialCacheKeyConfig` preserves an omitted `requestLocal` as `undefined` so the overlay can distinguish omission from an explicit `false`; the effective value still defaults to false after resolution. + +A provider result of `null` (or defensive `undefined`) applies no overrides. An empty `DialCacheKeyConfig` and omitted runtime fields also inherit the baseline. Use explicit values to override inherited policy: `requestLocal: false` disables request-local caching and a layer ramp of `0` disables that shared layer. `DialCacheKeyConfig.disabled()` is that explicit kill switch in one call: request-local off and both shared layers ramped to 0. + +DialCache validates `defaultConfig` when `cached()` registers the definition: TTLs must be positive safe integers, ramps must be finite percentages from 0 to 100, layer maps must be objects, and `requestLocal` must be a boolean when present. Invalid defaults are rejected immediately. + +Registration captures an immutable internal snapshot of `defaultConfig`; mutating the supplied config or its maps later does not change the use case's baseline. Runtime policy changes belong in the provider's returned overlay. + +Runtime TTL and ramp leaves are used as supplied instead of falling back to valid default leaves. An invalid TTL disables that layer with `invalid_ttl`; a non-finite or nonnumeric ramp disables it with `invalid_ramp`; finite runtime ramps retain the defensive clamp to 0–100. Other layers can still run, and invalid leaves also record a `config_resolution` error so provider garbage is alertable separately from intentional ramp-downs. A malformed runtime config object, layer-map shape, or `requestLocal` value fails config resolution for the invocation, records `config_error`, and executes the fallback uncached. `cacheConfigProvider` is called for every enabled cached-function invocation before DialCache performs any cache lookup. Keep it cheap, cache any remote/config-store reads inside the provider, and avoid work that would erase the benefit of a cache hit. @@ -188,17 +200,27 @@ const dialcache = new DialCache({ cacheConfigProvider: async (key) => { if (key.useCase === "GetUser") { return new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, - ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 25 }, + // Sparse override: inherit both TTLs and the local ramp from defaultConfig. + ramp: { [CacheLayer.REMOTE]: 25 }, }); } - return null; // use the cached function's defaultConfig + return null; // apply no overrides; use the cached function's baseline }, rampSampler: ({ key, layer }) => deterministicPercentFor(`${key.urn}:${layer}`), }); + +const getUser = dialcache.cached((userId: string) => db.fetchUser(userId), { + keyType: "user_id", + useCase: "GetUser", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + // Omitted ramps default to 100% because these layers have TTLs. + ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 300 }, + }), +}); ``` -`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values use `rampSampler`; the default sampler is deterministic by cache key and layer, so the same key is consistently sampled in or out of a partial rollout. Provider errors fail open and execute the fallback function. +`ramp` values are percentages from 0 to 100. `0` disables the layer, `100` enables it, and intermediate values use `rampSampler`; the default sampler is deterministic by cache key and layer, so the same key is consistently sampled in or out of a partial rollout. DialCache fetches and resolves one config snapshot per enabled invocation. Provider errors do not activate defaults: they fail open, record `config_error`, and execute the fallback function uncached. ## Cache layers @@ -223,7 +245,7 @@ const getUser = dialcache.cached( `requestLocal` is a runtime boolean rather than a TTL/ramp-controlled `CacheLayer`. The `cacheConfigProvider` can turn it on or off for each invocation. `DialCacheKeyConfig.enabled(ttlSec)` enables only process-local and Redis caching, so request-local caching must be selected explicitly. -DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When `requestLocal` is missing or false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. +DialCache resolves the runtime config once per enabled invocation and uses it for the entire lookup. When the effective `requestLocal` value is false, the invocation skips request-local lookup and storage without deleting an entry already memoized in the scope. A later invocation that enables request-local caching can reuse that entry. The outermost `enable()` call owns the request-local lifetime, and nested `enable()` calls reuse that scope. Request-local state is allocated lazily, only when an invocation enables the layer, so scopes that use only process-local or Redis caching do not allocate it. @@ -556,7 +578,7 @@ The Prometheus adapter emits: | --- | --- | --- | --- | | `dialcache_request_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache-layer requests that reached an enabled layer | | `dialcache_miss_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer` | Cache misses | -| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `missing_config`, `invalid_ttl`, `ramped_down`, `config_error`) | +| `dialcache_disabled_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `reason` | Cache skips (`context`, `policy_disabled`, `invalid_ttl`, `invalid_ramp`, `ramped_down`, `config_error`) | | `dialcache_error_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `layer`, `error`, `in_fallback` | Cache/fallback errors classified by a bounded failure site | | `dialcache_invalidation_counter` | Counter | `cache_namespace`, `key_type`, `layer` | Invalidation calls for the layers touched | | `dialcache_coalesced_counter` | Counter | `cache_namespace`, `use_case`, `key_type`, `scope` | Coalesced requests split by `request_local` or `process` scope | @@ -565,6 +587,8 @@ The Prometheus adapter emits: | `dialcache_serialization_timer` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer`, `operation` | Redis serializer dump/load latency | | `dialcache_size_histogram` | Histogram | `cache_namespace`, `use_case`, `key_type`, `layer` | Serialized Redis payload size in bytes | +`policy_disabled` means that a process-local or Redis layer has no effective TTL after runtime overlays are applied. It is an intentional policy outcome, including the default when `defaultConfig` is omitted, rather than a configuration-loading failure. + Every metric carries `cache_namespace`, including disabled-context, key-construction, coalescing, and invalidation paths that do not have a constructed key. Its value is `DialCacheConfig.namespace`, defaulting to `urn`. The `layer` label is `request_local`, `local` (process-local), or `remote`. Disabled-context, key-construction, and config-provider failures use `noop` because no cache layer was reached. The bounded `scope` label on `dialcache_coalesced_counter` distinguishes request-local from instance-scoped single-flight work. `scope="process"` coordinates calls only within one `DialCache` instance; separate instances in the same process do not share in-flight state. ### Datadog diff --git a/scripts/test-package.mjs b/scripts/test-package.mjs index b69b03e..fd2f86a 100644 --- a/scripts/test-package.mjs +++ b/scripts/test-package.mjs @@ -27,12 +27,15 @@ const rootConsumer = `import { type DialCacheKeyInit, type DialCacheMetricsAdapter, type DialCacheRedisClient, + type DisabledReason, type InvalidationMetricLabels, type MetricErrorKind, type ProcessCoalescingState, type RedisConfig, type Serializer, } from "dialcache"; +// @ts-expect-error The unused MissingKeyConfigError class was removed instead of deprecated. +import { MissingKeyConfigError } from "dialcache"; import { createNodeRedisDialCacheClient } from "dialcache/node-redis"; import { READ_CACHE_SCRIPT } from "dialcache/redis-protocol"; import { @@ -78,6 +81,7 @@ const redisProtocolError = new DialCacheRedisProtocolError("Invalid DialCache Re const fallbackTimeoutError = new FallbackTimeoutError("Load", 1_000); const coalescingState: CoalescingState = cache.getCoalescingState(); const processCoalescingState: ProcessCoalescingState = coalescingState.process; +const disabledOverlay: DialCacheKeyConfig = DialCacheKeyConfig.disabled(); const load = cache.cached(async (id: string) => id, { keyType: "id", useCase: "Load", @@ -192,6 +196,16 @@ const legacyKeyInit: DialCacheKeyInit = { keyType: "id", id: "123", useCase: "Lo const namespacedKey = new DialCacheKey(keyInit); const requestLocalCoalescingScope: CoalescingScope = "request_local"; const boundedErrorKind: MetricErrorKind = "cache_read"; +const disabledReasons: Readonly> = { + context: true, + policy_disabled: true, + invalid_ttl: true, + invalid_ramp: true, + ramped_down: true, + config_error: true, +}; +// @ts-expect-error Missing configuration now means the documented disabled policy, not a separate reason. +const legacyMissingConfigReason: DisabledReason = "missing_config"; const metricErrorKinds: Readonly> = { key_construction: true, config_resolution: true, @@ -260,6 +274,10 @@ void coalescingState.process; void processCoalescingState.activeLeaders; void requestLocalCoalescingScope; void boundedErrorKind; +void disabledReasons; +void legacyMissingConfigReason; +void MissingKeyConfigError; +void disabledOverlay; void metricErrorKinds; void unboundedErrorKind; void createNodeRedisDialCacheClient; @@ -432,6 +450,32 @@ try { if (!(error instanceof root.DialCacheRedisProtocolError)) { throw new Error("The node-redis protocol error does not match the root ESM export"); } +} +if ("MissingKeyConfigError" in root) { + throw new Error("The removed MissingKeyConfigError class must not be exported from the root ESM entry"); +} +const esmDisabledOverlay = root.DialCacheKeyConfig.disabled(); +if (esmDisabledOverlay.requestLocal !== false || esmDisabledOverlay.ramp[root.CacheLayer.LOCAL] !== 0 || esmDisabledOverlay.ramp[root.CacheLayer.REMOTE] !== 0) { + throw new Error("The packed ESM runtime did not build the disabled() kill-switch overlay"); +} +let calls = 0; +const overlayCache = new root.DialCache({ + cacheConfigProvider: () => new root.DialCacheKeyConfig({ + ramp: { [root.CacheLayer.LOCAL]: 100 }, + }), +}); +const load = overlayCache.cached(async (id) => ({ id, calls: ++calls }), { + keyType: "id", + useCase: "PackedRuntimeOverlay", + cacheKey: (id) => id, + defaultConfig: new root.DialCacheKeyConfig({ + ttlSec: { [root.CacheLayer.LOCAL]: 60 }, + }), +}); +const first = await overlayCache.enable(() => load("123")); +const second = await overlayCache.enable(() => load("123")); +if (calls !== 1 || second !== first) { + throw new Error("The packed ESM runtime did not inherit the default local TTL through a sparse overlay"); }`, ], { cwd: workspace }, @@ -483,7 +527,38 @@ try { if (!(error instanceof root.DialCacheRedisProtocolError)) { throw new Error("The node-redis protocol error does not match the root CommonJS export"); } -}`, +} +if ("MissingKeyConfigError" in root) { + throw new Error("The removed MissingKeyConfigError class must not be exported from the root CommonJS entry"); +} +const cjsDisabledOverlay = root.DialCacheKeyConfig.disabled(); +if (cjsDisabledOverlay.requestLocal !== false || cjsDisabledOverlay.ramp[root.CacheLayer.LOCAL] !== 0 || cjsDisabledOverlay.ramp[root.CacheLayer.REMOTE] !== 0) { + throw new Error("The packed CommonJS runtime did not build the disabled() kill-switch overlay"); +} +void (async () => { + let calls = 0; + const overlayCache = new root.DialCache({ + cacheConfigProvider: () => new root.DialCacheKeyConfig({ + ramp: { [root.CacheLayer.LOCAL]: 100 }, + }), + }); + const load = overlayCache.cached(async (id) => ({ id, calls: ++calls }), { + keyType: "id", + useCase: "PackedRuntimeOverlay", + cacheKey: (id) => id, + defaultConfig: new root.DialCacheKeyConfig({ + ttlSec: { [root.CacheLayer.LOCAL]: 60 }, + }), + }); + const first = await overlayCache.enable(() => load("123")); + const second = await overlayCache.enable(() => load("123")); + if (calls !== 1 || second !== first) { + throw new Error("The packed CommonJS runtime did not inherit the default local TTL through a sparse overlay"); + } +})().catch((error) => { + console.error(error); + process.exitCode = 1; +});`, ], { cwd: workspace }, ); diff --git a/src/config.ts b/src/config.ts index 8ebf01d..760d067 100644 --- a/src/config.ts +++ b/src/config.ts @@ -38,9 +38,17 @@ export class DialCacheKeyConfig { readonly requestLocal?: boolean; constructor(config: { ttlSec?: LayerConfig; ramp?: LayerConfig; requestLocal?: boolean }) { - this.ttlSec = { ...config.ttlSec }; - this.ramp = { ...config.ramp }; - this.requestLocal = config.requestLocal ?? false; + if (config === null || typeof config !== "object" || Array.isArray(config)) { + throw new TypeError("DialCache key config must be an object"); + } + this.ttlSec = cloneLayerConfig(config.ttlSec, "ttlSec"); + this.ramp = cloneLayerConfig(config.ramp, "ramp"); + if (config.requestLocal !== undefined && typeof config.requestLocal !== "boolean") { + throw new TypeError("DialCache requestLocal config must be a boolean"); + } + if (config.requestLocal !== undefined) { + this.requestLocal = config.requestLocal; + } } static enabled(ttlSec: number): DialCacheKeyConfig { @@ -55,6 +63,31 @@ export class DialCacheKeyConfig { }, }); } + + /** + * The explicit kill switch: request-local caching off and both shared + * layers ramped to 0. As a provider overlay it disables every inherited + * layer instead of relying on field omission, which inherits the baseline. + */ + static disabled(): DialCacheKeyConfig { + return new DialCacheKeyConfig({ + requestLocal: false, + ramp: { + [CacheLayer.LOCAL]: 0, + [CacheLayer.REMOTE]: 0, + }, + }); + } +} + +function cloneLayerConfig(config: LayerConfig | undefined, name: "ttlSec" | "ramp"): LayerConfig { + if (config === undefined) { + return {}; + } + if (config === null || typeof config !== "object" || Array.isArray(config)) { + throw new TypeError(`DialCache ${name} config must be a layer map`); + } + return { ...config }; } /** diff --git a/src/dialcache.ts b/src/dialcache.ts index 9edcfb4..0f8b133 100644 --- a/src/dialcache.ts +++ b/src/dialcache.ts @@ -2,12 +2,12 @@ import { performance } from "node:perf_hooks"; import { CacheLayer, + DialCacheKeyConfig, deterministicRampSampler, type Awaitable, type CacheConfigProvider, type CacheRampSampler, type DialCacheConfig, - type DialCacheKeyConfig, type Logger, } from "./config.js"; import { DialCacheContext, getOrCreateRequestLocalCache, type RequestLocalCache } from "./context.js"; @@ -19,6 +19,7 @@ import { labelsFor, type CacheMetricLabels, type DialCacheMetricsAdapter, + type DisabledReason, type MetricErrorKind, type MetricLayer, } from "./metrics.js"; @@ -246,6 +247,7 @@ export class DialCache { * values are shared by reference and must be treated as immutable. */ cached(fn: Fn, options: CachedOptions): CachedFn { + const defaultConfig = snapshotDefaultConfig(options.defaultConfig); const fallbackTimeoutMs = resolveFallbackTimeoutMs(options.fallbackTimeoutMs); this.registerUseCase(options.useCase); @@ -269,7 +271,7 @@ export class DialCache { let key: DialCacheKey; try { - key = this.buildKey(options, options.cacheKey(...args)); + key = this.buildKey(options, options.cacheKey(...args), defaultConfig); } catch (error) { this.logger.error("Could not construct DialCache key", error); this.metrics?.error({ @@ -495,6 +497,7 @@ export class DialCache { const result = await this.localCache.resolveLayerConfig(key, keyConfig); if (result.status === "disabled") { this.metrics?.disabled({ ...labelsFor(key, CacheLayer.LOCAL), reason: result.reason }); + this.recordInvalidLeaf(key, CacheLayer.LOCAL, result.reason); } return result; } catch (error) { @@ -533,11 +536,13 @@ export class DialCache { }); if (result.status === "disabled") { this.metrics?.disabled({ ...labelsFor(key, CacheLayer.REMOTE), reason: result.reason }); + this.recordInvalidLeaf(key, CacheLayer.REMOTE, result.reason); } return result; } catch (error) { this.logger.warn("Error resolving Redis cache config", error); this.recordError(key, CacheLayer.REMOTE, "config_resolution"); + this.metrics?.disabled({ ...labelsFor(key, CacheLayer.REMOTE), reason: "config_error" }); return { status: "disabled", reason: "config_error", ...(key.trackForInvalidation ? { skipCacheWrite: true } : {}) } as const; } } @@ -587,6 +592,18 @@ export class DialCache { this.metrics?.error({ ...labelsFor(key, layer), error: kind, inFallback: false }); } + /** + * Invalid runtime TTL/ramp leaves can only come from provider results, since + * static defaults are validated at registration. Count them as + * config_resolution errors as well as disabled skips so garbage config is + * alertable separately from intentional ramp-downs and disabled policy. + */ + private recordInvalidLeaf(key: DialCacheKey, layer: MetricLayer, reason: DisabledReason): void { + if (reason === "invalid_ttl" || reason === "invalid_ramp") { + this.recordError(key, layer, "config_resolution"); + } + } + private registerUseCase(useCase: string): void { if (useCase === "watermark") { throw new UseCaseNameIsReservedError(useCase); @@ -597,7 +614,11 @@ export class DialCache { this.useCases.add(useCase); } - private buildKey(options: CachedOptions, cacheKey: CacheKeySpec): DialCacheKey { + private buildKey( + options: CachedOptions, + cacheKey: CacheKeySpec, + defaultConfig: DialCacheKeyConfig | null, + ): DialCacheKey { const spec = typeof cacheKey === "object" ? cacheKey : { id: cacheKey }; return new DialCacheKey({ keyType: options.keyType, @@ -605,7 +626,7 @@ export class DialCache { useCase: options.useCase, args: normalizeArgs(spec.args ?? {}), namespace: this.namespace, - defaultConfig: options.defaultConfig ?? null, + defaultConfig, serializer: (options.serializer as Serializer | null | undefined) ?? null, trackForInvalidation: options.trackForInvalidation ?? false, }); @@ -692,6 +713,62 @@ function assertValidFutureBufferMs(futureBufferMs: number): void { } } +function snapshotDefaultConfig(config: DialCacheKeyConfig | null | undefined): DialCacheKeyConfig | null { + if (config === null || config === undefined) { + return null; + } + if (typeof config !== "object" || Array.isArray(config)) { + throw new TypeError("DialCache defaultConfig must be an object"); + } + const ttlSecConfig = config.ttlSec; + const rampConfig = config.ramp; + const requestLocal = config.requestLocal; + if (requestLocal !== undefined && typeof requestLocal !== "boolean") { + throw new TypeError("DialCache defaultConfig requestLocal must be a boolean"); + } + + assertDefaultLayerMap(ttlSecConfig, "ttlSec"); + assertDefaultLayerMap(rampConfig, "ramp"); + + const snapshot = new DialCacheKeyConfig({ + ttlSec: ttlSecConfig, + ramp: rampConfig, + ...(requestLocal === undefined ? {} : { requestLocal }), + }); + + for (const layer of [CacheLayer.LOCAL, CacheLayer.REMOTE]) { + const ttlSec = snapshot.ttlSec[layer]; + if (ttlSec !== undefined) { + if (typeof ttlSec !== "number") { + throw new TypeError(`DialCache defaultConfig ttlSec.${layer} must be a number`); + } + if (!Number.isSafeInteger(ttlSec) || ttlSec <= 0) { + throw new RangeError(`DialCache defaultConfig ttlSec.${layer} must be a positive safe integer`); + } + } + + const ramp = snapshot.ramp[layer]; + if (ramp !== undefined) { + if (typeof ramp !== "number") { + throw new TypeError(`DialCache defaultConfig ramp.${layer} must be a number`); + } + if (!Number.isFinite(ramp) || ramp < 0 || ramp > 100) { + throw new RangeError(`DialCache defaultConfig ramp.${layer} must be between 0 and 100`); + } + } + } + + Object.freeze(snapshot.ttlSec); + Object.freeze(snapshot.ramp); + return Object.freeze(snapshot); +} + +function assertDefaultLayerMap(config: unknown, name: "ttlSec" | "ramp"): void { + if (config === null || typeof config !== "object" || Array.isArray(config)) { + throw new TypeError(`DialCache defaultConfig ${name} must be a layer map`); + } +} + function resolveFallbackTimeoutMs(value: number | null | undefined): number | null { if (value === null) { return null; diff --git a/src/errors.ts b/src/errors.ts index ed748b7..6153c2a 100644 --- a/src/errors.ts +++ b/src/errors.ts @@ -27,9 +27,3 @@ export class UseCaseNameIsReservedError extends DialCacheError { super(`Use case name is reserved: ${useCase}`); } } - -export class MissingKeyConfigError extends DialCacheError { - constructor(useCase: string) { - super(`Missing key config for use case: ${useCase}`); - } -} diff --git a/src/index.ts b/src/index.ts index f66473b..38d8726 100644 --- a/src/index.ts +++ b/src/index.ts @@ -17,7 +17,6 @@ export type { export { DialCacheError, FallbackTimeoutError, - MissingKeyConfigError, UseCaseIsAlreadyRegisteredError, UseCaseNameIsReservedError, } from "./errors.js"; diff --git a/src/internal/runtime-config.ts b/src/internal/runtime-config.ts index 6bfbd66..e68dc40 100644 --- a/src/internal/runtime-config.ts +++ b/src/internal/runtime-config.ts @@ -1,4 +1,10 @@ -import { CacheLayer, type CacheConfigProvider, type CacheRampSampler, type DialCacheKeyConfig } from "../config.js"; +import { + CacheLayer, + DialCacheKeyConfig, + type CacheConfigProvider, + type CacheRampSampler, + type LayerConfig, +} from "../config.js"; import type { DialCacheKey } from "../key.js"; import type { DisabledReason } from "../metrics.js"; @@ -22,7 +28,12 @@ export async function fetchKeyConfig( configProvider: CacheConfigProvider, key: DialCacheKey, ): Promise { - return (await configProvider(key)) ?? key.defaultConfig; + const defaultConfig = key.defaultConfig; + const runtimeConfig = (await configProvider(key)) as DialCacheKeyConfig | null | undefined; + if (runtimeConfig === null || runtimeConfig === undefined) { + return defaultConfig; + } + return mergeKeyConfig(defaultConfig, runtimeConfig); } export async function resolveLayerConfig(options: ResolveLayerConfigOptions): Promise { @@ -33,23 +44,21 @@ export async function resolveLayerConfig(options: ResolveLayerConfigOptions): Pr export async function resolveLayerConfigResult(options: ResolveLayerConfigOptions): Promise { const config = options.config; if (config === null) { - return { status: "disabled", reason: "missing_config" }; + return { status: "disabled", reason: "policy_disabled" }; } const ttlSec = config.ttlSec[options.layer]; if (ttlSec === undefined) { - return { status: "disabled", reason: "missing_config" }; + return { status: "disabled", reason: "policy_disabled" }; } if (!Number.isSafeInteger(ttlSec) || ttlSec <= 0) { return { status: "disabled", reason: "invalid_ttl" }; } - const configuredRamp = config.ramp[options.layer]; - if (configuredRamp === undefined) { - return { status: "disabled", reason: "missing_config" }; - } + const configuredRampValue = config.ramp[options.layer]; + const configuredRamp = configuredRampValue === undefined ? 100 : configuredRampValue; if (!Number.isFinite(configuredRamp)) { - return { status: "disabled", reason: "ramped_down" }; + return { status: "disabled", reason: "invalid_ramp" }; } const ramp = clampPercentage(configuredRamp); @@ -70,6 +79,59 @@ export async function resolveLayerConfigResult(options: ResolveLayerConfigOption : { status: "disabled", reason: "ramped_down" }; } +function mergeKeyConfig( + defaultConfig: DialCacheKeyConfig | null, + runtimeConfig: DialCacheKeyConfig | null | undefined, +): DialCacheKeyConfig { + const overlay = runtimeConfig ?? undefined; + assertKeyConfig(defaultConfig); + assertKeyConfig(overlay); + const defaultRequestLocal = defaultConfig?.requestLocal; + const overlayRequestLocal = overlay?.requestLocal; + const requestLocal = overlayRequestLocal !== undefined + ? overlayRequestLocal + : defaultRequestLocal !== undefined + ? defaultRequestLocal + : false; + + return new DialCacheKeyConfig({ + ttlSec: mergeLayerConfig(defaultConfig?.ttlSec, overlay?.ttlSec, "ttlSec"), + ramp: mergeLayerConfig(defaultConfig?.ramp, overlay?.ramp, "ramp"), + requestLocal, + }); +} + +function assertKeyConfig(config: DialCacheKeyConfig | null | undefined): void { + if (config !== null && config !== undefined && (typeof config !== "object" || Array.isArray(config))) { + throw new TypeError("DialCache key config must be an object"); + } +} + +function mergeLayerConfig( + defaults: LayerConfig | undefined, + overlay: LayerConfig | undefined, + name: "ttlSec" | "ramp", +): LayerConfig { + assertLayerConfig(defaults, name); + assertLayerConfig(overlay, name); + + const merged: LayerConfig = {}; + for (const layer of [CacheLayer.LOCAL, CacheLayer.REMOTE]) { + const overlayValue = overlay?.[layer]; + const value = overlayValue !== undefined ? overlayValue : defaults?.[layer]; + if (value !== undefined) { + merged[layer] = value; + } + } + return merged; +} + +function assertLayerConfig(config: LayerConfig | undefined, name: "ttlSec" | "ramp"): void { + if (config !== undefined && (config === null || typeof config !== "object" || Array.isArray(config))) { + throw new TypeError(`DialCache ${name} config must be a layer map`); + } +} + function clampPercentage(value: number): number { if (value <= 0) { return 0; diff --git a/src/metrics.ts b/src/metrics.ts index ed4796e..df6de99 100644 --- a/src/metrics.ts +++ b/src/metrics.ts @@ -8,7 +8,8 @@ type NoCacheLayer = typeof NO_CACHE_LAYER; type RequestLocalCacheLayer = typeof REQUEST_LOCAL_CACHE_LAYER; export type MetricLayer = CacheLayer | RequestLocalCacheLayer | NoCacheLayer; export type CoalescingScope = "request_local" | "process"; -export type DisabledReason = "context" | "missing_config" | "invalid_ttl" | "ramped_down" | "config_error"; +/** Bounded reasons for skipping cache work; policy_disabled means a shared layer has no effective TTL. */ +export type DisabledReason = "context" | "policy_disabled" | "invalid_ttl" | "invalid_ramp" | "ramped_down" | "config_error"; /** Stable failure sites used instead of backend- or application-defined error names. */ export type MetricErrorKind = | "key_construction" diff --git a/test/datadog.test.ts b/test/datadog.test.ts index 09323b8..9a6368a 100644 --- a/test/datadog.test.ts +++ b/test/datadog.test.ts @@ -75,13 +75,15 @@ const remoteOnly = () => }); const observationMetricTypes: readonly DatadogObservationMetricType[] = ["histogram", "distribution"]; -const disabledReasons: readonly DisabledReason[] = [ - "context", - "missing_config", - "invalid_ttl", - "ramped_down", - "config_error", -]; +const DISABLED_REASONS: Readonly> = { + context: true, + policy_disabled: true, + invalid_ttl: true, + invalid_ramp: true, + ramped_down: true, + config_error: true, +}; +const disabledReasons = Object.keys(DISABLED_REASONS) as DisabledReason[]; const errorKinds: readonly MetricErrorKind[] = [ "key_construction", "config_resolution", diff --git a/test/dialcache-config-ramp.test.ts b/test/dialcache-config-ramp.test.ts index 4977d10..e24d992 100644 --- a/test/dialcache-config-ramp.test.ts +++ b/test/dialcache-config-ramp.test.ts @@ -6,6 +6,7 @@ import { DialCacheKey, DialCacheKeyConfig, deterministicRampSampler, + type LayerConfig, } from "../src/index.js"; import { FakeRedis } from "./fake-redis.js"; @@ -13,6 +14,49 @@ const configFor = (ttlSec: Partial>, ramp: Partial { + it("preserves requestLocal omission until baseline and runtime config are merged", () => { + expect(new DialCacheKeyConfig({}).requestLocal).toBeUndefined(); + expect(new DialCacheKeyConfig({ requestLocal: false }).requestLocal).toBe(false); + }); + + it("captures an immutable default policy snapshot when the use case is registered", async () => { + const suppliedDefault = new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + ramp: { [CacheLayer.LOCAL]: 100 }, + }); + const observedDefaults: Array = []; + const dialcache = new DialCache({ + cacheConfigProvider: (key) => { + observedDefaults.push(key.defaultConfig); + (key as unknown as { defaultConfig: DialCacheKeyConfig | null }).defaultConfig = null; + return null; + }, + }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: "ImmutableRegisteredDefault", + cacheKey: (userId) => userId, + defaultConfig: suppliedDefault, + }); + + suppliedDefault.ttlSec[CacheLayer.LOCAL] = 0; + suppliedDefault.ramp[CacheLayer.LOCAL] = 0; + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(second).toBe(first); + expect(calls).toBe(1); + expect(observedDefaults).toHaveLength(2); + expect(observedDefaults[0]).not.toBe(suppliedDefault); + expect(observedDefaults[1]).toBe(observedDefaults[0]); + expect(observedDefaults[0]?.ttlSec[CacheLayer.LOCAL]).toBe(60); + expect(observedDefaults[0]?.ramp[CacheLayer.LOCAL]).toBe(100); + expect(Object.isFrozen(observedDefaults[0])).toBe(true); + expect(Object.isFrozen(observedDefaults[0]?.ttlSec)).toBe(true); + expect(Object.isFrozen(observedDefaults[0]?.ramp)).toBe(true); + }); + it("enables request-local caching without TTL, ramp, or ramp sampling", async () => { const rampSampler = vi.fn(() => { throw new Error("request-local caching must not use the layer ramp sampler"); @@ -246,6 +290,258 @@ describe("DialCache runtime config and ramp controls", () => { expect(cacheConfigProvider).toHaveBeenCalled(); }); + it("inherits defaults per field when the runtime config is sparse", async () => { + // Given the static baseline enables local and remote caching with distinct policies, + // while runtime config changes only the local ramp and remote TTL. + const redis = new FakeRedis(); + const dialcache = new DialCache({ + redis: { client: redis }, + cacheConfigProvider: async () => new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.REMOTE]: 30 }, + ramp: { [CacheLayer.LOCAL]: 0 }, + }), + }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: "SparseRuntimeOverlay", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60, [CacheLayer.REMOTE]: 120 }, + ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + }), + }); + + // When the same key is read twice, the explicit local ramp disables local + // caching while the remote TTL override inherits the baseline remote ramp. + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(first).toEqual({ userId: "123", calls: 1 }); + expect(second).toEqual({ userId: "123", calls: 1 }); + expect(calls).toBe(1); + expect(redis.getCalls).toBe(2); + expect(redis.setCalls).toBe(1); + }); + + it.each([ + ["null", null], + ["undefined", undefined], + ["an empty config", new DialCacheKeyConfig({})], + [ + "undefined leaves", + new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: undefined } as unknown as LayerConfig, + ramp: { [CacheLayer.LOCAL]: undefined } as unknown as LayerConfig, + }), + ], + ] as const)("inherits the full baseline when the provider returns %s", async (_name, runtimeConfig) => { + const cacheConfigProvider = vi.fn(async () => runtimeConfig as DialCacheKeyConfig | null); + const dialcache = new DialCache({ cacheConfigProvider }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: `WholeBaselineInheritance${String(_name)}`, + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + }), + }); + + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(first).toEqual({ userId: "123", calls: 1 }); + expect(second).toEqual({ userId: "123", calls: 1 }); + expect(cacheConfigProvider).toHaveBeenCalledTimes(2); + }); + + it("defaults a shared layer ramp to 100 when its effective TTL is configured", async () => { + const rampSampler = vi.fn(() => { + throw new Error("the default 100 ramp must not sample"); + }); + const dialcache = new DialCache({ rampSampler }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: "ImplicitFullRamp", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 60 } }), + }); + + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(second).toBe(first); + expect(calls).toBe(1); + expect(rampSampler).not.toHaveBeenCalled(); + }); + + it.each([ + ["a null config", () => new DialCacheKeyConfig(null as never), "DialCache key config must be an object"], + [ + "a null layer map", + () => new DialCacheKeyConfig({ ttlSec: null as unknown as LayerConfig }), + "DialCache ttlSec config must be a layer map", + ], + [ + "a non-boolean requestLocal value", + () => new DialCacheKeyConfig({ requestLocal: null as unknown as boolean }), + "DialCache requestLocal config must be a boolean", + ], + ])("rejects $0 in the public config constructor", (_name, construct, message) => { + expect(construct).toThrow(message); + }); + + it.each([ + ["zero TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 0 } }), RangeError, "positive safe integer"], + ["negative TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: -1 } }), RangeError, "positive safe integer"], + ["fractional TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 0.5 } }), RangeError, "positive safe integer"], + ["non-finite TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: Number.NaN } }), RangeError, "positive safe integer"], + ["negative ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: -1 } }), RangeError, "between 0 and 100"], + ["over-100 ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: 101 } }), RangeError, "between 0 and 100"], + ["non-finite ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: Number.POSITIVE_INFINITY } }), RangeError, "between 0 and 100"], + [ + "wrong-type TTL", + new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: "60" as unknown as number } }), + TypeError, + "must be a number", + ], + [ + "wrong-type ramp", + new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: null as unknown as number } }), + TypeError, + "must be a number", + ], + ["primitive config", 42 as unknown as DialCacheKeyConfig, TypeError, "must be an object"], + ["array config", [] as unknown as DialCacheKeyConfig, TypeError, "must be an object"], + [ + "null TTL map", + { ttlSec: null, ramp: {} } as unknown as DialCacheKeyConfig, + TypeError, + "must be a layer map", + ], + ])("rejects a malformed static defaultConfig with $0 at registration", (_name, defaultConfig, ErrorType, message) => { + const dialcache = new DialCache(); + + expect(() => dialcache.cached(async () => "value", { + keyType: "item_id", + useCase: "InvalidStaticPolicy", + cacheKey: () => "123", + defaultConfig, + })).toThrow(ErrorType); + expect(() => dialcache.cached(async () => "value", { + keyType: "item_id", + useCase: "InvalidStaticPolicy", + cacheKey: () => "123", + defaultConfig, + })).toThrow(message); + + // Validation runs before use-case registration, so a corrected policy can + // reuse the same name instead of being rejected as a duplicate. + expect(() => dialcache.cached(async () => "value", { + keyType: "item_id", + useCase: "InvalidStaticPolicy", + cacheKey: () => "123", + defaultConfig: new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 60 } }), + })).not.toThrow(); + }); + + it.each([ + ["a primitive", 42], + ["an array", []], + ["a null layer map", { ttlSec: null, ramp: {} }], + ["a null requestLocal value", { ttlSec: {}, ramp: {}, requestLocal: null }], + ] as const)("fails open instead of inheriting defaults when the provider returns %s", async (_name, runtimeConfig) => { + const cacheConfigProvider = vi.fn(async () => runtimeConfig as unknown as DialCacheKeyConfig); + const dialcache = new DialCache({ cacheConfigProvider }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: `MalformedRuntimeConfig${String(_name)}`, + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 60 } }), + }); + + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(first.calls).toBe(1); + expect(second.calls).toBe(2); + expect(cacheConfigProvider).toHaveBeenCalledTimes(2); + }); + + it("returns the explicit kill-switch overlay from DialCacheKeyConfig.disabled()", () => { + expect(DialCacheKeyConfig.disabled()).toEqual(new DialCacheKeyConfig({ + requestLocal: false, + ramp: { [CacheLayer.LOCAL]: 0, [CacheLayer.REMOTE]: 0 }, + })); + }); + + it.each([ + [ + "explicit runtime field values", + () => new DialCacheKeyConfig({ + requestLocal: false, + ramp: { [CacheLayer.LOCAL]: 0, [CacheLayer.REMOTE]: 0 }, + }), + ], + ["the DialCacheKeyConfig.disabled() kill switch", () => DialCacheKeyConfig.disabled()], + ])("uses %s to disable every inherited layer", async (_name, overlayFor) => { + const redis = new FakeRedis(); + const dialcache = new DialCache({ + redis: { client: redis }, + cacheConfigProvider: async () => overlayFor(), + }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: "ExplicitDisableAll", + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + requestLocal: true, + ttlSec: { [CacheLayer.LOCAL]: 60, [CacheLayer.REMOTE]: 60 }, + ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + }), + }); + + const values = await dialcache.enable(async () => [await getUser("123"), await getUser("123")] as const); + + expect(values).toEqual([ + { userId: "123", calls: 1 }, + { userId: "123", calls: 2 }, + ]); + expect(redis.getCalls).toBe(0); + expect(redis.setCalls).toBe(0); + }); + + it.each([ + ["null TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: null as unknown as number } })], + ["NaN TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: Number.NaN } })], + ["wrong-type TTL", new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: "60" as unknown as number } })], + ["null ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: null as unknown as number } })], + ["NaN ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: Number.NaN } })], + ["wrong-type ramp", new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: "50" as unknown as number } })], + ])("does not inherit a valid default over an explicit malformed runtime $0", async (_name, runtimeConfig) => { + const dialcache = new DialCache({ cacheConfigProvider: async () => runtimeConfig }); + let calls = 0; + const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { + keyType: "user_id", + useCase: `MalformedRuntimeLeaf${String(_name)}`, + cacheKey: (userId) => userId, + defaultConfig: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + ramp: { [CacheLayer.LOCAL]: 100 }, + }), + }); + + const first = await dialcache.enable(async () => await getUser("123")); + const second = await dialcache.enable(async () => await getUser("123")); + + expect(first.calls).toBe(1); + expect(second.calls).toBe(2); + }); + it("applies runtime config changes to subsequent calls", async () => { // Given a provider whose config can change without redeploying the cached function. let runtimeConfig: DialCacheKeyConfig | null = DialCacheKeyConfig.enabled(60); @@ -260,7 +556,7 @@ describe("DialCache runtime config and ramp controls", () => { // When the provider disables local caching after the first cached read. const first = await dialcache.enable(async () => await getUser("123")); - runtimeConfig = configFor({}, {}); + runtimeConfig = new DialCacheKeyConfig({ ramp: { [CacheLayer.LOCAL]: 0 } }); const second = await dialcache.enable(async () => await getUser("123")); // Then the second call honors the new disabled config instead of returning the existing local entry. @@ -393,27 +689,36 @@ describe("DialCache runtime config and ramp controls", () => { const deterministicSampler = vi.fn(() => { throw new Error("clamped terminal ramps should not need random sampling"); }); - const clampedCache = new DialCache({ rampSampler: deterministicSampler }); + const clampedCache = new DialCache({ + rampSampler: deterministicSampler, + cacheConfigProvider: async (key) => configFor( + { [CacheLayer.LOCAL]: 60 }, + { + [CacheLayer.LOCAL]: key.useCase === "NegativeConfiguredRamp" + ? -10 + : key.useCase === "OverHundredConfiguredRamp" + ? 150 + : Number.NaN, + }, + ), + }); let negativeCalls = 0; const negativeRamp = clampedCache.cached(async (userId: string) => ({ userId, calls: ++negativeCalls }), { keyType: "user_id", useCase: "NegativeConfiguredRamp", cacheKey: (userId) => userId, - defaultConfig: configFor({ [CacheLayer.LOCAL]: 60 }, { [CacheLayer.LOCAL]: -10 }), }); let overHundredCalls = 0; const overHundredRamp = clampedCache.cached(async (userId: string) => ({ userId, calls: ++overHundredCalls }), { keyType: "user_id", useCase: "OverHundredConfiguredRamp", cacheKey: (userId) => userId, - defaultConfig: configFor({ [CacheLayer.LOCAL]: 60 }, { [CacheLayer.LOCAL]: 150 }), }); let nanConfiguredCalls = 0; const nanConfiguredRamp = clampedCache.cached(async (userId: string) => ({ userId, calls: ++nanConfiguredCalls }), { keyType: "user_id", useCase: "NanConfiguredRamp", cacheKey: (userId) => userId, - defaultConfig: configFor({ [CacheLayer.LOCAL]: 60 }, { [CacheLayer.LOCAL]: Number.NaN }), }); // When the same keys are read twice. @@ -523,17 +828,22 @@ describe("DialCache runtime config and ramp controls", () => { it("treats non-finite and fractional TTLs as invalid config", async () => { // Given invalid TTL values are configured for local and remote layers. const redis = new FakeRedis(); - const dialcache = new DialCache({ redis: { client: redis } }); const badTtls = [Number.NaN, Number.POSITIVE_INFINITY, 0.5]; // When each cached function is called twice. for (const ttl of badTtls) { + const dialcache = new DialCache({ + redis: { client: redis }, + cacheConfigProvider: async () => configFor( + { [CacheLayer.LOCAL]: ttl, [CacheLayer.REMOTE]: ttl }, + { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + ), + }); let calls = 0; const getUser = dialcache.cached(async (userId: string) => ({ userId, ttl: String(ttl), calls: ++calls }), { keyType: "user_id", useCase: `InvalidTtl${String(ttl)}`, cacheKey: (userId) => userId, - defaultConfig: configFor({ [CacheLayer.LOCAL]: ttl, [CacheLayer.REMOTE]: ttl }, { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }), }); const first = await dialcache.enable(async () => await getUser("123")); @@ -553,7 +863,7 @@ describe("DialCache runtime config and ramp controls", () => { const redis = new FakeRedis(); const dialcache = new DialCache({ redis: { client: redis }, - cacheConfigProvider: async () => configFor({ [CacheLayer.REMOTE]: 60 }, { [CacheLayer.REMOTE]: 100 }), + cacheConfigProvider: async () => new DialCacheKeyConfig({ ttlSec: { [CacheLayer.REMOTE]: 60 } }), }); let calls = 0; const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { diff --git a/test/dialcache-local.test.ts b/test/dialcache-local.test.ts index 823dfea..ccf8ce3 100644 --- a/test/dialcache-local.test.ts +++ b/test/dialcache-local.test.ts @@ -633,19 +633,22 @@ describe("DialCache local-only MVP", () => { }); }); - it("treats non-positive local ttl as disabled local config", async () => { - // Given a cached function with an invalid local TTL. + it("treats a non-positive runtime local TTL as disabled config", async () => { + // Given a valid static baseline and an invalid dynamic local TTL override. const logger = { debug: vi.fn(), warn: vi.fn(), error: vi.fn() }; - const dialcache = new DialCache({ logger }); + const dialcache = new DialCache({ + logger, + cacheConfigProvider: () => new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 0 }, + ramp: { [CacheLayer.LOCAL]: 100 }, + }), + }); let calls = 0; const getUser = dialcache.cached(async (userId: string) => ({ userId, calls: ++calls }), { keyType: "user_id", useCase: "InvalidLocalTtl", cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 0 }, - ramp: { [CacheLayer.LOCAL]: 100 }, - }), + defaultConfig: DialCacheKeyConfig.enabled(60), }); // When the function is called in an enabled scope. diff --git a/test/dialcache-metrics.test.ts b/test/dialcache-metrics.test.ts index 117fe4b..93d394a 100644 --- a/test/dialcache-metrics.test.ts +++ b/test/dialcache-metrics.test.ts @@ -269,23 +269,42 @@ describe("DialCache observability metrics", () => { it("classifies disabled cache skips by reason", async () => { // Given one cache call is outside context and other enabled calls have disabled layer config. const metrics = new RecordingMetrics(); - const dialcache = new DialCache({ metrics }); + const invalidRuntimeTtl = localOnly(0); + const invalidRuntimeRamp = new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.LOCAL]: 60 }, + ramp: { [CacheLayer.LOCAL]: Number.NaN }, + }); + const dialcache = new DialCache({ + metrics, + cacheConfigProvider: (key) => + key.useCase === "DisabledByInvalidTtl" + ? invalidRuntimeTtl + : key.useCase === "DisabledByInvalidRamp" + ? invalidRuntimeRamp + : null, + }); const contextDisabled = dialcache.cached(async (userId: string) => userId, { keyType: "user_id", useCase: "DisabledByContext", cacheKey: (userId) => userId, defaultConfig: localOnly(), }); - const missingConfig = dialcache.cached(async (userId: string) => userId, { + const policyDisabled = dialcache.cached(async (userId: string) => userId, { keyType: "user_id", - useCase: "DisabledByMissingConfig", + useCase: "DisabledByPolicy", cacheKey: (userId) => userId, }); const invalidTtl = dialcache.cached(async (userId: string) => userId, { keyType: "user_id", useCase: "DisabledByInvalidTtl", cacheKey: (userId) => userId, - defaultConfig: localOnly(0), + defaultConfig: localOnly(), + }); + const invalidRamp = dialcache.cached(async (userId: string) => userId, { + keyType: "user_id", + useCase: "DisabledByInvalidRamp", + cacheKey: (userId) => userId, + defaultConfig: localOnly(), }); const rampedDown = dialcache.cached(async (userId: string) => userId, { keyType: "user_id", @@ -300,18 +319,31 @@ describe("DialCache observability metrics", () => { // When each path is called. await contextDisabled("123"); await dialcache.enable(async () => { - await missingConfig("123"); + await policyDisabled("123"); await invalidTtl("123"); + await invalidRamp("123"); await rampedDown("123"); }); // Then disabled metrics preserve the operational reason labels. expect(events(metrics, "disabled", { useCase: "DisabledByContext", layer: "noop", reason: "context" })).toHaveLength(1); expect( - events(metrics, "disabled", { useCase: "DisabledByMissingConfig", layer: CacheLayer.LOCAL, reason: "missing_config" }), + events(metrics, "disabled", { useCase: "DisabledByPolicy", layer: CacheLayer.LOCAL, reason: "policy_disabled" }), ).toHaveLength(1); expect(events(metrics, "disabled", { useCase: "DisabledByInvalidTtl", layer: CacheLayer.LOCAL, reason: "invalid_ttl" })).toHaveLength(1); + expect(events(metrics, "disabled", { useCase: "DisabledByInvalidRamp", layer: CacheLayer.LOCAL, reason: "invalid_ramp" })).toHaveLength(1); expect(events(metrics, "disabled", { useCase: "DisabledByRamp", layer: CacheLayer.LOCAL, reason: "ramped_down" })).toHaveLength(1); + + // And invalid runtime leaves count as config_resolution errors, while + // intentional ramp-downs and absent policy do not. + expect( + events(metrics, "error", { useCase: "DisabledByInvalidTtl", layer: CacheLayer.LOCAL, error: "config_resolution", inFallback: false }), + ).toHaveLength(1); + expect( + events(metrics, "error", { useCase: "DisabledByInvalidRamp", layer: CacheLayer.LOCAL, error: "config_resolution", inFallback: false }), + ).toHaveLength(1); + expect(events(metrics, "error", { useCase: "DisabledByRamp" })).toHaveLength(0); + expect(events(metrics, "error", { useCase: "DisabledByPolicy" })).toHaveLength(0); }); it("labels cache errors separately from fallback errors", async () => { @@ -562,6 +594,20 @@ describe("DialCache observability metrics", () => { inFallback: false, }), ).toHaveLength(1); + expect( + events(metrics, "disabled", { + useCase: "ProviderConfigErrorClassification", + layer: "noop", + reason: "config_error", + }), + ).toHaveLength(1); + expect( + events(metrics, "disabled", { + useCase: "RemoteConfigErrorClassification", + layer: CacheLayer.REMOTE, + reason: "config_error", + }), + ).toHaveLength(1); expect(JSON.stringify(events(metrics, "error", {}))).not.toMatch( /Tenant123ConfigProviderError|Tenant456RemoteRampError|tenant-123|tenant-456/, ); diff --git a/test/dialcache-observability-internals.test.ts b/test/dialcache-observability-internals.test.ts index 035d5f4..4beb703 100644 --- a/test/dialcache-observability-internals.test.ts +++ b/test/dialcache-observability-internals.test.ts @@ -3,13 +3,50 @@ import { describe, expect, it, vi } from "vitest"; import { CacheLayer, DialCacheKey, DialCacheKeyConfig } from "../src/index.js"; import { LocalCache } from "../src/internal/local-cache.js"; import { RedisCache } from "../src/internal/redis-cache.js"; -import { resolveLayerConfig } from "../src/internal/runtime-config.js"; +import { fetchKeyConfig, resolveLayerConfig } from "../src/internal/runtime-config.js"; import { encodeFrame, FakeRedis } from "./fake-redis.js"; const key = (defaultConfig: DialCacheKeyConfig | null = DialCacheKeyConfig.enabled(60)) => new DialCacheKey({ keyType: "user_id", id: "123", useCase: "ObservabilityInternals", defaultConfig }); describe("DialCache observability internal compatibility paths", () => { + it("merges every runtime policy leaf independently", async () => { + const defaultConfig = new DialCacheKeyConfig({ + requestLocal: true, + ttlSec: { [CacheLayer.LOCAL]: 60, [CacheLayer.REMOTE]: 120 }, + ramp: { [CacheLayer.LOCAL]: 25, [CacheLayer.REMOTE]: 50 }, + }); + const cases = [ + { + runtime: new DialCacheKeyConfig({ + requestLocal: false, + ttlSec: { [CacheLayer.LOCAL]: 30 }, + ramp: { [CacheLayer.REMOTE]: 75 }, + }), + expected: new DialCacheKeyConfig({ + requestLocal: false, + ttlSec: { [CacheLayer.LOCAL]: 30, [CacheLayer.REMOTE]: 120 }, + ramp: { [CacheLayer.LOCAL]: 25, [CacheLayer.REMOTE]: 75 }, + }), + }, + { + runtime: new DialCacheKeyConfig({ + ttlSec: { [CacheLayer.REMOTE]: 90 }, + ramp: { [CacheLayer.LOCAL]: 10 }, + }), + expected: new DialCacheKeyConfig({ + requestLocal: true, + ttlSec: { [CacheLayer.LOCAL]: 60, [CacheLayer.REMOTE]: 90 }, + ramp: { [CacheLayer.LOCAL]: 10, [CacheLayer.REMOTE]: 50 }, + }), + }, + ]; + + for (const { runtime, expected } of cases) { + await expect(fetchKeyConfig(async () => runtime, key(defaultConfig))).resolves.toEqual(expected); + } + }); + it("keeps LocalCache get/getIfPresent compatibility while exposing disabled reads", async () => { // Given a local cache with enabled config and a second key with no config. const cache = new LocalCache(async () => null, () => 0, 10); @@ -25,7 +62,7 @@ describe("DialCache observability internal compatibility paths", () => { // Then compatibility helpers still behave like the pre-metrics API, and disabled state is explicit. expect(first).toEqual({ calls: 1 }); expect(hit).toEqual({ calls: 1 }); - expect(disabled).toEqual({ status: "disabled", reason: "missing_config" }); + expect(disabled).toEqual({ status: "disabled", reason: "policy_disabled" }); expect(calls).toBe(1); }); @@ -60,7 +97,7 @@ describe("DialCache observability internal compatibility paths", () => { }); it("preserves runtime-config edge behavior used by metrics", async () => { - // Given configs for missing ramp and non-finite ramp samples. + // Given configs for an omitted ramp and non-finite ramp samples. const missingRamp = new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 60 }, ramp: {} }); const partialRamp = new DialCacheKeyConfig({ ttlSec: { [CacheLayer.LOCAL]: 60 }, @@ -87,9 +124,9 @@ describe("DialCache observability internal compatibility paths", () => { rampSampler: () => Number.NaN, }); - // Then disabled configuration paths remain explicit. + // Then absent policy stays disabled, an omitted ramp defaults to 100%, and invalid samples stay disabled. expect(noConfig).toBeNull(); - expect(noRamp).toBeNull(); + expect(noRamp).toEqual({ ttlSec: 60, ramp: 100 }); expect(nonFiniteSample).toBeNull(); }); }); diff --git a/test/dialcache-redis.test.ts b/test/dialcache-redis.test.ts index 3eda082..5ef0e74 100644 --- a/test/dialcache-redis.test.ts +++ b/test/dialcache-redis.test.ts @@ -183,8 +183,8 @@ describe("DialCache Redis TTL layer", () => { useCase: "RedisFailOpen", cacheKey: (userId) => userId, defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 0, [CacheLayer.REMOTE]: 60 }, - ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + ttlSec: { [CacheLayer.REMOTE]: 60 }, + ramp: { [CacheLayer.REMOTE]: 100 }, }), }); @@ -354,8 +354,8 @@ describe("DialCache Redis TTL layer", () => { cacheKey: (userId) => userId, trackForInvalidation: true, defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 0, [CacheLayer.REMOTE]: 60 }, - ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + ttlSec: { [CacheLayer.REMOTE]: 60 }, + ramp: { [CacheLayer.REMOTE]: 100 }, }), serializer, }); @@ -435,8 +435,8 @@ describe("DialCache Redis TTL layer", () => { cacheKey: (userId) => userId, trackForInvalidation: true, defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 0, [CacheLayer.REMOTE]: 60 }, - ramp: { [CacheLayer.LOCAL]: 100, [CacheLayer.REMOTE]: 100 }, + ttlSec: { [CacheLayer.REMOTE]: 60 }, + ramp: { [CacheLayer.REMOTE]: 100 }, }), }); @@ -517,10 +517,7 @@ describe("DialCache Redis TTL layer", () => { keyType: "user_id", useCase: "RedisMissingRemoteTtl", cacheKey: (userId) => userId, - defaultConfig: new DialCacheKeyConfig({ - ttlSec: { [CacheLayer.LOCAL]: 0 }, - ramp: { [CacheLayer.LOCAL]: 100 }, - }), + defaultConfig: new DialCacheKeyConfig({}), }); // When a cache read is attempted. diff --git a/test/dialcache-request-local.test.ts b/test/dialcache-request-local.test.ts index b3e5f3d..4a671e2 100644 --- a/test/dialcache-request-local.test.ts +++ b/test/dialcache-request-local.test.ts @@ -103,8 +103,8 @@ describe("DialCache request-local cache", () => { expect(calls).toBe(2); }); - it("reads runtime config once per invocation and preserves scoped values across a true-false-true sequence", async () => { - const configs = [requestLocalConfig(), requestLocalConfig(false), requestLocalConfig()]; + it("inherits request-local defaults unless the runtime overlay explicitly disables them", async () => { + const configs = [new DialCacheKeyConfig({}), requestLocalConfig(false), new DialCacheKeyConfig({})]; const cacheConfigProvider = vi.fn(async () => configs.shift() ?? null); const dialcache = new DialCache({ cacheConfigProvider }); let calls = 0; @@ -112,6 +112,7 @@ describe("DialCache request-local cache", () => { keyType: "user_id", useCase: "RuntimeRequestLocalToggle", cacheKey: (id) => id, + defaultConfig: requestLocalConfig(), }); const result = await dialcache.enable(async () => { diff --git a/test/prometheus.test.ts b/test/prometheus.test.ts index 230b3b9..4688301 100644 --- a/test/prometheus.test.ts +++ b/test/prometheus.test.ts @@ -8,7 +8,13 @@ import { } from "prom-client"; import { describe, expect, it } from "vitest"; -import { CacheLayer, DialCache, DialCacheKeyConfig, type MetricErrorKind } from "../src/index.js"; +import { + CacheLayer, + DialCache, + DialCacheKeyConfig, + type DisabledReason, + type MetricErrorKind, +} from "../src/index.js"; import { PrometheusDialCacheMetrics, createPrometheusDialCacheMetrics } from "../src/prometheus.js"; import { FakeRedis } from "./fake-redis.js"; @@ -40,6 +46,14 @@ const METRIC_ERROR_KINDS: Readonly> = { fallback: true, unknown: true, }; +const DISABLED_REASONS: Readonly> = { + context: true, + policy_disabled: true, + invalid_ttl: true, + invalid_ramp: true, + ramped_down: true, + config_error: true, +}; interface IncompatibleCollectorCase { readonly schemaPart: string; @@ -217,6 +231,34 @@ describe("Prometheus metrics adapter", () => { } }); + it("exports every bounded disabled reason without rewriting labels", async () => { + const registry = new Registry(); + const metrics = new PrometheusDialCacheMetrics({ registry, prefix: "disabled_reason_" }); + const labels = { + cacheNamespace: "users", + useCase: "PrometheusDisabledReasons", + keyType: "user_id", + layer: CacheLayer.LOCAL, + } as const; + + const disabledReasons = Object.keys(DISABLED_REASONS) as DisabledReason[]; + for (const reason of disabledReasons) { + metrics.disabled({ ...labels, reason }); + } + + for (const reason of disabledReasons) { + await expect( + sumMetric(registry, "disabled_reason_dialcache_disabled_counter", { + cache_namespace: labels.cacheNamespace, + use_case: labels.useCase, + key_type: labels.keyType, + layer: labels.layer, + reason, + }), + ).resolves.toBe(1); + } + }); + it("reuses existing collectors when multiple adapters share a registry", async () => { const registry = new Registry(); const firstCache = new DialCache({ metrics: createPrometheusDialCacheMetrics({ registry }) });