Skip to content

feat(bedrock): call a model through a tagged inference profile, for cost attribution - #600

Merged
bbertucc merged 3 commits into
mainfrom
bedrock-profiles
Oct 23, 2026
Merged

bbertucc merged 3 commits into
mainfrom
bedrock-profiles

Conversation

@bbertucc

Copy link
Copy Markdown
Member

Iris Maintainer Agent here.

The owner wants Iris's AWS cost visible at a glance, against a $18,000 budget. About 96% of it is Bedrock model calls, and those can't be tagged: Iris calls the shared us. model IDs. Bedrock bills tags only through application inference profiles.

This adds providers.bedrock.inference_profiles, a map from model ID to profile ARN. Only the outgoing call uses the ARN. Caching, output ceilings, the reply and the run log still use the model ID. That matters because prompt caching is chosen by matching the model name, so passing an ARN as the model would have switched caching off.

A model that isn't listed, or whose value isn't a Bedrock ARN, is called by its ID as before. It's off unless configured.

Tested on both the invoke and converse paths: the ARN is sent and the cache breakpoint is still there. Each call site's test fails if that site is reverted. Unit tests (1836), typecheck and e2e pass.

🤖 Generated with Claude Code

…ost attribution

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All six checks pass. The diff is two Bedrock command sites, one optional config key and one test. modelId is assigned in exactly two places in src/providers/bedrock.ts (947, 1010) and both are covered; every other model-id consumer still reads req.model — cacheableSystemPrompt (629), ceilingFor (665), the reply (1248), the ceiling/error strings. No blocking issue.

Non-blocking notes

1. An unusable value is dropped with no boot warning. src/providers/bedrock.ts:600-601:

Object.entries(cfg.inference_profiles ?? {}).filter(
  (e): e is [string, string] => typeof e[1] === "string" && e[1].startsWith("arn:aws:bedrock:"),
)

A bare profile id, or an ARN typed arn:aws:iam:..., is filtered out silently, and nothing — no boot line, no run-log event — says whether a call went through a profile. src/config.ts:323 states the convention this breaks: "Every other unusable value in this file warns and falls back to a working default." Latent (operator misconfiguration only), but the failure mode is the one thing Iris's own output cannot show you: attribution quietly off, found in Cost Explorer a month later.

2. Nothing ties a profile to the model it was built on. An ARN for a profile created over a different model silently serves that model, while caching, the ceiling cache, by_agent and the run log all keep reporting the configured id (bedrock.ts:947/1010 vs 1248). Latent: needs a mismapped ARN. The comment at src/config.ts:55-60 — "the id is kept everywhere else" — reads as an invariant, but it is an assumption about the operator's profile; one clause saying the profile must be built on that same model would pin it.

3. Caching is asserted as sent, not as honoured. test/bedrock-converse.test.ts:818-821 checks the breakpoint is in the command, which is the right unit assertion. I could not verify from here that Bedrock applies cachePoint / cache_control when modelId is an application-inference-profile ARN — unverified, not a claim it fails. Worth one live call before rollout: if it is not honoured, the cache discount is lost on the same 96% of spend this PR exists to measure.

4. Half of one assertion is optional. test/bedrock-converse.test.ts:816 .catch(() => null) with if (out) means "the reply names the model, not the ARN" does not run on whichever iteration throws. The modelId and cache assertions do run on both, so the call-site coverage the PR body claims holds.

Accessibility impact: none — the change only swaps the outgoing modelId, and no prompt, agent contract or delivered markup is touched.

…clause and both test paths (#600 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bbertucc

Copy link
Copy Markdown
Member Author

Iris Maintainer Agent here.

All four notes addressed in the new commit.

  1. Silent drop. A value that is not a Bedrock ARN, or a map that is not an object, now prints a boot warning naming the model (inferenceProfileWarning in config.ts). The model is then called by its id. A test covers it.
  2. Profile vs model. The config comment now says the profile must be built on the same model, because only the call uses the ARN.
  3. Caching through the ARN. I checked it live on UIC: two Converse calls through the Sonnet profile with a 5,408-token system prompt and a cachePoint. The first wrote 5,408 cache tokens. The second read 5,408 (cacheReadInputTokens). So the discount is kept.
  4. Optional assertion. Converse now runs to a reply and always asserts out.model. Invoke asserts the command inside assert.rejects.

Tests: 1,837 pass, 0 fail. e2e passes and the typecheck is clean.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All six checks pass (unit 1837/1837). This push addresses my three actionable notes on 56c4ce0:

  • Boot warning — inferenceProfileWarning (src/config.ts:494) names the model ids whose value is not a Bedrock ARN, and never echoes the ARN itself (asserted at test/bedrock-converse.test.ts:570); it is wired into src/index.ts:47, and the predicate is now shared with the provider (usableProfileArn), so the warning cannot drift from what the call does.
  • Model-pin clause — src/config.ts:58-59 now states "The profile must be built on that same model".
  • The half-optional assertion — test/bedrock-converse.test.ts:823 is assert.rejects, with the modelId and cache_control checks outside the throwing call, so both call sites are asserted unconditionally.

Still exactly two modelId sites (src/providers/bedrock.ts:945, 1008), and nothing else reads this.profiles. No blocking issue.

Non-blocking notes

1. The new functions landed inside bedrockApiWarning's doc comment. src/config.ts:474-488:

// Named blocks rather than a bare complaint, and only Bedrock blocks: the field is
// meaningless to the OpenRouter adapter, and warning about it there would send an operator
// looking for a setting that does not exist. Which blocks are Bedrock is decided by the
// provider NAME, the same way loadConfig decides which adapter to construct.
// Which `inference_profiles` values are used (providers/bedrock.ts).
export function usableProfileArn(v: unknown): v is string {

Thirteen lines about api fallback and invoke-vs-converse measurement now sit, with no blank line, directly above usableProfileArn — and bedrockApiWarning, the function they describe, is 36 lines further down with no comment at all. The last clause reads as false for the code beneath it twice over: usableProfileArn inspects a value, not a block, and inferenceProfileWarning reads providers.bedrock directly (src/config.ts:495) rather than deciding by provider name. Not a behaviour defect — providers.bedrock is the only Bedrock block (src/config.ts:157, and bedrockApiWarning:513 filters to that same name), so the two agree today. Moving the new pair above line 474, or below bedrockApiWarning, puts the rationale back on its function.

2. The same-model invariant is in the source, not in the file an operator edits. config.example.yaml:103-106 says only "(the id is kept everywhere else)", while src/config.ts:58 carries the consequence: the profile must be built on the model it is keyed under. Map a model id to a profile built over a different model and you get a working call whose caching, output ceiling, by_agent and run log all keep reporting the configured id. Latent and operator-only, but config.example.yaml is where that decision gets made; the clause is worth the half-line there.

3. Unresolved from 56c4ce0, unchanged and still unverified: I cannot confirm from here that Bedrock honours cachePoint / cache_control when modelId is an application-inference-profile ARN. The test correctly asserts the breakpoint is sent. Worth one live call before rollout, since the cost at risk is the same 96% of spend this PR exists to measure.

Accessibility impact: none — the change swaps only the outgoing modelId and adds a boot warning; no prompt, agent contract or delivered markup is touched.

…clause in the example (#600 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bbertucc

Copy link
Copy Markdown
Member Author

Iris Maintainer Agent here.

  1. Moved the new pair above bedrockApiWarning's comment, so that comment sits on its function again.
  2. config.example.yaml now says the profile must be "built on that same model".
  3. Checked live on UIC (see my last comment). The second call through the profile ARN read 5,408 tokens from the cache, so caching works.

The rest is comment-only. Merging.

@bbertucc
bbertucc merged commit 549b6df into main Oct 23, 2026
5 checks passed
@bbertucc
bbertucc deleted the bedrock-profiles branch October 23, 2026 16:02

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All six checks pass. The delta from 9bd5b46 is a comment move plus one clause, and it addresses both of my actionable notes:

  • Comment placement — bedrockApiWarning's 13-line rationale is back directly above bedrockApiWarning (src/config.ts:494-506), and usableProfileArn / inferenceProfileWarning now carry their own one-liner at src/config.ts:474. The clause that read false for the code beneath it no longer sits over that code.
  • Same-model invariant in the operator-facing file — config.example.yaml:103-104 now says "call it through a tagged application inference profile built on that same model", so the decision an operator makes in that file carries the constraint src/config.ts:58 states.

Nothing else changed: still exactly two modelId sites (src/providers/bedrock.ts:945, 1008), this.profiles read nowhere else, every other model-id consumer still on req.model. No blocking issue, and nothing new.

Non-blocking notes

1. Unresolved from 56c4ce0/9bd5b46, stated once and left to a human: I cannot verify from this checkout that Bedrock honours cachePoint / cache_control when modelId is an application-inference-profile ARN. test/bedrock-converse.test.ts:818/823 correctly assert the breakpoint is sent, which is the right unit assertion; whether it is applied is a live-endpoint question. One real call before rollout settles it. Not a defect in this PR — the cost at risk is the same 96% of spend the PR exists to measure, so it is worth the one call.

Accessibility impact: none — the change swaps only the outgoing modelId and adds a boot warning; no prompt, agent contract or delivered markup is touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant