Overview
matchWeakness (app/queries/claude.ts) returns "novel" on any failure — API error, malformed response, or an out-of-range match index. Its docstring frames this as mirroring classifyMisconception's "never let a flaky model corrupt state" bias, but the two failure modes are not the same risk class:
classifyMisconception failing returns null, and runMisconceptionPipeline simply skips the write. Worst case: one missed data point.
matchWeakness failing returns "novel", which runMisconceptionPipeline treats as "no existing match, insert a new row." Worst case: a duplicate weakness row written and persisted with no reconciliation job to ever merge it back.
A Haiku timeout or rate-limit blip during a busy grading period would currently cause every in-flight wrong answer's dedup check to silently start duplicating weakness rows, rather than just skipping the (comparatively harmless) increment.
Suggested fix
Give matchWeakness a third outcome distinct from both { id } and "novel" — e.g. "unknown" — for its failure path, and have runMisconceptionPipeline treat "unknown" the same way it already treats a null from classifyMisconception: skip the write, log it, move on. This makes the actual behavior match what the docstring already claims instead of asserting parity while writing wrong data.
Notes
Overview
matchWeakness(app/queries/claude.ts) returns"novel"on any failure — API error, malformed response, or an out-of-range match index. Its docstring frames this as mirroringclassifyMisconception's "never let a flaky model corrupt state" bias, but the two failure modes are not the same risk class:classifyMisconceptionfailing returnsnull, andrunMisconceptionPipelinesimply skips the write. Worst case: one missed data point.matchWeaknessfailing returns"novel", whichrunMisconceptionPipelinetreats as "no existing match, insert a new row." Worst case: a duplicate weakness row written and persisted with no reconciliation job to ever merge it back.A Haiku timeout or rate-limit blip during a busy grading period would currently cause every in-flight wrong answer's dedup check to silently start duplicating weakness rows, rather than just skipping the (comparatively harmless) increment.
Suggested fix
Give
matchWeaknessa third outcome distinct from both{ id }and"novel"— e.g."unknown"— for its failure path, and haverunMisconceptionPipelinetreat"unknown"the same way it already treats anullfromclassifyMisconception: skip the write, log it, move on. This makes the actual behavior match what the docstring already claims instead of asserting parity while writing wrong data.Notes
feat/async-misconception-write-path).matchWeaknessfails. Fixing one doesn't fix the other.