Add entropy-based detection for unknown-prefix high-entropy secrets

[fahadsiddiqui]

Problem

Detection today is pure pattern-matching against known vendor prefixes (sk-, xox-, etc.). A random 40-character hex or base64 blob with no recognizable prefix — e.g. a raw API key from a vendor not in the ruleset, or a generic bearer token — sails through completely undetected. Competing tools (gitleaks, trufflehog) mix pattern-matching with Shannon-entropy scoring specifically to close this gap.

Fix

Add an entropy-based detection layer:

• Compute Shannon entropy over base64/hex-shaped token candidates in the scanned text
• Only flag high-entropy tokens that also have supporting context (nearby keyword like key, token, secret, password, or sit on the right side of an assignment) — entropy alone is too false-positive-prone (flags UUIDs, hashes, git SHAs, etc.)
• Follow the same pattern as gitleaks: pattern-or-keyword-proximity and entropy threshold, never entropy alone

Acceptance criteria

• A high-entropy unrecognized token following a keyword like token:/api_key= is detected
• Common high-entropy-but-benign strings (git SHAs, UUIDs, hashes without keyword context) are added to the FP corpus and do not match
• Entropy threshold and keyword list are easy to tune (constants, not magic numbers buried in logic)
• Performance impact is measured — entropy computation must not meaningfully affect scan time within the 10s hook timeout even on large prompts