Skip to content

Commit b28b95b

Browse files
sunnylqmclaude
andauthored
feat(hermes-base): redact and fingerprint the reported detail (#87)
The outcome column has been collecting rejections that nobody reads, and the one that finally got looked at turned out to carry a customer's property name -- the detail is whatever hermesc printed, which is the user's own code. Both problems are in the report, not in the check. redactFailureDetail replaces what can only come from user code -- quoted string operands, function names, and the paths a compiler's stderr drags in -- with stable tokens that keep each value's length and character class. The shape a fix is reasoned about (opcodes, registers, counts, literal kinds) is untouched, so a rejection is still triageable from the report alone, and the local console keeps the real text, which is where the property name actually helps. failureFingerprint groups the same defect across apps and builds by stripping registers, ids and offsets from the redacted line. It ships as one implementation on purpose: the fuzzer's dedup key is now this same function, so a finding here and the same defect in the field land in one bucket. Two implementations would make the counts fiction. Reporting stays additive -- hermesBaseFingerprint is ignored by servers that do not know it, like hermesBaseOutcome was -- and the detail is redacted exactly once, where the check ran; versions.ts only forwards it. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 51e2d32 commit b28b95b

6 files changed

Lines changed: 194 additions & 14 deletions

File tree

‎docs/hermes-base-verification.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ pretty 输出本身会截断长字符串与 BigInt、用函数名替代函数索
5959

6060
CLI(`26764b1`)在 `version/create` 附带 `hermesBaseOutcome: 'used' | 'rejected' | 'dump-failed' | 'none'` 与可选 `hermesBaseDetail`(首处差异或失败原因,≤ 500 个码点)。规则同其它链路字段:只发已知值、绝不发 JSON null、未知就省略字段(单独 `pushy publish` 一个 ppk 时没有校验结果,字段不出现)。outcome 从 `HermesCompileResult.outcome` 带出,与 `base` 分开:base 被拒时 `base` 仍为 null,但 outcome 说明是被拒而不是没找到。base 编译本身失败记为 `none` 并附 `base compile failed: …`。
6161

62+
上报前 detail 会经 `src/utils/failure-fingerprint.ts` 的 `redactFailureDetail` 脱敏:引号内的字符串操作数、`Function<…>` 的函数名、编译器 stderr 里的路径都换成 `str#<hash8>/<长度>` / `fn#<hash8>` / `path#<hash8>.<ext>`,指令形态、寄存器、计数原样保留。本地控制台仍打印未脱敏的原文——属性名在本机排查时才有用;离开这台机器的那份不该带客户代码。同时上报 `hermesBaseFingerprint`(脱敏后再抹掉寄存器号/id/偏移,取 SHA-256 前 16 字节,32 个十六进制字符),同一个缺陷在不同 app、不同寄存器分配下归到同一组。**这个指纹函数只有一份实现**:上报、`scripts/fuzz-hermes-base.ts` 的去重、以后的线上语料回放共用它和同一套测试,否则聚合出来的次数是假的。
63+
6264
服务端(pushy-go 分支 `hermes-base-outcome`,提交 `9208c24`)新增可空列 `versions.hermesBaseOutcome` / `hermesBaseDetail`,解析器接受缺字段与 JSON null,只拒绝类型错误与未知枚举值;版本列表接口一并透出。全体应用的拒绝率:
6365

6466
```sql

‎scripts/fuzz-hermes-base.ts‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ import { spawnSync } from 'node:child_process';
2727
import fs from 'fs-extra';
2828
import os from 'os';
2929
import path from 'path';
30-
30+
import { failureFingerprint } from '../src/utils/failure-fingerprint';
3131
import { compareHermesBytecode } from '../src/utils/hermes-base';
3232
import { fuzzStringLiterals } from './hermes-fuzz-literals';
3333
import { hermesFuzzSucceeded } from './hermes-fuzz-result';
@@ -560,14 +560,13 @@ function compile(
560560
return (run.stderr || run.stdout || `exit ${run.status}`).trim();
561561
}
562562

563-
/** collapse ids/offsets/registers so one normalization gap counts once */
564-
function dedupeKey(detail: string): string {
565-
return detail
566-
.replace(/Function<[^>]*>/g, 'Function<…>')
567-
.replace(/\br\d+\b/g, 'r#')
568-
.replace(/\d+/g, '#')
569-
.replace(/"[^"]*"/g, '"…"');
570-
}
563+
/**
564+
* Collapse ids/offsets/registers so one normalization gap counts once. This is
565+
* the same key the CLI reports and the server groups by: a finding here and
566+
* the same defect seen in the field have to land in one bucket, which they
567+
* only do while both sides call this one function.
568+
*/
569+
const dedupeKey = failureFingerprint;
571570

572571
interface Finding {
573572
key: string;

‎src/utils/failure-fingerprint.ts‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
/**
2+
* One implementation of the failure fingerprint, shared by everything that
3+
* groups the same failure: the version/create report, the fuzzer's dedup of
4+
* its findings, and (later) the replay of the stored corpus. Two
5+
* implementations would mean the counts behind "how often does this happen"
6+
* are fiction, so callers import from here rather than writing their own
7+
* regexes.
8+
*
9+
* Redaction is the other half. A detail line carries whatever hermesc printed,
10+
* which is the user's own code: the first rejection seen in production named
11+
* the property `promotionRequestItemId`. Details travel to the server, into
12+
* issue lists and -- once the fix loop runs -- into public pull requests and CI
13+
* fixtures, so the identifiers are replaced by tokens *before* the report
14+
* leaves the machine. The local console keeps the unredacted text: that is
15+
* where the name is actually useful.
16+
*/
17+
import { createHash } from 'node:crypto';
18+
19+
const sha = (value: string) => createHash('sha256').update(value).digest('hex');
20+
21+
/** stable stand-in for one redacted value; the same input always yields it */
22+
const token = (kind: string, value: string) =>
23+
`${kind}#${sha(value).slice(0, 8)}`;
24+
25+
/**
26+
* Replace the parts of a detail line that can only come from the user's code:
27+
* quoted string operands (property names, string literals), function names,
28+
* and filesystem paths that reach the line through a compiler's stderr. What
29+
* stays is the shape a fix is reasoned about -- opcodes, registers, counts,
30+
* literal kinds -- plus each redacted value's length and character class.
31+
*
32+
* This is redaction by class, not a proof: it covers the shapes the comparison
33+
* and the compilers are known to emit. Anything that arrives in an unknown
34+
* shape still has its paths and quoted runs stripped, so a new detail format
35+
* cannot silently start leaking identifiers.
36+
*/
37+
export function redactFailureDetail(detail: string): string {
38+
return (
39+
detail
40+
// Paths first: a compiler's stderr reaches the line with them, and
41+
// running this pass after the others would eat the `/<length>` suffix
42+
// the string pass writes.
43+
.replace(/(?:\.{0,2}\/)[^\s:,)"']*/g, (path: string) => {
44+
const ext = /\.([A-Za-z0-9]+)$/.exec(path);
45+
return `${token('path', path)}${ext ? `.${ext[1]}` : ''}`;
46+
})
47+
// Function<name>(…) headers, including the raw-audit variants
48+
.replace(
49+
/\b(Function|NCFunction|Constructor)<([^>]*)>/g,
50+
(_all, kind: string, name: string) =>
51+
`${kind}<${name ? token('fn', name) : ''}>`,
52+
)
53+
// Quoted operands. hermesc does not escape quotes inside strings, so the
54+
// run is taken as-is up to the next quote; a stray tail keeps whatever
55+
// the earlier passes left rather than being reconstructed.
56+
.replace(/"([^"\n]*)"/g, (_all, value: string) => {
57+
const units = Array.from(value);
58+
const ascii = units.every((char) => char.charCodeAt(0) < 0x80);
59+
return `"${token('str', value)}/${units.length}${ascii ? '' : '/u16'}"`;
60+
})
61+
);
62+
}
63+
64+
/**
65+
* The grouping key: a redacted detail with everything that varies between two
66+
* occurrences of the same defect removed -- registers, ids, offsets, labels,
67+
* counts and the redaction tokens themselves. Sixteen bytes; the server stores
68+
* it as 32 hex characters.
69+
*/
70+
export function failureFingerprint(detail: string): string {
71+
const shape = redactFailureDetail(detail)
72+
.replace(/#[0-9a-f]{8}/g, '#')
73+
.replace(/\br\d+\b/g, 'r')
74+
.replace(/\bL\d+\b/g, 'L')
75+
.replace(/\d+/g, 'N')
76+
.replace(/\s+/g, ' ')
77+
.trim();
78+
return sha(shape).slice(0, 32);
79+
}

‎src/utils/hermes-base.ts‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import path from 'path';
1818
import { PassThrough, Readable } from 'stream';
1919
import { pipeline } from 'stream/promises';
2020
import { tempDir } from './constants';
21+
import { failureFingerprint, redactFailureDetail } from './failure-fingerprint';
2122
import { getHbcVersion } from './hbcTransform';
2223
import { normalizeCachedObjectInstruction } from './hermes-cached-object';
2324
import {
@@ -95,8 +96,17 @@ export interface HermesBaseMeta {
9596
baseHash: string | null;
9697
/** absent (never null) when the bundle step did not run hermesc */
9798
hermesBaseOutcome?: HermesBaseOutcome;
98-
/** first difference / failure reason; absent when there is none */
99+
/**
100+
* First difference / failure reason, redacted (see redactFailureDetail):
101+
* the raw text carries the user's own property and string names. Absent
102+
* when there is none.
103+
*/
99104
hermesBaseDetail?: string;
105+
/**
106+
* Grouping key for the same defect across builds and apps, computed from
107+
* the unredacted detail. Absent with the detail.
108+
*/
109+
hermesBaseFingerprint?: string;
100110
}
101111

102112
// ---------------------------------------------------------------------------
@@ -988,8 +998,15 @@ export function hermesBaseMeta(
988998
};
989999
if (check) {
9901000
meta.hermesBaseOutcome = check.outcome;
991-
const detail = truncateHermesBaseDetail(check.detail);
992-
if (detail) meta.hermesBaseDetail = detail;
1001+
// The console above keeps the real text -- that is where the property
1002+
// name helps. What leaves the machine is redacted and fingerprinted.
1003+
const detail = truncateHermesBaseDetail(
1004+
redactFailureDetail(check.detail ?? ''),
1005+
);
1006+
if (detail) {
1007+
meta.hermesBaseDetail = detail;
1008+
meta.hermesBaseFingerprint = failureFingerprint(check.detail ?? '');
1009+
}
9931010
}
9941011
return meta;
9951012
}

‎tests/failure-fingerprint.test.ts‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
import { describe, expect, test } from 'bun:test';
2+
import {
3+
failureFingerprint,
4+
redactFailureDetail,
5+
} from '../src/utils/failure-fingerprint';
6+
7+
// The detail of a rejected Hermes base, in the shape the comparison emits.
8+
const REJECTED =
9+
'Function<h>(3 params, 21 registers, 1 numbers, 2 non-pointers): +72: ' +
10+
'DefineOwnById r3, r6, 1, "shipmentTrackingR"... vs ' +
11+
'DefineOwnById r3, r6, 1, "shipmentTrackingReference"';
12+
13+
describe('redactFailureDetail', () => {
14+
test('keeps the shape and drops every name that comes from user code', () => {
15+
const redacted = redactFailureDetail(REJECTED);
16+
expect(redacted).not.toContain('shipmentTracking');
17+
expect(redacted).not.toContain('Function<h>');
18+
// what a fix is reasoned about survives
19+
expect(redacted).toContain('DefineOwnById r3, r6, 1,');
20+
expect(redacted).toContain('+72:');
21+
// each redacted value keeps its length, so a truncated operand still
22+
// reads as the shorter one
23+
expect(redacted).toContain('/17');
24+
expect(redacted).toContain('/25');
25+
});
26+
27+
test('marks non-ASCII strings without revealing them', () => {
28+
const redacted = redactFailureDetail(
29+
'Array Buffer entry 1: [String "中文属性名"] vs [String "bar"]',
30+
);
31+
expect(redacted).not.toContain('中文');
32+
expect(redacted).toContain('/5/u16');
33+
expect(redacted).toContain('/3');
34+
});
35+
36+
test('strips paths that reach the line through a compiler stderr', () => {
37+
const redacted = redactFailureDetail(
38+
'base dump: exit 3: boom: /Users/someone/app/build/delta.hbc',
39+
);
40+
expect(redacted).not.toContain('someone');
41+
expect(redacted).toContain('.hbc');
42+
expect(redacted).toContain('exit 3');
43+
});
44+
45+
test('the same value always redacts to the same token', () => {
46+
expect(redactFailureDetail(REJECTED)).toBe(redactFailureDetail(REJECTED));
47+
});
48+
});
49+
50+
describe('failureFingerprint', () => {
51+
test('groups the same defect across apps, registers and ids', () => {
52+
const otherApp = REJECTED.replace(/shipmentTracking/g, 'promotionRequest')
53+
.replace('r3, r6', 'r9, r2')
54+
.replace('+72', '+8');
55+
expect(failureFingerprint(otherApp)).toBe(failureFingerprint(REJECTED));
56+
});
57+
58+
test('the jump-table offsets of one SwitchImm gap are one group', () => {
59+
const at = (offset: number) =>
60+
`Function<ui>(4 params, 21 registers, 0 symbols): +5: SwitchImm r0, ${offset}, L4, 3, 31 vs SwitchImm r0, 616, L4, 3, 31`;
61+
expect(failureFingerprint(at(620))).toBe(failureFingerprint(at(618)));
62+
});
63+
64+
test('a different instruction is a different group', () => {
65+
expect(failureFingerprint(REJECTED)).not.toBe(
66+
failureFingerprint(REJECTED.replace(/DefineOwnById/g, 'PutByIdLoose')),
67+
);
68+
});
69+
70+
test('is 32 hex characters, as the server column stores it', () => {
71+
expect(failureFingerprint(REJECTED)).toMatch(/^[0-9a-f]{32}$/);
72+
});
73+
});

‎tests/hermes-base.test.ts‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -612,12 +612,22 @@ describe('helpers', () => {
612612
outcome: 'rejected',
613613
detail: 'Function<f> line 3:\n a\n b',
614614
});
615-
expect(rejected).toEqual({
615+
// the function name is redacted on the way out; the shape is not
616+
expect(rejected.hermesBaseDetail).toMatch(
617+
/^Function<fn#[0-9a-f]{8}> line 3: a b$/,
618+
);
619+
expect(rejected.hermesBaseFingerprint).toMatch(/^[0-9a-f]{32}$/);
620+
expect({
621+
...rejected,
622+
hermesBaseDetail: '',
623+
hermesBaseFingerprint: '',
624+
}).toEqual({
616625
bytecodeVersion: 98,
617626
baseVersionId: null,
618627
baseHash: null,
619628
hermesBaseOutcome: 'rejected',
620-
hermesBaseDetail: 'Function<f> line 3: a b',
629+
hermesBaseDetail: '',
630+
hermesBaseFingerprint: '',
621631
});
622632
// no detail → no key (the server rejects JSON null, and '' is noise)
623633
expect(hermesBaseMeta(null, 98, { outcome: 'none' })).toEqual({

0 commit comments

Comments
 (0)