Skip to content

Commit e1d0995

Browse files
authored
ClusterFuzz: Show invalid executions in test, and update expectation (WebAssembly#8087)
The % valid declined from 99% to 95%, likely to to custom descriptors, but I think it's still high enough. Also log them out - there are usually just a few, and it can be interesting to see them.
1 parent c0bcdd6 commit e1d0995

File tree

1 file changed

+14
-2
lines changed

1 file changed

+14
-2
lines changed

test/unit/test_cluster_fuzz.py

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -454,8 +454,20 @@ def test_file_contents(self):
454454
# rare, and in a sense they are actually unuseful).
455455
if proc.returncode == 0 and b'[fuzz-exec] calling ' in proc.stdout:
456456
valid_executions += 1
457-
458-
print('Valid executions are distributed as ~ mean 0.99')
457+
else:
458+
print('====')
459+
print('invalid execution, returncode: ', proc.returncode)
460+
print('stdout:')
461+
print(proc.stdout)
462+
print('stderr:')
463+
print(proc.stderr)
464+
print('====')
465+
466+
# We do not want valid executions to be 100%, as some amount of
467+
# traps during startup are interesting (VMs can have bugs there),
468+
# but the vast majority should be valid. (Startup traps can include
469+
# null descriptors, segments out of bounds, etc.)
470+
print('Valid executions are distributed as ~ mean 0.95')
459471
print(f'mean valid executions: {valid_executions / N}')
460472
# Assert on having at least half execute properly. Given the true mean
461473
# is 0.9, for half of 100 to fail is incredibly unlikely.

0 commit comments

Comments
 (0)