Skip to content

[bug] query phase aborts deterministically after successful build (beam search returns 1 element, point id stable per dataset; single-threaded too) #1

Description

@sagitrs-architect

Summary

Both neighbors-vamana and neighbors-pipnn binaries build a healthy graph, then abort deterministically during the query phase:

Error: for point id 9996 beam search returned 1 elements, which is less than k = 10
terminate called after throwing an instance of 'std::logic_error'
  what():  Error in beam search return value
Aborted (core dumped)   # exit 134

Key facts established:

  • Single-threaded reproduces (PARLAY_NUM_THREADS=1: build takes 12.0s, then same abort) — so this is not a data race;
  • Stable point id per dataset: always 9996 on SIFT-100K (ids 0-9999) across T=1/8/32; on a different (synthetic) 100K dataset it aborted at id 0 instead — data-dependent, thread-count-independent → looks like a data-dependent edge case in the query path (or UB with a data-dependent manifestation), not concurrency;
  • The built graph is fine: we exported the graph before the abort and evaluated it with an independent beam-search implementation — recall@10 at L=100: vamana 1-pass 0.9993 (deg 40.7), vamana 2-pass 0.9998 (deg 38.3), pipnn 0.9993 (deg 28.8). The defect is isolated to the query phase inside the binaries; build-phase beam search is healthy.

Environment

  • Ubuntu, GCC 15.2.0 (/usr/bin/g++ — only gcc-15/16 available on this box, so we could not A/B an older toolchain; happy to test a patch)
  • CMake Release, parlaylib via your FetchContent pin, no custom flags
  • Data: first 100K points of ann-benchmarks sift-128-euclidean (float32, 10K queries, exact GT)

Reproduce

PARLAY_NUM_THREADS=32 ./vamana/neighbors-vamana_FLOAT_T_EUCLIDEAN \
  -base_path sift100k_base.fbin -query_path sift100k_query.fbin -gt_path sift100k_gt.ibin \
  -k 10 -R 64 -L 100 -alpha 1.2 -num_passes 1 -graph_outfile g.graph -res_path r.res
# same abort with PARLAY_NUM_THREADS=1, and with the pipnn binary

(Apologies for cross-posting: an earlier copy of this report was mistakenly filed on a different repo — thustorage/PipeANN#19 — please disregard that one.)

Happy to share the exact binaries, graph files, or data slices if useful. Thanks for the library!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions