Default `dedup=True` discards a same-ID node's attributes instead of merging them — LLM semantic summaries are silently lost

[SinghAman21]

Summary

build()'s docstring promises a key-wise union for same-ID nodes:

Extractions are merged in order. For nodes with the same ID, the last
extraction's attributes win (NetworkX add_node overwrites). Pass AST
results before semantic […]

But on the default path deduplicate_entities() runs first and collapses
same-ID nodes by keeping one whole dict and discarding the other. Any attribute
the loser had and the winner did not — summary, confidence_score,
references — is gone before NetworkX ever sees it.

Why it matters

This is exactly the AST↔semantic reconciliation the pipeline is built to
perform. _semantic_id_remap exists specifically to force the AST node and the
LLM node onto the same ID so they merge; when they do, the LLM's contribution is
thrown away and only the AST attributes survive.

The user has already paid for those summaries in API credits. The loss is
silent — the only output is a note: line about the label, which does not
mention that summary and confidence_score were dropped.

dedup=True is not an opt-in: it is hard-coded on the CLI path
(cli.py:3230, cli.py:3235), so this is what every graphify extract run
with a semantic pass does.

Reproduction

from graphify.build import build

AST = {"nodes": [{"id": "src_auth_login", "label": "login", "file_type": "code",
                  "source_file": "src/auth.py", "_origin": "ast",
                  "source_location": "L42"}], "edges": []}
SEM = {"nodes": [{"id": "src_auth_login", "label": "User login handler",
                  "file_type": "code", "source_file": "src/auth.py",
                  "summary": "Authenticates a user.",
                  "confidence_score": 0.9}], "edges": []}

for d in (False, True):
    n = dict(build([AST, SEM], directed=True, dedup=d).nodes["src_auth_login"])
    print("dedup=%-5s label=%-22r summary=%r score=%r"
          % (d, n.get("label"), n.get("summary"), n.get("confidence_score")))

Actual:

[graphify] note: node 'src_auth_login' was extracted twice from 'src/auth.py' under
different labels — keeping 'login', dropping 'User login handler'.
dedup=False label='User login handler'   summary='Authenticates a user.' score=0.9
dedup=True  label='login'                summary=None                    score=None

Expected: dedup=True yields at least the attribute set that dedup=False
yields, with the survivor's values winning on genuine conflicts. Instead
summary and confidence_score are dropped entirely.

Root cause

graphify/dedup.py:321-338 — the survivor replaces the incumbent wholesale
rather than gap-filling from it:

incumbent = seen_ids.get(nid)
if incumbent is None:
    seen_ids[nid] = node
elif _collision_rank(node) < _collision_rank(incumbent):
    seen_ids[nid] = node          # <-- whole-dict replace, incumbent's keys lost
    dropped[nid].append(incumbent)
else:
    dropped[nid].append(node)     # <-- node's keys lost

The comment immediately above (dedup.py:318-320) states the premise that makes
this look safe:

[…] a same-entity merge costs nothing and stays quiet.

That premise is false whenever the two records carry disjoint attribute sets,
which is the normal AST-plus-semantic case.

Suggested fix

Gap-fill instead of replacing, leaving the existing dropped /
_report_id_collision bookkeeping untouched:

if incumbent is None:
    seen_ids[nid] = dict(node)
elif _collision_rank(node) < _collision_rank(incumbent):
    merged = dict(node)
    for k, v in incumbent.items():
        merged.setdefault(k, v)
    seen_ids[nid] = merged
    dropped[nid].append(incumbent)
else:
    for k, v in node.items():
        incumbent.setdefault(k, v)
    dropped[nid].append(node)

Conflicting keys still resolve to the survivor's value, so the collision
reporting and the _collision_rank ordering semantics are unchanged.

Test coverage

tests/test_dedup.py asserts on len(result_nodes), label, source_file,
and stderr text. No test asserts the absence of extra keys, and none covers a
same-ID pair with disjoint attributes, which is why this survived.
test_defining_file_wins_over_referencing_file and
test_same_file_relabel_is_noted are unaffected by the change above.

Environment

• graphify v8 @ 6bbc9d1 (pyproject version = "0.9.22")
• Python 3.12, Linux

Triaged by : @SinghAman21 and @claude

[safishamsi]

Fixed in v0.9.24 (#2102). When two exact-ID records from the same source file collapse, non-conflicting attributes are now retained deterministically, records from different files stay isolated, and a dropped record can never stamp a false _origin onto the survivor. Thanks for the report. https://github.com/Graphify-Labs/graphify/releases/tag/v0.9.24