Skip to content

Releases: Graphify-Labs/graphify

v0.9.25

Choose a tag to compare

@safishamsi safishamsi released this 22 Jul 22:56

Maintenance release: a license change to Apache 2.0 and a dead-code removal.

License

  • graphify is now licensed under the Apache License, Version 2.0 (previously MIT). Apache 2.0 adds an explicit patent grant, a patent-retaliation clause, and explicit inbound-contribution terms. Contributions made before the relicensing were submitted under MIT and remain available under those terms; the original MIT text is retained in LICENSE-MIT and referenced from NOTICE.

Removed

  • .graphifyinclude handling is gone (#2112). The file had been non-functional since dot directories became indexed by default (#873): its loader and matchers had no consumers, so detect parsed the file on every run and then discarded the result, making a .graphifyinclude a silent no-op. The dead loader and matchers are deleted, a leftover .graphifyinclude no longer appears in the unclassified list, and detect prints a one-time note when one is present at the scan root. To re-include ignored paths, use ! negation patterns in .graphifyignore.

v0.9.24

Choose a tag to compare

@safishamsi safishamsi released this 22 Jul 18:47

Maintenance release. Correctness fixes across extraction, dedup, query rendering, and the sensitive-file filter, plus a hang fix in the .NET/XAML path.

Fixes

  • The XAML code-behind .cs scan is now bounded and prunes noise directories, so it can no longer hang. A standalone extraction on a .xaml under a large or shared parent (a temp dir, a big monorepo) could resolve the project root to a broad ancestor and recursively scan the whole tree. It now walks with node_modules/.venv/.git/dot-dir pruning and a directory cap: a real project scans fully, a runaway root degrades to a fast partial scan.
  • The sensitive-file filter no longer silently drops topic docs and real source (#2106). Prose files whose slug merely ends in a keyword (privacy-tokens.md) and real source like service_account.py were dropped with no trace, while some genuine secrets (.npmrc, .pypirc, .git-credentials, case variants) were missed. The filter is now stricter on real secrets and no longer loses legitimate files, and both graphify extract and the skill flow now name the skipped files instead of only a count.
  • calls edges now resolve through an aliased Python import (#2082, thanks @Yyunozor). from pkg import mod as alias recorded the import but dropped every downstream alias.func() call, so the callee looked like dead code.
  • dedup preserves a node's attributes when two exact-ID records from the same source file collapse (#2091, thanks @Synvoya). Non-conflicting attributes are retained deterministically, records from different files stay isolated, and a dropped record can never stamp a false origin onto the survivor.
  • The claude-cli backend now reads the CLI's structured-output channel instead of free-form prose (#2076, thanks @Yyunozor), which had parsed to zero nodes and bisected forever on newer Claude Code.
  • graphify explain on a high-degree node groups the cut connections by file instead of a bare ... and N more (#2009, thanks @Yyunozor).
  • graphify query and MCP query_graph no longer print calls edges backwards (#2080, thanks @Yyunozor); the renderer recovers the stored direction from the edge.

Features

  • get_neighbors and get_community (MCP) now honor a token_budget (default 2000) so one call on a god node or large community can't flood the client's context (#2069, thanks @ojmucianski). Truncation is announced at the top of the output.

Docs

  • --code-only is surfaced in the extract usage text and README (#2071, thanks @HerenderKumar).
  • README troubleshooting note for an older graphifyy in system site-packages shadowing uv run --with graphifyy (#1540, thanks @HerenderKumar).

graphify 0.9.23

Choose a tag to compare

@safishamsi safishamsi released this 21 Jul 17:59
  • Fix: caller / "call sites" listings now report the actual call-site line, not the caller function's definition line. explain, affected, and the MCP get_neighbors/query tools printed the caller node's source_location (its def line) for an incoming call, so a precise-looking citation sent users to the wrong line. The calls edge already carries the true call-site line; every caller/relation listing now reads the traversed edge's source_file:source_location, falling back to the node's own line only when the edge has none.
  • Fix: query no longer silently drops the answer past its output budget. Rendered nodes were ordered by degree (so a low-degree definition node ranked last and was cut first), the queried symbol was not guaranteed to appear, and the truncation marker sat only at the end so silence read as absence. Nodes are now ranked by hop distance from the query seeds (deterministically), the seed the question named is always rendered first and never truncated, and a prominent notice at the TOP states how many of how many nodes were shown and how to widen the budget. (A branch merge had also silently dropped the seed-first ordering the renderer already supported; it is rewired.)
  • Fix: graphify uninstall no longer deletes a user-authored ### graphify section (#2062). The uninstall strip used an unanchored ## graphify pattern that matched inside a user's H3 heading (and the "already installed" guard was a substring test), so hand-written content was destroyed. The heading is now matched only when a line is exactly the marker (mirroring the install-side #1688 hardening), across all six strip sites (CLAUDE.md, AGENTS.md, GEMINI.md, copilot-instructions.md, CODEBUDDY.md, and the H1 skill registration).
  • Fix: graphify path (and the MCP shortest_path tool) now return a deterministic route and label each hop with the edge's actual stored relation (#2074). The route was computed over a hash-seeded undirected view, so it varied run-to-run among equal-length paths; and the printed relation was read from an arbitrarily-collapsed parallel edge, so it could show calls on a pair that only carries references. The traversal is now over a sorted graph, and each hop shows the real relation(s), falling back to an honest related when none is stored.
  • Fix: cluster-only --no-label no longer permanently suppresses real community labels (#2073). It wrote Community N placeholders (plus a matching signature) into .graphify_labels.json, which the reuse path then treated as fresh forever. Placeholder-only runs no longer persist the sidecar, a stored placeholder is treated as absent so already-polluted graphs self-heal, and the watch/update rebuild got the same treatment.
  • Fix: build_from_json's ghost-duplicate merge now keys on the full source path, not the bare basename (#2068). Unrelated nodes from different files sharing a common basename (index.md, README.md) and a generic label were silently merged onto one survivor with their edges rewired, corrupting multi-corpus doc graphs. The legitimate AST/LLM same-file merge is preserved; cross-directory false merges are eliminated.
  • Fix: Python import resolution no longer depends on the scan root (#2072). A src-layout project (code under src/) lost most of its imports/imports_from edges when scanned from the repo root, because absolute imports resolved only against the scan root while file-node ids are scan-root-relative, so the dangling edges were silently dropped. Absolute imports now resolve against nested package roots (detected via the __init__.py chain), and import edges are repointed to the real file nodes, so the graph is identical whether scanned from the repo root or from src/.

graphify 0.9.22

Choose a tag to compare

@safishamsi safishamsi released this 20 Jul 15:41
  • Fix: a node whose source_file is a URL/virtual scheme (gdoc://, s3://, http://, ...) is no longer evicted on the second graphify update (follow-up to #2051). The #2051 disk-absence sweep guarded such sources with a literal "://" check, but write-side path normalization collapses the double slash (gdoc://x becomes gdoc:/x), so the guard missed the node on the next run and dropped it into the disk-absence eviction branch. The scheme is now matched tolerantly (and a Windows drive letter like C:/ is not misread as remote).
  • Fix: a real source directory named env/.env/*_env is no longer silently pruned as a false-positive Python virtualenv (#2058). detect's directory-noise heuristic matched those names before .graphifyignore negation and with no trace in any output bucket, so codebases using them as source dirs (common in UVM/ASIC verification) lost large subtrees undetectably. The venv heuristic for those names is now gated on an actual marker (pyvenv.cfg, an activate script, lib/python*, or conda-meta/); venv/.venv/*_venv stay name-only, and every pruned-as-noise directory is now recorded in a pruned_noise_dirs bucket for traceability.
  • Fix: Office (.docx/.xlsx) and Google-Workspace sidecars are now named from the scan-root-relative path, not the absolute path (#2059). The absolute-path hash salted the sidecar name with the checkout location, so committing graphify-out/ (a supported workflow) produced a new duplicate .md per clone/worktree, each ingested as a distinct source document. The relative hash is stable across checkouts while still disambiguating same-stem files; the Google-Workspace sidecar path additionally gains the NFC normalization it was missing.
  • Fix: serve.py's "graph.json is corrupted" recovery message is now reachable (#2005, thanks @kimdzhekhon). json.JSONDecodeError subclasses ValueError, and the broad except (ValueError, FileNotFoundError) clause was ordered first, so a truncated graph printed the bare Expecting value... instead of the documented rebuild hint. The JSONDecodeError clause now comes first.
  • Fix: graphify god-nodes/god_nodes is now a real CLI subcommand, and graphify extract --output DIR is honored as an alias of --out (#2004). god_nodes was an analyzer, an MCP tool, and a documented capability but had no CLI command; --output was silently dropped on extract even though graphify tree documents it. (The affected/reverse-dep import-id mismatch from the same report is tracked separately.)
  • Fix: a nested class/object/trait now gets its contains edge from the enclosing type instead of the file node (#2040). Across ~19 languages the edge was hard-coded to source from the file, so the containment tree was flat (file -> Inner) rather than nested (file -> Outer -> Inner); it now sources from the enclosing type when present, with top-level types still contained by the file.
  • Fix: file nodes that share a basename now get a directory-qualified label so explain/discovery can tell them apart (#2032). In directory-per-entrypoint repos (Supabase Edge Functions, Next.js page.tsx, Rust mod.rs, Python __init__.py) dozens of files named e.g. index.ts collided under one label, breaking free-text discovery for exactly those files. Colliding file nodes are relabelled to the shortest unique path suffix (process-order/index.ts); unique basenames stay bare, and node ids/edges are unchanged.

graphify 0.9.21

Choose a tag to compare

@safishamsi safishamsi released this 20 Jul 12:37
  • Fix: graphify extract (headless, no --backend) now auto-detects Ollama from the standard OLLAMA_HOST env var, not only graphify's OLLAMA_BASE_URL (#1940, thanks @kimdzhekhon). An explicit OLLAMA_BASE_URL still wins; OLLAMA_HOST is normalized the way the Ollama client does (adds http://, defaults the port to 11434 when omitted, appends the /v1 OpenAI-compat suffix). Wired through both the client base URL and backend auto-detection, so ollama stays opt-in and never shadows a configured paid key. (Supersedes the vendored-bulk #1966.)
  • Fix: a flag-less graphify extract now honors the persisted --exclude patterns instead of silently re-including them (#2027, thanks @oleksii-tumanov). Mirrors the #1971 gitignore-persistence fix: the exclude set is read from .graphify_build.json when --exclude is absent and applied to the scan, and a flag-less run no longer clobbers it; an explicit --exclude still replaces the persisted list.
  • Fix: a pathless --postgres extract (introspect a live DB with no filesystem corpus) no longer crashes before introspection (#2030, thanks @oleksii-tumanov). The no-path branch left detection unbound; it's now initialized, the semantic-cache prune and manifest writes are guarded so a DB-only run can't wipe the file cache or leave a poisoning manifest, and a stale manifest from a prior filesystem run is invalidated.
  • Fix: bare import aliases no longer collapse into file-level self-loops (#2037, thanks @Endogen). A single-file import whose bare stem matched the file's own legacy id was remapped onto the importing file, producing a source == target self-loop reported as a 1-file import cycle. build_from_json now drops any imports/imports_from/re_exports edge whose endpoints are identical; pre-existing self-loops self-heal on the next rebuild.
  • Fix: alias re-exports and imports through a barrel resolve to the defining symbol (#1983, thanks @HerenderKumar). import { X } from './barrel' where the barrel re-exports X from another module now points at X's real defining node, walking the barrel chain (bounded, cycle-safe). When a barrel re-exports the same local name from two different modules the name is ambiguous and left unresolved rather than guessed, so no wrong edge is fabricated. Builds on #1984.
  • Fix: a full graphify update now evicts semantic nodes whose non-code source file (a .txt/.pdf/.png with no AST extractor) was deleted from disk (#2051). The corpus sweep only checked files it could re-extract, so a deleted doc's or image's LLM-derived nodes survived indefinitely and were served as authoritative. Disk absence is now used as the deletion signal for such sources; remote and virtual sources (anything with a :// scheme) are left untouched.
  • Fix: an incremental rebuild whose change set names a file that exists but has no AST extractor (a doc, paper, image, or an excluded path) no longer treats it as a deletion (#2056). The change-set loop routed any present-but-untracked file to the deletion path, which both evicted its semantic nodes and disabled the shrink guard that would otherwise have caught the loss. Such files are now preserved; a genuine on-disk deletion is still evicted, and the shrink guard now falls through to its per-source accounting instead of being waved off wholesale by the mere presence of a deletion in the change set.
  • Fix: code-typed nodes that the semantic pass surfaces from within a document now count as that document's semantic layer (#2014). A doc represented only by code-typed nodes was not recognized as semantically backed, so a rebuild re-scanned it for headings and dropped those nodes. The doc is now correctly treated as semantic-backed and left alone.
  • Fix: the --update runbook no longer marks a semantic file as done when its extraction produced no output (#2015). Step 9 stamped the entire detected corpus into the manifest, so a doc, paper, or image whose chunk failed or was omitted was recorded as complete and never re-queued on the next update, losing its content permanently. The runbook now builds the manifest with the same stamping the library uses (only files that actually produced nodes, edges, or hyperedges are stamped; dispatched-but-empty files have their stale hash cleared so they are retried), across the Claude, Aider, and Devin skill bodies and the shared update reference.
  • Fix: build_merge (the --update runbook path) now prunes a deleted file's nodes, edges, and hyperedges regardless of whether their stored source_file is absolute or relative (#2012). When the caller passed no scan root, a node that had kept an absolute path slipped past the relative prune set and the deleted file's graph survived silently. Matching is now form-insensitive (raw, normalized-relative, then an absolute-identity fallback), a re-extracted file is still never pruned, and graphify extract records the scan root marker after every write so a later update relativizes paths correctly even under a custom --out.

graphify 0.9.20

Choose a tag to compare

@safishamsi safishamsi released this 18 Jul 21:29
  • Fix: the graphify-first search nudge now fires on Claude Code's dedicated Grep tool, not just Bash (#1986, thanks @mdshzb04). The installed PreToolUse hook only matched Bash, so a Grep tool call (whose tool_input is {pattern, path, glob, ...}, not {command}) slipped through and never got nudged toward graphify query. The matcher is now Bash|Grep and the search guard recognizes the Grep shape; it stays nudge-only (never the strict deny), and the uninstall filters + #1840 gating are unchanged.
  • Fix: installed hook commands now use forward slashes in the graphify exe path so Git Bash doesn't strip them (#1987, thanks @varuntej07). On Windows the resolved exe path had backslashes, which Git Bash (how Claude Code shells hooks) treats as escapes and drops, breaking the hook with "command not found". _resolve_graphify_exe now normalizes \ to / at the single choke point, covering every emitter (Claude/CodeBuddy PreToolUse, Gemini BeforeTool, Codex); quoting and the --strict suffix are preserved and POSIX is unaffected.
  • Fix: with --out, semantic-cache writes now anchor correctly so the cache round-trips (#1990, #1991, thanks @mdshzb04). The final semantic-cache save resolved a relative source_file against the output dir and wrote 0 entries, and per-chunk recovery checkpoints landed in the wrong directory (under the corpus instead of --out). Cache entries now key on the scan root (portable, matching #1989) while the cache directory sits at the output root, so check/save/checkpoint/prune all agree; composes with the #1989 salt-keying and #1939 prompt-fingerprint namespacing.
  • Fix: alias-based named re-exports no longer emit dangling absolute-path symbol targets (#1983, thanks @oleksii-tumanov). export { X as Y } from './mod' produced a re_exports edge whose symbol target was an absolute-path-prefixed id with no matching node — the symbol-level residual left by #1967 (imports-only) and #1976 (file-level). The aliased re-export target is now rewritten to the canonical symbol node when unambiguous; external re-exports and owned ids are left untouched, so no real edge is dropped.

graphify 0.9.19

Choose a tag to compare

@safishamsi safishamsi released this 18 Jul 11:56
  • Feat: opt-in strict PreToolUse hook that actually makes agents use the graph. The installed Claude Code hook has always nudged the agent to run graphify query before reading raw files, but a nudge is advisory additionalContext the model routinely walks past mid-task. graphify install --project --strict (or graphify claude install --strict) now installs a hook that blocks the first raw source read of a session (permissionDecision: "deny") with a redirect to graphify query, then downgrades to the soft nudge — so it fires at most once per session and can never strand the agent (the next read proceeds even if no query ran, or if graphify query itself failed). Running any graphify query/explain/path refreshes a short-lived "recently oriented" stamp that suppresses the block. Strict mode is Claude Code only (Bash-grep and Glob stay nudge-only; Gemini/Codex/OpenCode can't hard-block and are unchanged); GRAPHIFY_HOOK_STRICT=1/0 toggles it at runtime without a reinstall. Default installs are unchanged (soft nudge).
  • Fix: the PreToolUse hook stops crying wolf (#1840), which applies to the default soft nudge too. It no longer fires for reads of files outside the indexed project (a common false trigger, e.g. a ~/.claude/.../SKILL.md read), and when the graph is stale for the target file (the file changed after the last build, or graphify watch flagged the tree) it softens to a non-mandatory nudge that suggests graphify update instead of demanding the query. Gating is ~3 stat calls — no corpus walk — so it stays fast on large monorepos, and fails open on any error.
  • Fix: a same-basename cross-extension re-export no longer manufactures a phantom self-cycle (#1814, thanks @Greg-Moskalenko for the report and @alphanury for the fix). A typed .ts wrapper that re-exports a hand-written .mjs runtime (export { N } from "./foo.mjs") had foo.ts and foo.mjs collapse onto one base file id (the id stem drops the extension), and while _disambiguate_colliding_node_ids correctly salts the two file nodes apart (foo_ts_foo / foo_mjs_foo), the re-export edge keyed its target salt by the importer's own source file — mis-pointing the ./foo.mjs target back at foo.ts, a source == target self-loop reported as a 1-file import cycle in GRAPH_REPORT.md. During disambiguation an import/re-export edge now carries the resolved target file as a transient salt key, so the salt lands on the real sibling node (generalizing the C/ObjC .h-sibling carve-out from #1475 to every language and to re_exports) and the phantom cycle disappears. That hint has no downstream reader and holds an absolute path, so it is popped once consumed and never persisted — and the graph serializer drops it as a backstop — keeping graph.json deterministic and byte-identical across checkout locations. Node ids are unchanged (the residual was purely at the edge layer). One caveat: a graph written by a pre-fix build still records the stale self-loop, and because graphify update only re-extracts changed files, an unchanged wrapper keeps that edge until it is next edited or a --force full rebuild runs — though any stale absolute hint a pre-fix graph happened to persist is dropped on the next build regardless. (The extension-aware-id alternative was rejected: it would rewrite every file and symbol id and force a full-rebuild migration in lockstep with the skill/validation id spec, #1033.)
  • Fix: file_hash's stat-index memo is now keyed by the path salt, not the absolute path alone (#1989). The digest salts content with the file's path relative to the scan root (for cache portability), but the memo returned whichever digest was computed first for a given absolute path — so the same file hashed under two different roots (which happens within one --out run) got an order-dependent result, and the wrong digest was persisted into stat-index.json across runs. Each entry now stores one digest per salt; legacy un-salted entries are recomputed rather than trusted. Digest computation is unchanged, so existing cache entries still hit.
  • Fix: --no-gitignore extraction opt-out for projects that keep useful code under .gitignore (#1971, thanks @JensD-git for the report and @Mzt00 for the fix). The flag disables only VCS ignore rules (.gitignore + $GIT_DIR/info/exclude); .graphifyignore, the sensitive-dir/secret screens (#1666/#1943), and noise-dir pruning all still apply, so .git/, node_modules/, and real secrets stay out of the graph. The setting persists across update/watch/hook rebuilds and is no longer clobbered back on by a later flag-less graphify extract.
  • Fix: .graphifyignore/.gitignore glob matching now keeps * within a single path segment (#1975, thanks @oleksii-tumanov). An anchored * used to cross / (hand-rolled fnmatch), so an exclude-all + re-include subtree pattern collapsed to zero files and /src/*.py wrongly ignored nested files. Matching now follows git segment semantics (* per segment, ** spans segments, dir/ matches directories not same-named files), verified against git check-ignore; composes with the #1873 anchor-scoping and #1922 diagnostic.
  • Fix: DreamMaker parent-relative #include paths are no longer mangled (#1978, thanks @Osamaali313). The extractor used str.lstrip("./") — a character-set strip that ate every leading .//, so ../shared/base.dm became shared/base.dm (and .hidden/x lost its dot), breaking include resolution. It now strips only a leading ./ prefix.
  • Fix: graphify prs decodes gh/git/claude output as UTF-8 instead of the Windows locale codec (#1980, thanks @luke J). On Windows text=True decoded child output as cp1252, mojibaking or crashing on emoji / non-ASCII names in PR titles and diffs; all subprocess reads now pass encoding="utf-8", errors="replace" (matching the #1505 precedent), which also fixes the encode side when feeding a non-ASCII prompt to the claude-cli backend.

graphify 0.9.18

Choose a tag to compare

@safishamsi safishamsi released this 17 Jul 13:57
  • Fix: an incomplete extraction no longer force-writes a partial graph over a complete one (#1951, thanks @TPAteeq). A crashed AST/semantic pass, a some-chunks-failed run, or a walk that couldn't fully enumerate the corpus (permission-denied subtree) produced a smaller graph that the to_json(force=True) path wrote anyway, bypassing the #479 shrink guard; the --no-cluster raw dump had no guard at all. Both paths now refuse to overwrite a larger existing graph when the run was incomplete (exit 1, nothing written) unless --allow-partial is passed, and a present-but-unparseable existing graph fails closed (a corrupt/mid-write file could be hiding a complete graph). detect()'s walk_errors now count as incomplete too.
  • Fix: graph.json, manifest.json, and the other JSON artifacts are now written atomically (#1952, thanks @TPAteeq). A kill, OOM, or ENOSPC mid-write used to leave a truncated file — and a truncated graph.json then wedged every later run via the shrink guard's fail-safe. Writes now go to a temp file in the same directory and os.replace into place (writing through a symlink so shared-store setups keep working); the writers the original change missed (the --no-cluster dump, merge-graphs/merge-chunks/merge-semantic, the analysis/labels sidecars, and the global graph/manifest) are routed through it too.
  • Fix: truncated LLM chunks are no longer promoted to the semantic cache as complete (#1950, thanks @TPAteeq). A chunk that hit the output-token limit and couldn't be split further was cached and manifest-stamped as authoritative, so its incomplete node set replayed forever. Such chunks are now marked partial (treated as a cache miss and re-dispatched), including the common case where the truncation parses to nothing: the give-up sites record the chunk's own files independently of parsed items, so a sliced document whose later slice truncated empty is no longer stamped complete, and a clean slice can't re-promote a partial entry.
  • Fix: untrusted subagent chunk JSON is validated before merge-chunks merges it (#1953, thanks @TPAteeq). A malformed chunk used to crash the whole merge (aborting good chunks) or silently pass an adversarial node id (path-escape) into the graph. Each chunk is now validated (reusing the #825 fragment validator, size-capped, id-charset checked) and a bad one is skipped with a warning rather than aborting; non-numeric token counts can no longer TypeError the merge either.
  • Fix: code-typed semantic nodes with no evidence in the source are flagged (#1949, thanks @TPAteeq). The LLM can surface a file_type:"code" node from a document whose symbol name never actually appears in that source (an inferred or hallucinated symbol); it's now flagged verification: "unverified" (a dedicated node field, reported by graphify diagnose) rather than presented as a read fact. The check verifies against the node's label and id, only touches nodes the model itself presented as solid, and never drops a node.
  • Fix: watch/update no longer re-scans and duplicates concept/rationale-only semantic docs on every rebuild (#1954, thanks @jw3b-dev). The #1915 semantic-doc gate in _rebuild_code recognized a doc as having a semantic (LLM) layer only via a file_type=="document" node, but the extraction spec's preferred shape for a doc full of named concepts is to represent it with ONLY concept/rationale nodes and no separate document node — such a doc never entered semantic_doc_identities, stayed in the AST quick-scan set, and got duplicate heading nodes minted on top of its real semantic nodes on every graphify update/watch/hook rebuild (the #1915 symptom returning for this doc shape). The gate now recognizes the full doc-shaped subset of the canonical six-value file_type enum (document, concept, rationale, paper — the same set build.py already treats as doc-representation types), while code/image nodes and AST-origin-marked nodes are still excluded, so the pre-#1865 legacy-graph safeguard the gate relies on is unchanged.
  • Fix: save_manifest no longer seeds a stale semantic_hash for a file that was dispatched this run but produced no stamped output, masking an LLM-omitted doc on the next run (#1948, thanks @rsolanilla). A file omitted from the semantic result (a --force re-run where the model drops its chunk) is dropped from the files dict cli.py's _stamped_manifest_files() passes to save_manifest, per the #933/#1890 never-stamp-a-failed-chunk contract — but the seed loop that carries forward untouched rows for subset saves (#917) then copied that file's row from the on-disk manifest verbatim, including its semantic_hash from an earlier successful run. detect_incremental(kind="semantic") compared current content against that inherited hash, found a match, and silently reported the file unchanged — defeating the #1890 retry promise the exact way the issue's manual "blank the hash by hand" workaround worked around. save_manifest now accepts a clear_semantic set; the seed loop forces semantic_hash to "" for any file in it instead of inheriting the stale value. cli.py derives the set as semantic_files — what was actually sent to the backend this run (narrowed by the incremental gate and --code-only, widened by deep mode) — minus _stamped_manifest_files()'s result: dispatched-but-not-stamped, regardless of why. Untouched live files that were never dispatched are deliberately not in the set, so a partial incremental run cannot blank the rest of the corpus's stamps. The set is passed at all three _save_manifest call sites. clear_semantic defaults to None (no-op), so every existing caller and the #917/#1908 seed/pruning behavior for untouched and excluded-but-alive rows is unchanged.
  • Fix: the semantic cache no longer replays extractions from an older prompt after an upgrade (#1939, thanks @HunterMcGrew and @SinghAman21). Entries were keyed on sha256(file content + path) alone, with no component for the extraction prompt that produced them, so a release that changed the prompt left every unchanged file a cache hit: the run exited 0, cost.json looked cheap, and the graph silently carried two prompt generations side by side. Semantic entries are now namespaced by a fingerprint of the extraction prompt (cache/semantic/p{fingerprint}/, mirroring the AST cache's v{version}/ layout), keeping both properties #1252 wanted — entries survive releases that don't touch the prompt, and invalidate only when it actually changed. The fingerprint normalizes line endings so a CRLF checkout doesn't look like a prompt change. Both extraction paths pass their prompt: the Python/CLI path (llm.py's _EXTRACTION_SYSTEM, all backends) automatically, and the skill path via a new prompt_file argument in Step B0/B3 pointing at the references/extraction-spec.md the subagents were handed. Pre-existing entries predate fingerprinting and have unknowable vintage: they are still served rather than re-billing a whole corpus, but check_semantic_cache now warns with the count, so the "no signal at all" the report describes becomes a visible one; --force (or GRAPHIFY_FORCE=1) re-extracts them. Old-fingerprint entries are pruned by liveness only, never swept wholesale the way stale AST versions are — two hosts with different prompts can share one graphify-out/, and a wholesale sweep would have each run delete the other's entries. (The two monolith skills, aider and devin, inline their prompt instead of shipping a spec sidecar and stay on the unfingerprinted path for now.)
  • Fix: the Stage 1 sensitive-directory check no longer silently drops legitimate source under secrets/ or credentials/ directories (#1943, thanks @HerenderKumar). A directory named secrets/, .secrets/, or credentials/ is as often a real source package (Go internal/secrets, a credentials/ service module) as a credential store, but _is_sensitive pruned everything beneath one wholesale, with no trace and no override. The dir list is now split: dedicated credential stores (.ssh, .gnupg, .aws, .gcloud) still drop everything unconditionally, while the ambiguous bare-name dirs spare genuine programming-language source — the same carve-out Stage 3 applies to keyword-named files (#1666), extracted into a shared _is_graphable_source predicate so the two stages can't drift. Rescued source still falls through the Stage 2/3 filename screens (secrets/service_account.py and credentials/id_rsa stay dropped), and data/config formats under those dirs (secrets/db.json, .secrets/token.yaml) remain flagged — those are exactly the formats credentials ship in.
  • Fix: PostgreSQL foreign-key references edges are no longer dropped when a routine in the same schema is unparseable (#1854, thanks @sekmur). pg_introspect builds one synthetic DDL document and parsed it with the function stubs emitted before the FK ALTER TABLEs, so a C-language (or otherwise unparseable) routine's stub parsed as a tree-sitter ERROR node that swallowed the trailing FK statements into the error region, losing every FK edge after it. The FK DDL is now emitted before the function stubs, so table-to-table references edges are produced first and can't be eaten by a later unparseable routine.
  • Fix: graphify extract <root> --out <dir> no longer reduces every node's source_file to a bare filename, so a graph.json stays resolvable against its scan root (#1941, thanks @JensD-git). --out passes the output dir as cache_root to relocate the cache, but that value also anchored relativization — so every scanned file failed relative_to(root), fell through to the #1899 out-of-root fallback, tripped its updepth > 3 walk-up guard, and collapsed to a basename; on Windows an --out on another drive hit the cross-drive branch and basenamed unconditionally. extract() now takes an explicit root anchor the CLI pins to the scan root, independent of where the cache li...
Read more

graphify 0.9.17

Choose a tag to compare

@safishamsi safishamsi released this 16 Jul 10:58
  • Acknowledgement: Amp (ampcode.com) platform support, which shipped earlier in v8, was contributed by @zuwasi in #948 (the skill-amp.md skill and the graphify amp install wiring). Belated credit for the work.

  • Fix: a missing manifest.json no longer degrades graphify extract --code-only into a full scan that discards the committed semantic layer (#1925). On a fresh clone (or when the manifest is deliberately untracked because its mtimes churn), the incremental gate required both manifest.json and graph.json; with only the graph present it fell to a full scan, and under --code-only that dropped every doc/paper/image node — silently replacing a curated graph with an AST-only skeleton. An existing graph.json is now a sufficient incremental baseline: detect_incremental already treats an absent manifest as "everything new / nothing deleted", so build_merge + _stale_graph_sources preserve files that are merely out of this run's scope while still evicting genuinely deleted sources.

  • Fix: hyperedge-only documents are now stamped in the manifest instead of being re-extracted on every run (#1920). _stamped_manifest_files (#1897) decided a semantic doc "produced output" by inspecting only nodes and edges, never hyperedges, so a chunk whose only output for a doc was a hyperedge (3+ nodes sharing a concept) left that doc unstamped and perpetually re-queued. Stamping now counts hyperedge output too, mirroring the per-source_file keying the semantic cache already uses.

  • Fix: the PHP extractor now disambiguates same-named classes across namespaces (#1923). A bare class reference (extends Page) collapsed onto the only internal class named Page, so App\Models\Page and an imported Filament\Pages\Page fused into one node, manufacturing a false inherits/imports edge and a bogus cross-community bridge. A new namespace/use-aware resolution pass (mirroring the Java resolver, running before the unique-name rewire) re-points supertype/import references to the real definition, or parks provably-external ones on a fully-qualified stub the bare-name rewire cannot collapse. Plain, non-namespaced PHP is unchanged.

  • Fix: detect() now records files and directories dropped by a .gitignore/.graphifyignore rule in a new ignored diagnostic field (#1922). The nested-ignore scoping bug itself was fixed in 0.9.16 (#1873); this closes the remaining gap where an ignored path left no trace in any diagnostic, so an over-broad rule looked like a clean scan. Entries are per-directory where a subtree is pruned, keeping the list bounded.

  • Fix: _semantic_id_remap is now idempotent, so incremental rebuilds stop churning (#1917). When a file's canonical stem contained its own legacy stem as a prefix (parent dir name equals the file stem, e.g. .claude/CLAUDE.md, docs/docs.md), an already-migrated semantic node id re-matched the legacy branch and gained another stem segment on every build (claude_x -> claude_claude_x -> ...). Because _origin is persisted, every graphify update re-fed nodes through the remap, so the ids grew unboundedly and the same_topology/same_graph/no_change short-circuits never fired — rewriting graph.json and re-running clustering on every zero-delta update. The remap now skips an id that already carries its canonical stem (mirroring the graph_has_legacy_ids check), while a genuine one-time legacy migration still applies. (An already-corrupted graph needs one graphify extract --force to reset the grown ids.)

  • Perf: graphify query now scores the graph once per query instead of T+1 times for a T-term query (#1889 / #1918, thanks @Sirhan1). The per-term-guarantee (#1445) previously re-scored the whole graph once per token; _score_query now computes the combined ranking and each token's singleton winner in a single traversal, feeding _pick_seeds via best_seed_by_term. Behavior is preserved (verified byte-identical against the old per-term scoring across a differential fuzz); ~1.3-1.4x faster and independent of query length.

  • Fix: --mode deep is now effective over a warm cache instead of a silent no-op (#1894). The semantic cache is namespaced by mode (semantic vs semantic-deep) so a shallow-cached file no longer satisfies a deep run; graphify extract gains --force (and honors GRAPHIFY_FORCE) to bypass the incremental gate and the cache read; and a deep incremental run widens its dispatch to the full live doc set so the deep namespace actually gets populated. Cache prune/clear now sweep both namespaces so the deep cache can't accumulate orphans. (The skill-side flow that passes the mode through is a follow-up; the CLI is complete and backward compatible — the new mode argument defaults to the existing behavior.)

  • Fix: the semantic cache no longer persists dangling edges/hyperedges (#1916). When a node group was skipped on write (out-of-scope per the batch guard, or a ghost source_file), edges/hyperedges in the kept groups that referenced those never-written nodes were saved anyway and re-surfaced on every cache replay. Those references are now pruned at write time (gated on the scoping allowlist, so unscoped callers are unchanged), and build_from_json validates hyperedge members against the node set so a dangling hyperedge can't reach graph.json even from a live extraction.

  • Fix: graphify update/watch no longer produces a bloated graph by double-representing documents (#1915). _rebuild_code AST-quick-scanned Markdown/doc files and then preserved their existing semantic (LLM) nodes on top, so each doc appeared twice (a real corpus came out ~4x). A doc that already has semantic nodes in the graph is no longer AST-quick-scanned (its semantic nodes are the sole representation), while a doc with no semantic layer still gets the structural quick-scan; incremental rebuilds now preserve a doc's semantic nodes instead of evicting them, and previously-bloated graphs self-heal on the next full rebuild.

  • Fix: .cjs (explicit CommonJS) files are now recognized as code and parsed with the JavaScript grammar (#1912, thanks @Kookwater). The extension was half-registered (present in some internal maps but missing from the classification and dispatch sets), so .cjs files were silently dropped.

  • Fix: files that become excluded (.graphifyignore/.gitignore/--exclude) are now pruned from both the graph and the manifest instead of lingering (#1908 / #1909). Two coupled gaps: save_manifest retained any prior row that still existed on disk (disk-existence, not scan-membership), so an excluded-but-present file was reported as deleted on every run; and the incremental prune set was derived from the manifest alone, so a newly-excluded file's stale nodes carried forward from the existing graph.json were never removed (the state every 0.9.16 graph is in). Now the incremental prune set also derives from the existing graph's own source_files minus the post-exclude corpus (in-root only), save_manifest prunes rows outside the full scan corpus, and detect_incremental distinguishes truly-deleted files from excluded-but-present ones (which are no longer misreported as deleted).

  • Fix: PostgreSQL PL/pgSQL functions are no longer silently dropped (#1910). CREATE FUNCTION ... LANGUAGE plpgsql AS $$...$$ with OUT params, tagged dollar-quotes, or procedural body statements parses as a tree-sitter ERROR node, which the SQL extractor skipped entirely. It now recovers the function/procedure name from an ERROR node via the same regex-fallback pattern the extractor already uses elsewhere, so the function node and its contains edge are kept (the unparseable body is left opaque) and surrounding statements are unaffected.

  • Security/privacy follow-up: nodes whose source_file was never dispatched are now dropped from the graph, not just skipped from the cache (#1895). The #1757 guard stopped a mis-attributed node from clobbering another file's cache entry, but the node itself still flowed into graph.json; it is now filtered out of the merged result (real-file, non-dispatched attributions only), consistent with the cache rejection.

  • Fix: manifest.json now records every successfully-extracted file, not just the zero-node ones (#1897). The #933 stamping filter compared root-relative node source_files against absolute detect() paths, so it dropped every freshly-extracted semantic document from the manifest and broke the incremental-update baseline. Both sides are now resolved before comparison; genuinely omitted/zero-node docs stay unstamped so they retry.

  • Fix: graphify hook install now registers the graph.json union merge driver that the README and CHANGELOG have long documented (#1902). It writes the merge.graphify config via git config and an idempotent, append-only graphify-out/graph.json merge=graphify line in .gitattributes; uninstall removes them.

  • Fix: hook install/status no longer print a spurious "could not read core.hooksPath" warning on repos whose .git/config contains git-legal duplicate keys (VS Code writes these) (#1907). Config is now resolved via git rev-parse --git-path hooks instead of a strict configparser, which rejected duplicate keys.

  • Fix: graphify export obsidian prunes notes for nodes that left the graph instead of merging old and new on re-export (#1896). Only notes graphify itself wrote (tracked in its ownership manifest) are removed, with a vault-containment guard, so user-authored notes are never touched.

  • Fix: non-English query sentences no longer pick wrong BFS seeds because their filler words were unfiltered (#1900). The query stopword set now covers German and the major Romance languages (curated to avoid clobbering English content words), so Wie funktioniert die Authentifizierung? seeds the keyword, not the stopwords.

  • Fix: Python calls to an imported module now resolve (already shipped in 0.9.16); .skill files (Markdown-with-frontmatter agent files) are now ...

Read more

v0.9.16

Choose a tag to compare

@safishamsi safishamsi released this 14 Jul 23:04

graphify turns any folder of code, docs, papers, images, or videos into a queryable knowledge graph. 0.9.16 is a correctness and privacy release: it fixes two regressions from 0.9.15 that could zero out a graph, closes a residual path leak, and lands a batch of extraction and reliability fixes, most from community reports.

Highlights

  • Two 0.9.15 regressions that could empty your graph are fixed (#1873 / #1887 / #1880). A nested .gitignore with a bare * (a common scratch-dir idiom) was applied to the whole tree instead of its own directory, so detect() returned zero files and produced an empty graph; graphify update then saw 0 nodes and refused to overwrite. Nested ignore patterns are now scoped to their own subtree.
  • Residual path/username leak closed (#1899). Out-of-scan-root reference targets (an out-of-root .csproj/.sln project, a bash source) and degenerate symbols (a minified $, a JSONC "//" key) could still embed the absolute scan path in a committed graph.json. Both are now portable.
  • Silently-lost documents are now surfaced (#1890). When the semantic pass returns a clean response that omits some dispatched documents, the run now reconciles dispatched-vs-returned and warns instead of dropping them without a trace.

All fixes

  • Fix: nested .gitignore/.graphifyignore patterns are scoped to their own subtree; a nested bare * no longer zeroes the corpus (#1873 / #1887 / #1885, thanks @Alwyn93).
  • Fix: graphify update no longer emits 0 nodes and refuses to overwrite when the tree has a nested broad .gitignore (#1880).
  • Fix: close residual absolute-path/username leaks for out-of-root reference targets and degenerate symbols (#1899).
  • Fix: the incremental semantic-cache checkpoint no longer fails on sliced (oversized) documents (#1870).
  • Fix: --exclude patterns persist into update/watch/hook rebuilds instead of applying only to the initial scan (#1886).
  • Fix: Python calls to an imported module (module.function()) now produce a calls edge (#1883).
  • Fix: semantic cache writes are scoped so a mis-attributed source_file cannot clobber another file's entry — including the per-chunk checkpoint (#1757 / #1835, thanks @TPAteeq).
  • Fix: a full graphify update no longer evicts the LLM semantic edges of a re-extracted document (#1865 / #1868, thanks @xor-xe).
  • Fix: ID-collision resolves to a deterministic survivor and warns only about real loss (#1851 / #1852, thanks @bchan84x).
  • Fix: --cargo honors Cargo's package = "..." dependency rename (#1858 / #1861, thanks @thejesh23).
  • Fix: detect_incremental re-extracts a legacy-manifest file when its mtime moves backwards (#1859 / #1862, thanks @thejesh23).
  • Fix: the dedup summary reports the fuzzy-merge count even with zero exact merges (#1857 / #1860, thanks @thejesh23).
  • Fix: uppercase TypeScript extensions (.TS/.TSX/.MTS/.CTS) are parsed with the TypeScript grammar (#1881, thanks @xkam7ar).
  • Fix: Kotlin builtin/stdlib types are filtered from the references graph (#1876, thanks @kebwlmbhee).
  • Fix: the stale/missing-skill version warning prints to stderr, not stdout (#1805 / #1893, thanks @Mzt00).
  • Fix: semantic extraction reconciles dispatched vs returned files and warns on silently-omitted documents (#1890).

Install

pip install --upgrade graphifyy
# or
uv tool install graphifyy@0.9.16

Then graphify install to update the skill for your agent.