feat(core): support project-level tree-sitter extension language aliases - #588
feat(core): support project-level tree-sitter extension language aliases#588newnight wants to merge 4 commits into
Conversation
Add project-level configuration for mapping custom file extensions to
tree-sitter languages. This enables analysis of codebases that use
non-standard extensions (e.g., .view, .inc, .customts) by treating them
as known languages (e.g., php, typescript).
Changes:
- Add TreeSitterPluginOptions with extensionLanguageMap parameter
- Add LanguageRegistry.registerExtensionAlias() method
- Add config.mjs to read .understand-anything/config.json
- Update scan-project.mjs, extract-structure.mjs, extract-import-map.mjs,
compute-batches.mjs, and build-fingerprints.mjs to use config
- Add comprehensive input validation and error messages
- Update documentation across all READMEs and skill files
Configuration format:
{
"treeSitter": {
"extensionLanguageMap": {
".customts": "typescript",
".view": "php"
}
}
}
The configuration persists across runs and integrates with existing
language detection. Invalid language IDs and malformed extensions
throw descriptive errors during initialization.
Backward compatible: projects without config.json use default behavior.
|
@codex review this |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4ab459b9b4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| import { existsSync, readFileSync } from 'node:fs'; | ||
| import { join } from 'node:path'; | ||
|
|
||
| export const CONFIG_RELATIVE_PATH = '.understand-anything/config.json'; |
There was a problem hiding this comment.
When an already-analyzed project uses .ua/ and follows the new documentation to create this config, the mere existence of .understand-anything/ makes resolveUaDirName() in packages/core/src/persistence/index.ts switch all subsequent graph, metadata, and fingerprint reads to that directory. The existing .ua/knowledge-graph.json then appears missing and incremental state is abandoned, so the alias config must live in the resolved data directory or config-only directories must not trigger legacy-data selection.
Useful? React with 👍 / 👎.
| - **Language directive**: Generate all textual content in **$OUTPUT_LANGUAGE**. Keep file paths, IDs, code identifiers, and command snippets unchanged. | ||
|
|
||
| 3. **Read layers** — Grep for `"layers"` to get the full layers array. These define the architecture and will structure the guide. | ||
| 2. Check that `$PROJECT_ROOT/.understand-anything/knowledge-graph.json` exists. If not, tell the user to run `/understand` first. |
There was a problem hiding this comment.
For every new project whose graph is stored at .ua/knowledge-graph.json, step 1 successfully resolves and checks $UA_DIR, but this newly added second check immediately looks only in the legacy directory and tells the user to rerun /understand. Subsequent reads should continue using the already-resolved $UA_DIR/knowledge-graph.json rather than hard-coding .understand-anything.
Useful? React with 👍 / 👎.
|
|
||
| Get-ChildItem -Path $root -Filter '*.md' -File | Sort-Object Name | ForEach-Object { | ||
| $link = Join-Path $Target $_.Name | ||
| New-Junction $link $_.FullName |
There was a problem hiding this comment.
On Windows installs for every platform with an AgentsTarget, each source selected here is a Markdown file, but New-Junction always calls New-Item -ItemType Junction; NTFS junctions target directories, not files. The first profile therefore aborts the installer under $ErrorActionPreference = 'Stop' after skills were only partially installed, so these profiles need file symbolic links or copies rather than the directory-junction helper.
Useful? React with 👍 / 👎.
| const treeSitterExtensionLanguageMap = readTreeSitterExtensionLanguageMap( | ||
| projectRoot, | ||
| { validLanguageIds }, | ||
| ); | ||
|
|
There was a problem hiding this comment.
The configured map is passed to tree-sitter and the language registry, but it never reaches the resolver probes. For example, with .customts -> typescript, a.customts importing ./b is parsed correctly, yet resolveTsJsImport() only probes the hard-coded TS_EXT_PROBES (.ts, .tsx, etc.), so b.customts is never found; the resulting import map loses the edge and batching and the final graph are incorrect. Include configured extensions in the language-specific resolution indexes/probes.
Useful? React with 👍 / 👎.
| } | ||
|
|
||
| const validLanguageIds = new Set(builtinLanguageConfigs.map((config) => config.id)); | ||
| validLanguageIds.add('tsx'); |
There was a problem hiding this comment.
A configuration such as .view -> tsx is explicitly accepted here and by TreeSitterPlugin, but tsx is only a synthetic grammar key and is not an ID in LanguageRegistry.createDefault(). The subsequent registerExtensionAlias(ext, languageId) therefore throws unknown language "tsx", causing structural extraction to fail; register this alias through the TypeScript language config while retaining tsx only for grammar selection.
Useful? React with 👍 / 👎.
|
Hi @newnight , good point and thanks for the PR. Please ping me when you are ready, then I will take a deeper look again~ |
PR Egonex-AI#588 Codex review fixes: - config.mjs: read alias config from .ua/config.json first to avoid triggering data directory switch and discarding cached data - install.ps1: use file symbolic links instead of directory junctions for agent profile Markdown files - extract-import-map.mjs: inject configured TS/JS extension aliases into import resolution probes so cross-file imports resolve correctly - extract-structure.mjs, extract-import-map.mjs, compute-batches.mjs, build-fingerprints.mjs: map synthetic 'tsx' key to 'typescript' before registering extension aliases to avoid runtime errors - README: update config path references from .understand-anything/ to .ua/ across all 6 locale variants
Addressed all Codex review comments — config now reads from .ua/ first to avoid data directory switches, agent profiles use file symlinks instead of directory junctions, custom TS/JS extension aliases are injected into import resolution probes, and tsx is mapped to typescript before registering extension aliases. Core tests pass (979/979). |
2 participants
Add project-level configuration for mapping custom file extensions to tree-sitter languages. This enables analysis of codebases that use non-standard extensions (e.g., .view, .inc, .customts) by treating them as known languages (e.g., php, typescript).
Changes:
Configuration format:
{ "treeSitter": { "extensionLanguageMap": { ".customts": "typescript", ".view": "php" } } }The configuration persists across runs and integrates with existing language detection. Invalid language IDs and malformed extensions throw descriptive errors during initialization.
Backward compatible: projects without config.json use default behavior.
Summary
Add project-level configuration for mapping custom file extensions to tree-sitter languages. This enables
/understandto analyze codebases with non-standard extensions (e.g.,.view,.inc,.customts) by treating them as known languages (e.g.,php,typescript) via a.understand-anything/config.jsonfile.Linked issue(s)
How I tested this
pnpm lint— Passedpnpm --filter @understand-anything/core test— All 937 tests passed (including new extension alias tests).view,.inc) mapped tophpin configscan-project.mjsand verified files detected with correct languageextract-structure.mjsprocesses custom extensions as specified languageVersioning