Skip to content

feat(lsp): Add textDocument/completion to LSP#2398

Draft
Kara-Zor-El wants to merge 39 commits into
grain-lang:mainfrom
Kara-Zor-El:Kara-Zor-El/lsp-completions
Draft

feat(lsp): Add textDocument/completion to LSP#2398
Kara-Zor-El wants to merge 39 commits into
grain-lang:mainfrom
Kara-Zor-El:Kara-Zor-El/lsp-completions

Conversation

@Kara-Zor-El

@Kara-Zor-El Kara-Zor-El commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Hey, been working on getting completions working in the LSP. So far its going well but hopefully much more cleanup and work to come.

This pr adds context-aware textDocument/completion support to the LSP.

Completions are driven by Tree-sitter for syntax context (so they still work on incomplete / broken code while editing) and supplements it with the typed environment when we have a successful compilation. Depending on where the cursor is, we can offer keywords, in-scope values/types, import paths, module names, graindoc attributes, etc.

Huge credit to @spotandjake for helping out and answering questions throughout as well as his immense help with the tree-sitter work. Work here is in part based on his earlier work in #1900

Currently I am using:

Currently tree-sitter-grain is licensed as MIT. I think currently Jake and Oscar mentioned we might be able to change the license for the parser code to MIT. Will wait for confirmation but can change to LGPL-3 for the time being.

I do still have a couples areas I am trying to smoothen out and squash a few bugs so will leave this as a draft for now and would love some feedback even if not in a complete state

@Kara-Zor-El
Kara-Zor-El force-pushed the Kara-Zor-El/lsp-completions branch from 00796d1 to 92c4ca7 Compare July 11, 2026 18:43
Comment thread compiler/grainlsp/dune

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going through and adding some clarifying comments throughout. Had to expand this separately since the implementation for native vs js backends for grain-tree-sitter.ml use different underlying libraries for each target (ocaml tree_sitter bindings vs wasm tree-sitter-web bindings). Not sure if there's a better way to do it here or in the layer I wrote but open to feedback

Comment thread compiler/src/parsing/keywords_gen.re

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spotandjake mentioned wanted to have a discussion about how is best to test the lsp and completions going forwards. Obviously completions need to be thoroughly tested but the current manual positioning method could probably use some work. Will leave it up to Jake to explain more but my current thoughts are to have a helper function and a position and/or range character
for example

let x: §

where the section sign (§) specifies where the cursor is. And for ranges to use « and » for ranges

Comment thread compiler/dune-project
@@ -1,4 +1,4 @@
(lang dune 3.0)
(lang dune 3.7)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This had to be changed to allow for (include_subdirs qualified) (https://dune.readthedocs.io/en/stable/reference/dune/include_subdirs.html#qualified) to make it part of a module hierarchy. This was mainly done for organizational reasons. In my testing I haven't observed anything breaking due to this change. Happy to discuss further

Comment thread compiler/esy.json
},
"dependencies": {
"@grain/binaryen.ml": ">= 0.34.0 < 0.35.0",
"@grain/grain-tree-sitter.ml": "*",

@Kara-Zor-El Kara-Zor-El Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the intent here would be to ideally have grain-tree-sitter.ml and tree-sitter-grain (which I should probably change the name of one of them, lol) to become part of the grain project (not the repo but rather than org and under the grain umbrella)

Comment thread compiler/esy.json Outdated
%token EXCEPT [@keyword] FROM [@keyword] STAR
%token SLASH DASH PIPE
%token EOL EOF

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on L46 there's the reserved keywords not used yet. Whenever these are added, they'd need a attribute applied as well

Comment thread cli/package.json
"assets": "bin/*.js"
"assets": [
"bin/*.js",
"bin/*.wasm"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wasm file needed for js side of the completions

@Kara-Zor-El

Kara-Zor-El commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

A couple of other things I want to bring up related to tree-sitter. I think tree-sitter gives us great opportunity elsewhere within the language server.

  1. textDocument/hover (which already exists) and completionItem/resolve can have docblock support added to them. @spotandjake mentioned it might make sense (once we refine docblocks in the tree-sitter impl) to potentially rewrite the docblock parsing to take advantage of tree-sitter potentially when doing this and to have a better way of caching docblocks (maybe as json)

  2. Another one of Jake's ideas which I personally think is great is using tree-sitter to replace our current formatting code. Topiary currently is an example of this being done. It would make it much easier for textDocument/rangeFormatting to be supported. I think something part of the CLI instead of an external tool would be the way to go here but it does seem like it could potentially be helpful here

  3. Potentially can be used as a fallback for goto when dirty. Also rename and references

| Some(_) =>
let command: Protocol.command = {
title: "",
command: "editor.action.triggerSuggest",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when doing snippets, unfortunately we can't force the LSP to automatically suggest options. This is the closest I was able to figure out but unfortunately editor.action.triggerSuggest is vscode (and derivative) specific.

@Kara-Zor-El Kara-Zor-El changed the title feat(lsp): Add textDocument/Completions to LSP feat(lsp): Add textDocument/completion to LSP Jul 11, 2026
type t = {capabilities: lsp_capabilities};

let capabilities = {
position_encoding: "utf-8",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: This is used because grain assume UTF-8. It should be noted that the LSP protocol requires UTF-16 support so this is not spec compliant. The only major editor I am aware of that does not have UTF-8 support is Zed.

@Kara-Zor-El
Kara-Zor-El force-pushed the Kara-Zor-El/lsp-completions branch from 7ebfe9a to 8043c41 Compare July 16, 2026 17:20
@Kara-Zor-El
Kara-Zor-El force-pushed the Kara-Zor-El/lsp-completions branch from ffd3f53 to e3ebe83 Compare July 19, 2026 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant