Feat/codemap pr by GdoongMathew · Pull Request #565 · AsyncFuncAI/deepwiki-open

GdoongMathew · 2026-07-30T04:02:53Z

Summary

Features

Codemap Generation (AI genrated) (fix #396)

Adds a third mode (Fast / Deep Research / CodeMap), switched via a dropdown.
Split the progress into three parts, analyzing code -> generating initial codemap -> generating diagrams & guides, each streamed as NDJSON events.
Code-Viewer: auto opens at the first citation, syntax-highlights, and scrolls to the highlighted lines.
Streaming transport — /ws/codemap (WebSocket) and POST /codemap/stream (NDJSON HTTP fallback); GET /codemap/file serves source files with path-traversal guarding
New LineTrackingTextSplitter : attaches correct start -> end line to each chunk.

⚠️ Delete stale ~/.adalflow/databases/*.pkl so repos re-index with line numbers.
New schemas: CodeMap, CodeMapSection, CodeMapStep, CodeMapCitation, CodeMapRequest
New prompts: CODEMAP_SKELETON_PROMPT and CODEMAP_ENRICH_PROMPT
Fontend: new CodeMap.tsx, CodeViewer.tsx and the /api/repo/prepare route; Ask.tsx gains a handleCodemapAsk flow and an onOpenCodeViewer citation-click hook.

Fixes

Prevent large repo indexing timeout: indexing large repos used to block the request/WebSocket until the frontend proxy timeout and killed the connection
- Added a dedicated POST /repo/prepare endpoint that indexes the repo up-front and stream the server event. Should emit a heartbeat every 10 seconds while indexing a repo.
- Added a cheap GET /repo/index/status readiness probe so the frontend can poll instead of holding the prepare stream open.
Fix traditional Chinese detection logic. (related to fix: fix language detection GdoongMathew/deepwiki-open#14)

Changes

Backend restructuring
- Move all schemas into schemas module (original Refactor: centralize all pydantic schema models into schemas module #564)
- Refactor RAG module. (original refactor RAG, adding rag module #561)
Removing logging api key in openrouter logs (original removing logging api key in logger. #563)

Miscellaneous

Add pre-commit hook config.
python code base ruff reformatting and isort.
Update readme
Bumps next version to 15.5.22. (fix Security: CVE-2025-66478 (React2Shell) RCE in Dependencies #500)

* refactor chat streaming * update `_is_token_limit_error` logic * test: add chat provider unittests * move `_is_token_limit_error` to module level * remove unnecessary TYPE_CHECKING * add EOF * update prompt function

related to SylphAI-Inc/AdalFlow#496

… api. (#3)" (#4) This reverts commit 6bdc4e8.

* patch: patch adalflow OllamaClient to support batch embedding api. related to SylphAI-Inc/AdalFlow#496 * remove `OllamaDocumentProcessor`

* refactor chat streaming * update `_is_token_limit_error` logic * test: add chat provider unittests * move `_is_token_limit_error` to module level * remove unnecessary TYPE_CHECKING * add EOF * update prompt function * refactor `websocket_wiki.py` using `ChatStreamer` * remove unnecessary imports * add litellm test * update kwargs in `GoogleGenerativeChatStreamer`

# Conflicts: # api/websocket_wiki.py

# Conflicts: # api/chat/_stream.py # api/config.py # api/simple_chat.py

…branch

…tionRequest`

…ranch

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…ranch

…dition. (ai generated)

…nk to share the same page without overlapping each other.

…s_file`

* refactor: add Repo type * add repo schema * fix: fix import path * add prepare_index in frontend. (ai generated) * update preparing indexing in different languages. (ai generated) * update preparing indexing in different languages. (ai generated) * update preparing indexing in different languages. (ai generated)

… fields (#12)

* fix frontend parsing generated wiki. * fix frontend parsing generated wiki.

# Conflicts: # api/chat/_stream.py # api/chat_model.py # api/clients/__init__.py # api/clients/litellm.py # api/config.py # api/data_pipeline.py # api/poetry.lock # api/rag.py # api/simple_chat.py # api/websocket_wiki.py # src/components/Ask.tsx

gemini-code-assist · 2026-07-30T04:02:56Z

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

GdoongMathew · 2026-07-30T04:19:36Z

Hi @sng-asyncfunc,

Heads-up that this PR is fairly large — it bundles the new Codemap feature together with a broader backend refactor and a few fixes, so it may feel a bit overwhelming to review in one pass.

Any feedback is very welcome ~

GdoongMathew and others added 30 commits July 18, 2026 12:01

Refactor/chat streamer (#1)

a9a7b25

* refactor chat streaming * update `_is_token_limit_error` logic * test: add chat provider unittests * move `_is_token_limit_error` to module level * remove unnecessary TYPE_CHECKING * add EOF * update prompt function

update input_too_large threshold (#2)

149c6eb

patch: patch adalflow OllamaClient to support batch embedding api. (#3)

6bdc4e8

related to SylphAI-Inc/AdalFlow#496

Revert "patch: patch adalflow OllamaClient to support batch embedding…

98e499e

… api. (#3)" (#4) This reverts commit 6bdc4e8.

Feat/ollama batch embed patch (#5)

7ef9f19

* patch: patch adalflow OllamaClient to support batch embedding api. related to SylphAI-Inc/AdalFlow#496 * remove `OllamaDocumentProcessor`

fix: fix simple_chat format

a601f2b

refactor and add clients module (#7)

5066942

centralize prompt (related to AsyncFuncAI#306)

c411bee

centralize prompt (related to AsyncFuncAI#306) (#8)

0cb8957

fix prompt

075fbd5

Merge branch 'refs/heads/refactor/centralize_prompt'

9b8c5b5

# Conflicts: # api/websocket_wiki.py

Merge branch 'refs/heads/main' into feat/my_branch

f939434

# Conflicts: # api/chat/_stream.py # api/config.py # api/simple_chat.py

update check_ollama_model_exists using ollama.list api.

0843588

fix exception type

da71585

fix import scope

592523e

Merge branch 'refs/heads/ref/check_ollama_model_exists' into feat/my_…

c7c823a

…branch

fix catching exception

84f0517

Merge branch 'refs/heads/ref/check_ollama_model_exists' into feat/my_…

8613d94

…branch

remove dangling file.

0bf2d4b

use field_validator to simplify dirs and files parsing in `ChatComple…

9907fdc

…tionRequest`

Merge branch 'refs/heads/refactor/chat_completion_req' into feat/my_b…

d1afda8

…ranch

Apply suggestions from code review

e66ae52

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

prevent empty string

990f7b9

unify chat model in websocket_wiki and simple_chat

c233a44

Merge branch 'refs/heads/refactor/chat_completion_req' into feat/my_b…

d4bd5f2

…ranch

import cleanup

95d00b4

Merge branch 'refs/heads/refactor/chat_completion_req' into feat/my_b…

52ee7cf

…ranch

import documentations

b336142

Merge branch 'refs/heads/main' into feat/my_branch

a47da86

GdoongMathew added 22 commits July 27, 2026 11:37

include codemap generation into clear conversation triggering con…

9da3424

…dition. (ai generated)

reformat codemap generated section, split conversation and codemap li…

b61f882

…nk to share the same page without overlapping each other.

fix conversation sequence bug when mixing codemap with other modes

0b4fb8c

fix: fix rag return type, removing RAGAnswer dataclass object.

f51b29f

removing logging api key in logger.

cb9634d

rename utils to services

47fae9c

update type annotation of asave and aload.

c59fbc5

refactor excluded and included mechanism, and fix `_should_proces…

9e6b1b6

…s_file`

fix _should_process_file

47acbc4

pre-commit fix

374cd44

update readme. (ai generated for other languages)

68974e9

Add screenshot

d497480

add codemap screenshot in readme

58e8dce

update tests

dabf22b

add warning if repo's index is not pre-built

73dba1f

refactor schema, create RepoRequestBase to minimize duplicated repo…

a1dc51b

… fields (#12)

Fix/frontend wiki prompt (#13)

cb6f077

* fix frontend parsing generated wiki. * fix frontend parsing generated wiki.

Merge branch 'main' into feat/codemap_pr

158bca1

# Conflicts: # api/chat/_stream.py # api/chat_model.py # api/clients/__init__.py # api/clients/litellm.py # api/config.py # api/data_pipeline.py # api/poetry.lock # api/rag.py # api/simple_chat.py # api/websocket_wiki.py # src/components/Ask.tsx

fix ruff isort

46ac0cc

update logging config and mechanism

978d101

Merge branch 'feat/my_existing_branch' into feat/codemap_pr

5262c8a

This was referenced Jul 30, 2026

Refactor: centralize all pydantic schema models into schemas module #564

Closed

removing logging api key in logger. #563

Closed

refactor RAG, adding rag module #561

Closed

GdoongMathew added 2 commits July 30, 2026 13:24

fix: fix language detection (#14)

ad0c704

Merge branch 'feat/my_existing_branch' into feat/codemap_pr

5f31a78

sng-asyncfunc merged commit b5e7666 into AsyncFuncAI:main Jul 30, 2026
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Feat/codemap pr - #565

Feat/codemap pr#565
sng-asyncfunc merged 112 commits into
AsyncFuncAI:mainfrom
GdoongMathew:feat/codemap_pr

GdoongMathew commented Jul 30, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Jul 30, 2026

Uh oh!

GdoongMathew commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

GdoongMathew commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Features

Codemap Generation (AI genrated) (fix #396)

Fixes

Changes

Miscellaneous

Uh oh!

gemini-code-assist Bot commented Jul 30, 2026

Uh oh!

GdoongMathew commented Jul 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

GdoongMathew commented Jul 30, 2026 •

edited

Loading