feat(website): Add llms.txt support

[yamadashy]

Checklist

• Run npm run test
• Run npm run lint

[cloudflare-workers-and-pages]

Deploying repomix with    Cloudflare Pages

Latest commit:	e2adda9
Status:	🚫  Build failed.

View logs

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.32%. Comparing base (583d538) to head (16493ac).
Report is 325 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #442   +/-   ##
=======================================
  Coverage   90.32%   90.32%           
=======================================
  Files          96       96           
  Lines        4962     4962           
  Branches     1042     1042           
=======================================
  Hits         4482     4482           
  Misses        480      480           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yamadashy]

related:

• [Bug]: Plugin fails with VitePress rewrites configuration okineadev/vitepress-plugin-llms#47

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

A new plugin, vitepress-plugin-llms, was integrated into the VitePress configuration for the website client. The plugin is included with the excludeIndexPage option set to true. Additionally, the package was added as a development dependency in the project's package manifest.

Changes

File(s)	Change Summary
website/client/.vitepress/config/configShard.ts	Added import and configuration for vitepress-plugin-llms plugin with excludeIndexPage: true.
website/client/package.json	Added vitepress-plugin-llms as a devDependency with a URL-based version reference.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant VitePress
    participant vitepress-plugin-llms

    User->>VitePress: Start build or serve
    VitePress->>vitepress-plugin-llms: Initialize plugin (excludeIndexPage: true)
    vitepress-plugin-llms-->>VitePress: Plugin hooks and modifies build as configured
    VitePress-->>User: Serve or build site with plugin effects

Loading

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch feat/llmstxt

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull Request Overview

This PR adds support for llms.txt to the website. It introduces a new dependency and integrates a corresponding plugin in the VitePress configuration.

• Added "vitepress-plugin-llms" as a dependency.
• Imported and configured the new llms.txt plugin in the VitePress configuration.

Reviewed Changes

Copilot reviewed 2 out of 4 changed files in this pull request and generated 1 comment.

File	Description
website/client/package.json	Added a new dependency for the llms.txt support.
website/client/.vitepress/config/configShard.ts	Imported and configured the llms.txt plugin.

Comments suppressed due to low confidence (1)

website/client/.vitepress/config/configShard.ts:4

• [nitpick] Consider renaming the import 'llmstxt' to something more descriptive (e.g., 'llmsPlugin') to clearly reflect its functionality.

import llmstxt from 'vitepress-plugin-llms';

[Copilot]

Consider using a versioned package release instead of a URL dependency to ensure stability and easier maintenance in production environments.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

website/client/.vitepress/config/configShard.ts (1)

4-4: Consider using a more descriptive import alias.

The import alias llmstxt doesn't clearly match the package name vitepress-plugin-llms, which could cause confusion for future maintainers.

Consider using a more descriptive alias:

-import llmstxt from 'vitepress-plugin-llms';
+import { default as vitePressLlmsPlugin } from 'vitepress-plugin-llms';

Or simply use the default import without alias:

-import llmstxt from 'vitepress-plugin-llms';
+import vitePressLlms from 'vitepress-plugin-llms';

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e4fe507 and 16493ac.

⛔ Files ignored due to path filters (1)

• website/client/package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (2)

• website/client/.vitepress/config/configShard.ts (2 hunks)
• website/client/package.json (1 hunks)

🔇 Additional comments (1)

website/client/.vitepress/config/configShard.ts (1)

149-151: Plugin configuration looks correct.

The plugin is properly integrated into the Vite plugins array with appropriate configuration. The excludeIndexPage: true option suggests intentional behavior to exclude the index page from llms.txt generation.

Please verify that the excludeIndexPage option aligns with your intended llms.txt generation behavior:

What does the excludeIndexPage option do in vitepress-plugin-llms?

[coderabbitai]

🛠️ Refactor suggestion

🧩 Verification correct

Consider using a published npm package version instead of URL-based dependency.

Using a URL-based package reference (https://pkg.pr.new/vitepress-plugin-llms@d35010c) introduces security and stability concerns:

• External URL dependencies can pose security risks
• Commit-hash based versioning makes it unclear what functionality is being included
• This appears to be a preview/development version, which may not be suitable for production
• Dependency management becomes more complex

Please verify if there's a published npm version available:

---

🏁 Script executed:

#!/bin/bash
# Check if vitepress-plugin-llms is available on npm
npm view vitepress-plugin-llms versions --json 2>/dev/null || echo "Package not found on npm registry"

# Check the specific commit referenced
curl -s "https://api.github.com/repos/okineadev/vitepress-plugin-llms/commits/d35010c" | jq -r '.commit.message // "Commit not found"'

Length of output: 719

---

Consider using a published npm package version instead of URL-based dependency.

Using a URL-based package reference (https://pkg.pr.new/vitepress-plugin-llms@d35010c) introduces security and stability concerns:

• External URL dependencies can pose security risks
• Commit-hash based versioning makes it unclear what functionality is being included
• This appears to be a preview/development version, which may not be suitable for production
• Dependency management becomes more complex

Please verify if there's a published npm version available:

#!/bin/bash
# Check if vitepress-plugin-llms is available on npm
npm view vitepress-plugin-llms versions --json 2>/dev/null || echo "Package not found on npm registry"

# Check the specific commit referenced
curl -s "https://api.github.com/repos/okineadev/vitepress-plugin-llms/commits/d35010c" | jq -r '.commit.message // "Commit not found"'

🤖 Prompt for AI Agents

In website/client/package.json at line 21, the dependency
"vitepress-plugin-llms" is specified using a URL with a commit hash, which can
cause security and stability issues. To fix this, check if a published npm
version of "vitepress-plugin-llms" exists by running npm view commands as
suggested. If available, replace the URL-based dependency with the latest stable
npm version number to ensure better security, clarity, and dependency
management.

[claude]

PR Review: Add llms.txt Support

Thank you for adding llms.txt support to the Repomix documentation site! This is a valuable addition that will help LLMs better understand and navigate the documentation. Here's my comprehensive review:

---

✅ Strengths

1. Clean Implementation: The integration of vitepress-plugin-llms is straightforward and follows VitePress plugin conventions
2. Minimal Configuration: The excludeIndexPage: true option is appropriate for this use case
3. Proper Dependency Management: Added as a devDependency in the correct location

---

🔍 Code Quality & Best Practices

Details

Import Organization (website/client/.vitepress/config/configShard.ts:4)

The import is well-placed and follows the existing import structure. Good alphabetical ordering.

import llmstxt from 'vitepress-plugin-llms';

Plugin Configuration (website/client/.vitepress/config/configShard.ts:158-160)

The plugin is correctly configured within the Vite plugins array. The excludeIndexPage: true option makes sense for this documentation site.

llmstxt({
  excludeIndexPage: true,
}),

---

⚠️ Issues & Recommendations

Details

1. Commit Message Quality

Issue: The commit messages contain multiple "wip" (work in progress) commits, which don't follow the project's Conventional Commits specification.

Impact: This makes it harder to understand the change history and violates the project's commit guidelines (see CLAUDE.md).

Recommendation: Squash the WIP commits into a single, well-formatted commit:

feat(website): Add llms.txt support

Add vitepress-plugin-llms to generate llms.txt file for better LLM documentation navigation.
The plugin is configured to exclude the index page.

---

2. Missing PR Description

Issue: The PR description only contains the checklist without explaining what llms.txt is or why this feature is being added.

Impact: Reviewers and future maintainers won't understand the purpose of this change without additional research.

Recommendation: Add a summary section explaining:

• What llms.txt is (a standard for LLM-friendly documentation)
• Why this is valuable for Repomix
• What the plugin does
• Example: "This PR adds support for llms.txt, a proposed standard that helps LLMs better understand and navigate documentation sites. The vitepress-plugin-llms automatically generates the llms.txt file from our VitePress documentation."

---

3. Missing Tests

Issue: No tests were added for this feature. The website/client directory has no test coverage.

Impact: Can't verify the plugin generates the expected output or that configuration changes don't break it.

Recommendation: While VitePress plugins don't always need extensive testing, consider:

• Adding a build verification test to ensure llms.txt is generated
• Testing that the excludeIndexPage option works correctly
• Example test location: website/client/tests/llmstxt.test.ts

---

4. Auto-generated Schema File

Issue: website/client/src/public/schemas/1.8.0/schema.json appears to be auto-generated but is included in this PR about llms.txt support.

Impact: Mixing unrelated changes makes the PR harder to review.

Recommendation: If this schema generation is automated and unrelated to llms.txt:

• Remove it from this PR
• Create a separate PR for schema updates
• If it's related, explain the connection in the PR description

---

5. Package Lock File Size

Issue: The package-lock.json adds 1,371 lines for a single plugin dependency.

Impact: This seems excessive and suggests the plugin may have a large dependency tree.

Recommendation:

• Verify the plugin's dependency tree: npm ls vitepress-plugin-llms
• Consider if there are lighter alternatives
• Document this as a known tradeoff if accepted

---

6. Documentation

Issue: No documentation about the llms.txt feature was added.

Impact: Users and future maintainers won't know this feature exists or how it works.

Recommendation: Add documentation explaining:

• That the site now supports llms.txt
• Where to access it (likely at /llms.txt)
• What it's used for
• Consider adding this to a "For AI/LLM Users" section if one exists

---

🚀 Performance Considerations

Details

• Build Time: The plugin runs during VitePress build. Monitor for any build time increases.
• Bundle Size: Since this is a dev dependency and build-time plugin, it shouldn't affect client bundle size. ✅
• Runtime Impact: No runtime impact expected. ✅

---

🔒 Security Considerations

Details

• Dependency Security: The plugin is version ^1.8.0. Run npm audit to check for vulnerabilities.
• Content Exposure: llms.txt exposes documentation structure. Ensure no sensitive information is included (appears fine for public docs). ✅
• XSS/Injection: Plugin generates static text files, low risk. ✅

---

📊 Test Coverage

Details

Status: ❌ No tests added

Required Actions:

1. Add integration test verifying llms.txt generation
2. Test the excludeIndexPage option behavior
3. Verify the generated file format meets llms.txt spec

Suggested test structure:

// website/client/__tests__/llmstxt.test.ts
describe('llms.txt generation', () => {
  it('should generate llms.txt file during build', async () => {
    // Build the site
    // Check that dist/llms.txt exists
    // Verify content structure
  });
  
  it('should exclude index page when configured', async () => {
    // Verify index.md is not in llms.txt
  });
});

---

📝 Summary

Overall Assessment: This is a good feature addition with clean implementation, but it needs work on:

1. ✅ Squashing WIP commits into a proper commit message
2. ✅ Adding a comprehensive PR description
3. ⚠️ Adding test coverage (recommended)
4. ⚠️ Adding documentation (recommended)
5. ⚠️ Clarifying the schema.json inclusion
6. ⚠️ Investigating the large dependency tree

Recommendation: Request changes to address commit history, PR description, and schema file inclusion. Tests and documentation are strongly recommended but could be addressed in a follow-up PR if needed.

---

Review generated following the project's CLAUDE.md guidelines