docs: document Markdown plugins

Documents the `markdown` option, the Shiki plugin options, and how
generators declare their pipelines. The changeset also lists the
`@doc-kit/core` modules that are removed or moved.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

Author: ovflowd

.changeset/markdown-plugins.md

@@ -0,0 +1,7 @@
+---
+'@doc-kit/core': minor
+'@doc-kit/generator-react': minor
+'@node-core/doc-kit-legacy': patch
+---
+
+Add a `markdown` option to add remark, rehype, and recma plugins to the generators processing Markdown, or configure the ones they use, such as Shiki. The `@doc-kit/core` modules the generators' pipelines replace are removed (`utils/remark.mjs`, `utils/remark-shiki.mjs`, and `utils/highlighter.mjs`), and `utils/type-annotations` moves to `plugins/type-annotations`

docs/configuration.md

@@ -123,9 +123,11 @@ Everything under the `global` key applies to every generator:
   a URL or path to parse, or a pre-parsed array. **Default:** `[]`
   (single-version output).
 - `index` {string|URL|Array} Index URL.
+- `markdown` {Object} [Markdown plugins](#markdown-plugins) for every
+  generator processing Markdown.
 
 A generator's own section (e.g., `html`, `legacy-json`) can override any of
-these for that generator alone.
+these for that generator alone, except `markdown`, which it adds to.
 
 ## Execution options
 
@@ -155,6 +157,96 @@ export default {
 };
 ```
 
+## Markdown plugins
+
+`doc-kit` processes Markdown with [unified](https://unifiedjs.com/).
+`global.markdown` adds plugins to every generator processing Markdown, and a
+generator's own `markdown` adds plugins to it alone, after the global ones.
+
+```mjs displayName="doc-kit.config.mjs"
+export default {
+  global: {
+    markdown: {
+      remarkPlugins: ['remark-math'],
+    },
+  },
+
+  // The generator rendering the site's pages
+  'jsx-ast': {
+    markdown: {
+      rehypePlugins: [
+        'rehype-katex',
+        ['./plugins/rehype-diagrams.mjs', { theme: 'neutral' }],
+      ],
+    },
+  },
+};
+```
+
+Each plugin is a module specifier (a package name, or a path relative to the
+configuration file), alone or in a `[specifier, options]` pair. The module
+default-exports the plugin, a list of plugins, or a unified preset. Markdown is
+processed in worker threads, which import the plugins themselves, so options
+must be serializable: configure a plugin taking functions in a module of your
+own.
+
+- `remarkPlugins` {Array} The global ones run on each document once the `ast`
+  generator parses it, so every output sees their changes. A generator
+  rendering Markdown, such as `jsx-ast`, runs its own on what it renders.
+- `rehypePlugins` {Array} Run on the HTML of the generators rendering
+  Markdown, such as `jsx-ast`, before code is highlighted.
+- `recmaPlugins` {Array} Run on the JavaScript `jsx-ast` compiles the pages to.
+
+A generator only takes the plugins its pipeline has a place for: `jsx-ast`
+takes all three kinds, `ast`, `metadata`, and `json` take remark plugins, and
+generators such as `html` take none. Plugins listed for a generator that
+doesn't take them are ignored with a warning.
+
+A generator runs each plugin once: listing one it already has, its own or a
+global one, configures it instead, merging the options (arrays add up, and
+objects merge).
+
+The plugins configured for `jsx-ast` don't apply to the fragments it renders
+on their own: types, parameter descriptions, and change history notes.
+
+### Code highlighting
+
+`jsx-ast` highlights code with the `@doc-kit/core/plugins/shiki/rehype.mjs`
+plugin, which runs [Shiki](https://shiki.style/) with every language it
+bundles. To configure it, list it with options in
+`jsx-ast.markdown.rehypePlugins` (with pnpm, add `@doc-kit/core` to your
+dependencies, so your configuration file can resolve it):
+
+- `langs` {Array} More languages: grammars, or modules default-exporting them.
+- `langAlias` {Object} Aliases of languages, such as `{ conf: 'ini' }`.
+- `themes` {Object} The `light` and `dark` themes: names of themes Shiki
+  bundles, modules default-exporting themes, or themes.
+- `transformers` {Array} Modules default-exporting
+  [Shiki transformers](https://shiki.style/guide/transformers), or lists of
+  them.
+
+These modules are paths relative to the working directory, or URLs, such as
+the ones `import.meta.resolve()` gives:
+
+```mjs displayName="doc-kit.config.mjs"
+export default {
+  'jsx-ast': {
+    markdown: {
+      rehypePlugins: [
+        [
+          '@doc-kit/core/plugins/shiki/rehype.mjs',
+          {
+            langAlias: { conf: 'ini' },
+            themes: { light: 'github-light', dark: 'github-dark' },
+            transformers: [import.meta.resolve('./shiki-transformers.mjs')],
+          },
+        ],
+      ],
+    },
+  },
+};
+```
+
 ## Configuration Merging
 
 Configurations are merged in the following order (higher sources take

docs/creating-generators.md

@@ -468,6 +468,64 @@ Use a dependent when a generator transforms an intermediate representation
 format of its own. See the [`section-pages`](./generators/section-pages.md) generator for
 a worked example.
 
+## Markdown pipelines
+
+A generator processing Markdown declares its [unified](https://unifiedjs.com/)
+pipeline as `markdown`, listing plugins as the
+[`markdown` option](./configuration.md#markdown-plugins) does, with paths
+relative to its module. The rehype list starts with `remark-rehype`, and the
+recma list with `rehype-recma`. Each list takes the configured plugins in place
+of its `'...'`, and a list without one takes none. As every thread running the
+generator imports it, options may be anything, functions included.
+
+```mjs displayName="index.mjs"
+export default {
+  name: 'my-generator',
+
+  dependsOn: '@doc-kit/core/metadata',
+
+  markdown: {
+    remarkPlugins: ['remark-parse', 'remark-gfm', '...'],
+    rehypePlugins: [
+      ['remark-rehype', { allowDangerousHtml: true }],
+      '...',
+      ['rehype-stringify', { allowDangerousHtml: true }],
+    ],
+  },
+
+  generate,
+};
+```
+
+`getProcessor(name)` gives the generator's processor, on each thread it runs
+on:
+
+```mjs displayName="generate.mjs"
+import { getProcessor } from '@doc-kit/core/utils/markdown/processor.mjs';
+
+export async function generate(input) {
+  const processor = getProcessor('my-generator');
+
+  return Promise.all(
+    input.map(async ({ content }) =>
+      processor.stringify(await processor.run(content))
+    )
+  );
+}
+```
+
+`getProcessor(name, { mdx: true })` returns an MDX processor (plugins can
+check `this.data('mdx')`), and `getProcessor(name, { configured: false })` one
+without the configured plugins. A generator rendering Markdown (with rehype or
+recma plugins) only takes its own remark plugins, as the global ones already
+ran when `ast` parsed the Markdown.
+
+A plugin needing asynchronous setup can export an async `load(options)`
+returning the plugin, instead of a default export. The Shiki plugin does, and
+`getHighlighter(name)` (from `@doc-kit/core/plugins/shiki/highlighter.mjs`)
+returns the highlighter of a generator's Shiki plugin, to highlight code as its
+pipeline does.
+
 ## File Output
 
 ### Writing Output Files