Skip to content

Commit 5d62e85

Browse files
authored
ci: add markdown, website, and image lint checks (#882)
Adds Markdown, website, and image lint checks to CI, mirroring the setup in `apify-docs` and `apify-core`. The docs and website now go through the same kind of automated checks as the Python code. The website tooling moves from eslint + prettier (which weren't wired into CI) to [oxlint](https://oxc.rs/) and [oxfmt](https://oxc.rs/), and [markdownlint](https://github.com/DavidAnson/markdownlint) now covers the Markdown files. Three jobs run in the existing Checks workflow: - **Markdown lint** — `markdownlint` over `README.md`, `CONTRIBUTING.md`, and the `docs/` folder. - **Website lint and format** — `oxlint` plus an `oxfmt --check` formatting gate for the Docusaurus site. - **Image lint** — fails when a PR adds unoptimized raster images, which should be converted with `pnpm opt:images` first. Notes: - The website JS keeps single quotes, set via `.editorconfig` (`quote_type`) and `.oxfmtrc.json` (`singleQuote`). - The markdownlint config matches `apify-docs`: line length is disabled and inline HTML is allowed. - Minor doc and code fixes were applied where needed to pass the new checks. Closes #594
1 parent 9c266f2 commit 5d62e85

21 files changed

Lines changed: 868 additions & 2353 deletions

.editorconfig

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ charset = utf-8
77
trim_trailing_whitespace = true
88
insert_final_newline = true
99
end_of_line = lf
10+
quote_type = single
1011

1112
[Makefile]
1213
indent_style = tab

.github/workflows/_checks.yaml

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ permissions:
1717

1818
env:
1919
PYTHON_VERSION: 3.14
20+
NODE_VERSION: 24
2021

2122
jobs:
2223
actions_lint_check:
@@ -72,6 +73,86 @@ jobs:
7273
- name: Async docstrings check
7374
run: uv run poe check-docstrings
7475

76+
markdown_lint_check:
77+
name: Markdown lint check
78+
runs-on: ubuntu-latest
79+
steps:
80+
- name: Checkout repository
81+
uses: actions/checkout@v7
82+
83+
- name: Set up Node
84+
uses: actions/setup-node@v6
85+
with:
86+
node-version: ${{ env.NODE_VERSION }}
87+
88+
- name: Install pnpm and website dependencies
89+
uses: apify/actions/pnpm-install@v1.2.0
90+
with:
91+
working-directory: website
92+
93+
- name: Lint Markdown
94+
run: pnpm lint:md
95+
working-directory: website
96+
97+
website_lint_check:
98+
name: Website lint check
99+
runs-on: ubuntu-latest
100+
steps:
101+
- name: Checkout repository
102+
uses: actions/checkout@v7
103+
104+
- name: Set up Node
105+
uses: actions/setup-node@v6
106+
with:
107+
node-version: ${{ env.NODE_VERSION }}
108+
109+
- name: Install pnpm and website dependencies
110+
uses: apify/actions/pnpm-install@v1.2.0
111+
with:
112+
working-directory: website
113+
114+
- name: Lint website code
115+
run: pnpm lint:code
116+
working-directory: website
117+
118+
- name: Check website formatting
119+
run: pnpm format:check
120+
working-directory: website
121+
122+
image_lint_check:
123+
name: Image lint check
124+
runs-on: ubuntu-latest
125+
steps:
126+
- name: Checkout repository
127+
uses: actions/checkout@v7
128+
with:
129+
fetch-depth: 0
130+
131+
# Doc images must be committed as optimized `.webp`. This fails when a PR adds raster
132+
# images in another format so they get converted via `pnpm opt:images` first.
133+
- name: Get changed unoptimized images
134+
id: changed-files
135+
uses: tj-actions/changed-files@v47
136+
with:
137+
files: |
138+
docs/**/*.{png,jpg,jpeg,gif,bmp,tif,tiff,avif}
139+
website/static/**/*.{png,jpg,jpeg,gif,bmp,tif,tiff,avif}
140+
separator: "\n"
141+
142+
- name: Fail on unoptimized images
143+
if: steps.changed-files.outputs.any_changed == 'true'
144+
env:
145+
UNOPTIMIZED_IMAGE_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
146+
run: |
147+
echo "Unoptimized images detected! Convert each one to WebP, e.g.:"
148+
echo ""
149+
while IFS= read -r file_path; do
150+
echo " (cd website && pnpm opt:images \"../$file_path\")"
151+
done <<< "$UNOPTIMIZED_IMAGE_FILES"
152+
echo ""
153+
echo "Then reference the resulting .webp files in your Markdown."
154+
exit 1
155+
75156
unit_tests:
76157
name: Unit tests
77158
if: inputs.run_tests

.github/workflows/manual_release_docs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ permissions:
2323
contents: read
2424

2525
env:
26-
NODE_VERSION: 22
26+
NODE_VERSION: 24
2727
PYTHON_VERSION: 3.14
2828

2929
jobs:

.github/workflows/manual_version_docs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ permissions:
2727
contents: read
2828

2929
env:
30-
NODE_VERSION: "22"
30+
NODE_VERSION: "24"
3131
PYTHON_VERSION: "3.14"
3232

3333
jobs:

.markdownlint.yaml

Lines changed: 28 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,30 @@
1+
# markdownlint config for the docs and top-level Markdown files.
2+
# Run via `pnpm lint:md` / `pnpm lint:md:fix` from the `website/` directory.
13
default: true
2-
line-length:
3-
line_length: 150
4-
ul-style: dash
4+
5+
# Prose is written one sentence per line, so line length is not enforced.
6+
line-length: false
7+
8+
ul-style:
9+
style: dash
10+
11+
# Nested unordered lists use 4-space indentation.
12+
ul-indent:
13+
indent: 4
14+
15+
# Docs are MDX and embed JSX components.
516
no-inline-html: false
17+
18+
# MDX pages set their title via front matter, so multiple/duplicate H1s are fine.
19+
single-title: false
20+
no-duplicate-heading:
21+
siblings_only: true
22+
23+
# Anchor links into other pages can't be validated locally.
24+
link-fragments: false
25+
26+
no-bare-urls: false
27+
no-trailing-punctuation:
28+
punctuation: ".,;:。,;:"
29+
no-multiple-blanks:
30+
maximum: 2

.rules.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ Integration tests require `APIFY_TEST_USER_API_TOKEN` and `APIFY_TEST_USER_2_API
3535

3636
`ApifyClient`/`ApifyClientAsync` are the entry points. They provide methods that return resource sub-clients:
3737

38-
```
38+
```text
3939
ApifyClient
4040
├── .actor(id) → ActorClient (single resource operations)
4141
├── .actors() → ActorCollectionClient (list/create)
@@ -80,8 +80,8 @@ Each input-side TypedDict ships in two casings: snake_case (`RequestDict`) and c
8080
- Generated by `datamodel-code-generator` from the OpenAPI spec at `https://docs.apify.com/api/openapi.json` (config in `pyproject.toml` under `[tool.datamodel-codegen]`, aliases in `datamodel_codegen_aliases.json`)
8181
- After generation, `scripts/postprocess_generated_models.py` is run to apply additional fixes
8282
- To regenerate locally:
83-
- From the live published spec: `uv run poe generate-models`
84-
- From a local spec file: `uv run poe generate-models-from-file path/to/openapi.json`
83+
- From the live published spec: `uv run poe generate-models`
84+
- From a local spec file: `uv run poe generate-models-from-file path/to/openapi.json`
8585
- In CI, model regeneration is triggered automatically by the `apify/apify-docs` repo when its OpenAPI spec changes (workflow `manual_regenerate_models.yaml`). It downloads the pre-built `openapi-bundles` artifact from the apify-docs workflow run, opens a PR titled `[TODO]: update generated models from apify-docs PR #N`, assigns it to the docs PR author, and posts a cross-repo comment on the original apify-docs PR
8686
- Manual regeneration is also possible from the GitHub Actions UI (`Regenerate models` workflow)
8787

@@ -93,12 +93,12 @@ Each input-side TypedDict ships in two casings: snake_case (`RequestDict`) and c
9393
- **Docstrings**: Google style format, single backticks for inline code references (`` `ApifyClient` `` not ``` ``ApifyClient`` ```)
9494
- **Imports**: `from __future__ import annotations` used throughout
9595
- **Commits**: [Conventional Commits](https://www.conventionalcommits.org/) format. Choose the type based on *what* changed, not just *why*:
96-
- `feat:` / `fix:` / `perf:` / `refactor:` / `style:`**source code only**; these trigger a release and appear in the changelog
97-
- `test:` — test additions or changes (no release triggered)
98-
- `docs:` — documentation changes; also triggers a doc release on master
99-
- `ci:` — CI/workflow changes
100-
- `chore:` — dependency bumps, tooling, generated files (e.g. model regeneration PRs), and other housekeeping
101-
- `build:` — build system changes
96+
- `feat:` / `fix:` / `perf:` / `refactor:` / `style:`**source code only**; these trigger a release and appear in the changelog
97+
- `test:` — test additions or changes (no release triggered)
98+
- `docs:` — documentation changes; also triggers a doc release on master
99+
- `ci:` — CI/workflow changes
100+
- `chore:` — dependency bumps, tooling, generated files (e.g. model regeneration PRs), and other housekeeping
101+
- `build:` — build system changes
102102

103103
## Testing
104104

@@ -108,6 +108,7 @@ Each input-side TypedDict ships in two casings: snake_case (`RequestDict`) and c
108108
- **Parallelism**: Tests run in parallel via `pytest-xdist`.
109109

110110
Unit test pattern:
111+
111112
```python
112113
def test_example(httpserver: HTTPServer) -> None:
113114
httpserver.expect_request('/v2/endpoint').respond_with_json({'data': ...})

CONTRIBUTING.md

Lines changed: 28 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,22 @@ To run the documentation locally (requires Node.js):
121121
uv run poe run-docs
122122
```
123123

124+
### Linting the docs and website
125+
126+
Markdown content (this guide, `README.md`, and the `docs/` folder) is checked with
127+
[markdownlint](https://github.com/DavidAnson/markdownlint). The Docusaurus website code is linted
128+
with [oxlint](https://oxc.rs/) and formatted with [oxfmt](https://oxc.rs/). All of them run in CI.
129+
To run them locally (requires Node.js 22.12 or newer and pnpm), from the `website/` directory:
130+
131+
```sh
132+
pnpm lint # lint Markdown and website code
133+
pnpm lint:fix # auto-fix both
134+
pnpm format # format the website code
135+
```
136+
137+
Doc images are committed as optimized `.webp`. To convert a new image, run
138+
`pnpm opt:images <path-to-image>` from the `website/` directory.
139+
124140
## Commits
125141

126142
We use [Conventional Commits](https://www.conventionalcommits.org/) format for commit messages. This convention is used to automatically determine version bumps during the release process.
@@ -152,25 +168,22 @@ Publishing new versions to [PyPI](https://pypi.org/project/apify_client) is auto
152168

153169
1. **Do not do this unless absolutely necessary.** In all conceivable scenarios, you should use the `release` workflow instead.
154170
2. **Make sure you know what you're doing.**
171+
3. Update the version number by modifying the `version` field under `project` in `pyproject.toml`:
155172

156-
3. Update the version number:
157-
158-
- Modify the `version` field under `project` in `pyproject.toml`.
159-
160-
```toml
161-
[project]
162-
name = "apify_client"
163-
version = "x.z.y"
164-
```
173+
```toml
174+
[project]
175+
name = "apify_client"
176+
version = "x.z.y"
177+
```
165178

166179
4. Build the package:
167180

168-
```sh
169-
uv run poe build
170-
```
181+
```sh
182+
uv run poe build
183+
```
171184

172185
5. Upload to PyPI:
173186

174-
```sh
175-
uv publish --token YOUR_API_TOKEN
176-
```
187+
```sh
188+
uv publish --token YOUR_API_TOKEN
189+
```

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ For end-to-end recipes — passing input, managing tasks for reusable input, ret
166166
The full documentation lives at **[docs.apify.com/api/client/python](https://docs.apify.com/api/client/python)**.
167167

168168
| Section | What you'll find |
169-
|---|---|
169+
| --- | --- |
170170
| [Introduction](https://docs.apify.com/api/client/python/docs) | Overview, prerequisites, and a tour of the client. |
171171
| [Quick start](https://docs.apify.com/api/client/python/docs/quick-start) | Authenticate, run an Actor, and fetch its results step by step. |
172172
| [Concepts](https://docs.apify.com/api/client/python/docs/concepts/asyncio-support) | Asyncio, single vs. collection clients, nested clients, error handling, retries, logging, convenience methods, pagination, streaming, custom HTTP clients, timeouts. |

docs/02_concepts/04_error_handling.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ When you use the Apify client, it automatically extracts all relevant data from
1919
The Apify client provides dedicated error subclasses based on the HTTP status code of the failed response, so you can branch on error kind without inspecting `status_code` or `type`:
2020

2121
| Status | Subclass |
22-
|---|---|
22+
| --- | --- |
2323
| 400 | <ApiLink to="class/InvalidRequestError">`InvalidRequestError`</ApiLink> |
2424
| 401 | <ApiLink to="class/UnauthorizedError">`UnauthorizedError`</ApiLink> |
2525
| 403 | <ApiLink to="class/ForbiddenError">`ForbiddenError`</ApiLink> |

docs/02_concepts/10_custom_http_clients.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ It must return an object satisfying the <ApiLink to="class/HttpResponse">`HttpRe
7878
<ApiLink to="class/HttpResponse">`HttpResponse`</ApiLink> is not a concrete class. Any object with the following attributes and methods will work:
7979

8080
| Property / Method | Description |
81-
|---|---|
81+
| --- | --- |
8282
| `status_code: int` | HTTP status code |
8383
| `text: str` | Response body as text |
8484
| `content: bytes` | Raw response body |

0 commit comments

Comments
 (0)