cms-utils

A typed Python toolkit for CMS inventory, normalization, metadata audits, and safe WordPress → HubSpot migration planning.

Project status

cms-utils is a portfolio and reference implementation. It demonstrates the read-side and migration-planning side of a CMS content toolkit. The current supported workflow is:

WordPress content
    ↓  read-only adapter
Normalized models
    ↓  audit + transformation
WordPress → HubSpot dry-run
    ↓  receipts, checkpoints, reports

Real destination writes are intentionally outside the current release. The codebase is not under active maintenance; see CHANGELOG.md and docs/adr/0005-current-scope-and-non-goals.md.

What it demonstrates

• Typed integration boundaries — provider adapters never leak provider dicts into the migration engine
• Defensive HTTP — retries, rate limits, secret redaction in one shared client (cms_utils.http.BaseHTTPClient)
• Deterministic content transformations — slug normalization, HTML stripping, status mapping, all exercised by the demo
• Safe migration planning — dry-run by default, with receipts and checkpoints
• Resumable workflows — per-item checkpoints rewritten after every item
• Testable provider adapters — respx makes wire-level behavior unit-testable
• Secret-safe logging — every log handler installs a SecretRedactor filter

The serious implementation files are short and self-contained; reviewers can read the whole thing in an afternoon.

The vertical slice

The five-minute reviewer path:

git clone https://github.com/jenksed/cms-utils
cd cms-utils
make demo

The demo parses a hand-written WordPress fixture, runs the audit and metadata suite, applies the WordPress → HubSpot mapping, runs the migration engine in dry-run mode, and writes thirteen deterministic artifacts under demo-output/. The full set is byte-stable across runs.

demo-output/
├── audit-findings.json
├── audit-summary.md
├── checkpoint.json
├── demo.summary.json
├── failures.csv
├── inventory.csv
├── inventory.json
├── migration-plan.csv
├── migration-plan.json
├── migration-summary.json
├── migration-summary.md
└── receipts.json

A small curated sample of expected outputs is committed under examples/expected-output/ so the artifact set can be inspected without running anything.

Architecture (one diagram)

flowchart TD
    A["Provider adapters<br/>(WordPress · HubSpot)"] --> B
    B["Normalized models<br/>ContentItem · MigrationReceipt"]
    B --> C
    C["Audit · comparison · transformation"]
    C --> D
    D["Migration engine<br/>(dry-run by default)"]
    D --> E
    E["Receipts · checkpoints · reports"]

Loading

The architectural rules:

1. Adapters convert provider payloads ↔ normalized models. The engine never sees a provider dict.
2. The migration engine defaults to dry-run. Real writes are an explicit --no-dry-run opt-in.
3. Checkpoints are rewritten after every item; receipts survive any interruption.
4. Provider-specific structure stays under ContentItem.metadata["wp_meta"] (etc.). Core code ignores it.

Each rule has an Architectural Decision Record under docs/adr/.

CLI examples

cms-utils version                                  # 0.2.0
cms-utils doctor                                   # masked-config dump
cms-utils validate exports/wp.json                 # shape-check an export
cms-utils wordpress inventory --format json        # live, but safe to read
cms-utils hubspot inventory                        # same on HubSpot
cms-utils audit metadata --file exports/wp.json    # metadata audit on a fixture
cms-utils audit links    --file exports/wp.json \
              --base-url https://example.com        # broken-link check
cms-utils compare wordpress-hubspot \
              --wordpress-export exports/wp.json \
              --hubspot-export exports/hs.json       # diff between inventories
cms-utils migrate wordpress-hubspot \
              --mapping config/mapping.example.yaml \
              --dry-run                              # the headline command

The --dry-run flag is on by default. Real writes require --no-dry-run and are not implemented in this release.

Safety and failure handling

Hazard	Mitigation
Bearer tokens in logs	SecretRedactor filter on every console and file handler
Permanent 4xx errors	Typed exceptions; engine records FAILED receipt and continues
Transient 5xx / 429	Exponential backoff with jitter; honors Retry-After
Long migrations interrupted	Per-item checkpoint rewrite; --resume continues from the last completed external id
Unknown configuration	ConfigurationError with the missing variable listed by name
Mutation on a live destination	Dry-run is the default; --no-dry-run required for any write

Repository structure

cms-utils/
├── README.md
├── LICENSE
├── pyproject.toml
├── CHANGELOG.md
├── ROADMAP.md
├── Makefile
├── .env.example
├── .github/workflows/ci.yml        # ruff · ruff-check · mypy · pytest
├── docs/
│   ├── architecture.md
│   ├── technical-tour.md
│   ├── portfolio-readiness-audit.md
│   ├── extending-cms-utils.md
│   ├── resuming-development.md
│   ├── source-audit.md
│   ├── migration-model.md
│   ├── provider-adapters.md
│   ├── retirement-plan.md
│   └── adr/
│       ├── 0001-normalized-content-model.md
│       ├── 0002-provider-boundaries.md
│       ├── 0003-dry-run-before-writes.md
│       ├── 0004-checkpoints-and-receipts.md
│       └── 0005-current-scope-and-non-goals.md
├── examples/
│   ├── README.md
│   ├── fixtures/wp-pages.json
│   └── expected-output/
│       ├── audit-summary.md
│       ├── failures.csv
│       ├── inventory.json
│       ├── migration-summary.json
│       └── migration-summary.md
├── src/cms_utils/                  # the package
└── tests/unit/                    # 169 unit tests

Testing and validation

make test          # pytest tests/unit/  ->  169 tests in ~1 second
make coverage      # term-missing coverage report  ->  ~81% line coverage
make verify        # format + lint + type + test + demo

What is not in this release

The release deliberately excludes:

• Real destination writes. HubSpot and WordPress write endpoints are not implemented. The engine's HubSpotDryRunAdapter is a fixture.
• HubSpot blog support. cms_utils.hubspot.blogs raises NotImplementedError.
• Browser-driven audits. No Playwright, no Selenium. The audit module is offline-only.
• Pantheon / Terminus integration. The empty cms_utils.integrations.pantheon package is a documented extension point only.

Each of these is named in ROADMAP.md with the prerequisites that would be needed before it makes sense to start.

Extension model

The architecture supports new providers without engine changes. See docs/extending-cms-utils.md for a 60-line provider skeleton and a list of which files a new adapter would touch.

Key invariants:

• Adapters never see provider dicts in core code.
• Migration engine never inspects provider-specific metadata.
• HTML transforms are static and registered by name in _RECOGNIZED_HTML_TRANSFORMS.

Project history

cms-utils consolidates and reimplements ideas from several earlier personal CMS utilities. Those predecessor projects are no longer required to understand or run this repository. This repository is the canonical version of the work.

The ideas it draws on include:

• hs_utils — HubSpot CMS API client
• wp_utils — WordPress REST API client
• pantheon-scripts — Pantheon automation scripts
• sitesage — site audit tooling
• rh-selenium-crawler — one-off scraper

Their code was rewritten here rather than copied; the audit document at docs/source-audit.md records what was reused, rewritten, or rejected for each upstream. License attribution where applicable is preserved in LICENSE.

The related-content idea (a WordPress plugin for surfacing related content) was an inspiration for cms_utils.audit.comparison.related_items, but the plugin itself remains an independent project.

License

MIT — see LICENSE.