Epic: transport subcommands

[JPHutchins]

This is an epic issue to track the progress of the high level feature "transport subcommands", in both this repo and smp/smpclient.

Briefly, this is necessary because we currently pool transport options across all transports, meaning that it's possible to give transport options to the "wrong transport" - a no-op, but spaghetti code + UX. Instead the options should be sum types on the subcommands so that invalid options are not representable.

smp: https://github.com/JPHutchins/smp
smpclient: https://github.com/intercreate/smpclient

Organization

Breaking changes

Must be merged to a feature branch (on the respective repository) named screaming-goblin.

Other

Can merge to main as usual. Consider a PR or rebase of screaming-goblin after the merge to main to keep the breaking change feature branches relatively ready for merge.

Issues

Issues may be added or edited as work is completed.

smpmgr

• Migrate from Typer -> Cyclopts #68 (we may use argtree instead for speed/shell support, but that depends on time available)
• completion will close RFC: Use URIs for transport selection and config #35
• Switch from poetry to uv #57
• add camas #101
• Breaking: transport subcommands #102
• Feature request: Allow forcing cached GATT services (Bleak/WinRT) #82

smpclient

• breaking (this is the main pre-req): breaking: remove address from SMPClient; SMPTransport owns address smpclient#58
• Bleak transport options smpclient#90
• Remove the generic request/response pattern smpclient#124

smp

• Switch from poetry to uv JPHutchins/smp#47
• Use camas as runner instead of envr JPHutchins/smp#61
• Add coverage for python 3.15 JPHutchins/smp#60
• Add coverage for macos-15-intel & ubuntu arm runners JPHutchins/smp#53
• SMP message excludes fields from the SMP data JPHutchins/smp#45
  • update the existing impl: breaking: header and smp_data in Frame (related to #45) JPHutchins/smp#46
• Move generic request/response from smpclient to here JPHutchins/smp#62

[JPHutchins]

Warning

LLM Disclosure

This comment was authored by claude-opus-4-8 on behalf of @JPHutchins, summarizing recent smp progress on this epic and confirming the cross-repo staging plan.

smp — on screaming-goblin

• Frame[T]/Data on msgspec (breaking: rework SMP messages to Frame[T]/Data on msgspec (#45, #26, #5) JPHutchins/smp#65) — an SMP message is now Frame[T] = (Header, Data[T]); de/serialization moved from pydantic + cbor2 to msgspec + msgspec-cbor.
• Generic request/response moved into smp (Move generic request/response from smpclient to here JPHutchins/smp#62) — SMPRequest binds each request to its Response/ErrorV1/ErrorV2; the TypeIs narrowing helpers live in smp's static-typing tests as the consumer contract rather than shipping from the lib. This unblocks Remove the generic request/response pattern smpclient#124.
• custom grp: Import custom groups #45 (SMP data may contain fields named header/version/sequence/smp_data) — fixed by the Frame/Data split; regression coverage added in test: regression for #45 — SMP payload may contain header/version/sequence/smp_data JPHutchins/smp#69. Closes via the release PR breaking: screaming-goblin JPHutchins/smp#66.

smp — landed on main (non-breaking), rebased into screaming-goblin

• Python 3.15 (Add coverage for python 3.15 JPHutchins/smp#60, closed) — CI now matrixes 3.15, with the GHA python axis sourced from camas (camas matrix --github-matrix). The earlier import failure was a stale-uv alpha predating a CPython C-API addition (PEP 820), not an smp/cbor2 bug — closed upstream at 6.1.3 fails to import on CPython 3.15.0a2: undefined symbol PyType_FromSlots (no pure-Python fallback) agronholm/cbor2#328.

Cross-repo staging

To migrate all three repos in lockstep without cutting an smp release first: smpclient's screaming-goblin branch will pin smp's screaming-goblin (git dependency). smpclient — then smpmgr — build against the new smp API while it's still on the branch; when the breaking set is complete and downstreams are ready, smp cuts the release and the pins move to the released version.

[JPHutchins]

Warning

LLM Disclosure

This comment was posted by claude-opus-5[1m] on behalf of @JPHutchins, who asked for the smpclient phase of this epic to be kicked off — audit the outstanding issues, get as many breaking changes staged as possible, and record the disposition here before signing off. Everything below is either merged, filed, or a decision @JPHutchins settled during the session.

smpclient phase: kicked off, groundwork landed

Two PRs merged to main (both non-breaking, so per this epic's own rule they went to main rather than screaming-goblin), six issues filed, and the design questions blocking the port are now settled.

Merged

PR	What
intercreate/smpclient#127	pyright locked alongside mypy (typecheck = Parallel(mypy, pyright)), camas 0.1.18 → 0.1.29, camas MCP/Claude wiring regenerated. Fixed 19 real defects pyright found in code mypy already passed.
intercreate/smpclient#128	Fixed a ~3% integration flake that was failing CI on unrelated PRs.

Why pyright first, and what it found

The port rewrites the request path wholesale, so it is far cheaper to have both checkers gating before it starts than to retrofit one after. All 19 findings were genuine:

• @override applied to the property object, not the function — 7 properties across 5 transports. mypy accepts it; pyright rejects it. Verified empirically that @property outermost is what both accept.
• bytes/bytearray promotion. mypy silently promotes bytearray → bytes; the typing spec deprecated that. Configuring pyright to match mypy was tried and rejected — it makes case bytes() look exhaustive when at runtime it does not match a bytearray, and duly broke a real assert_never. The four boundaries are now stated honestly instead.
• bleak 3 drift — BleakGATTCharacteristic moved off the top level; the backend predicates needed a union type; and the Windows MTU workaround turns out to be legitimate (bleak's own getter accepts an int "for backwards compatibility"), so only the type error is suppressed.

camas matrix green on Python 3.10–3.14.

The flake, because the root cause is a real protocol limitation

test_upload_to_mcuboot_recovery[mps2_an385.serial_recovery_raw-raw], failing ~1 in 30.

The raw transport writes a 1024 B chunk as a single burst. While the server is still flashing the previous chunk its UART RX pool overruns, and the overflow is silently dropped — the raw protocol is length-prefixed with no delimiter and no CRC, so the server waits forever for a tail that never arrives and the client eats its whole timeout.

Measuring the latency distribution proved it is not "qemu is slow": passing runs peaked at 0.31 s, the failing run sat at 15.001 s, with nothing in between. Raising the timeout would only have made it hang longer.

The fix paces writes with wall-clock delay; an asyncio.sleep(0) yield measured no better than no pacing at all, because the guest needs real CPU time. The first version of the fix paced every socket chardev and regressed a different fixture (qemu_cortex_m0, 2/15 → the encoded transport already self-paces), so the final version scopes pacing to the raw transport by construction.

Settled design decisions

• smp does not ship the narrowers. smp#62 merged as smp#67; screaming-goblin is at 6bce38b. It exports smp.SMPRequest (in smp/__init__.py — there is no smp.generics module, despite smp#67's PR body describing one) and bakes _Response/_ErrorV1/_ErrorV2 onto every request. It deliberately does not ship success/error/error_v1/error_v2 or TRep/TEr1/TEr2 — smp's own typing test says so outright: "without the smp library having to ship them."
• Remove the generic request/response pattern smpclient#124 therefore means: delete smpclient/requests/** outright, take SMPRequest from smp, and keep the four narrowers in smpclient.generics. A deprecated re-export layer was considered and rejected — this is the breaking release, so it breaks here. Downstream absorbs smp's own names, which differ (GroupCountRequest, not CountSupportedGroups).
• SMPClient.connect() is going away, and lifecycle becomes composable context-manager brackets in the spirit of Haskell effects. That dissolves the "MCUmgr params must be opt-in" requirement as a separate question: negotiation becomes a scope the caller enters, not a negotiate=True kwarg. Shapes breaking: remove address from SMPClient; SMPTransport owns address smpclient#58.
• The generated-docstring machinery dies for free: deleting the request layer removes the only reason docs/_generate_requests_docstrings.py exists, along with its step in two workflows, seven ::: smpclient.requests.* pages, and docs/requests.md.

Filed for durable context

#	Title	Note
intercreate/smpclient#129	Raw serial transport desyncs permanently when a receive is cancelled	Buffers into a local bytearray, so a request timeout discards bytes already drained from the OS buffer. Blocks #56 retransmit from ever working on this transport.
intercreate/smpclient#130	send() discards write()'s return value	A short write truncates an SMP message silently.
intercreate/smpclient#131	Raw transport has no write pacing	The production half of #128 — USB CDC ignores baudrate, so real hardware has the same exposure.
intercreate/smpclient#132	qemu_cortex_m0 flakes under full-suite load	Ruled out as caused by #128 (15/15 both trees in isolation).
intercreate/smpclient#133	Remove pydantic from mcuboot.py	Deferred — see below.
intercreate/smpclient#134	Gate gaps: no format_check in CI, pyright warnings don't fail	Both left out of #127 to keep it single-purpose.

The pydantic question, resolved

smpclient has never declared pydantic — it arrives transitively via smp 4.x. Once the port pins smp @ screaming-goblin, pydantic leaves the dependency tree, and mcuboot.py is its only remaining importer, so import smpclient.mcuboot would break.

intercreate/smpclient#133 removes that need, using NamedTuple — stdlib, no type-creation cost, frozen by nature, and it takes methods. All six classes in mcuboot.py convert directly. (@JPHutchins first considered awaiting salix, a C-backed Struct base class, rather than accept stdlib dataclasses; NamedTuple settles it without waiting for a release.)

So the port does not need a temporary pydantic declaration. #133 can land on main ahead of it.

Next

The port itself, as one PR on screaming-goblin — request() onto Frame[T], the request-layer deletion, the size-math rework onto msgspec.structs.replace, docs teardown, and the test rework. Then intercreate/smpclient#58 and its effect-stack lifecycle.

Note that intercreate/smpclient#124 is currently closed, so the scope recorded on its thread should be carried in the port PR's own description rather than relied on there.