A Collection has no way to say a member is conditional

[cboulay]

Raising this as a problem statement rather than a proposal. It shows up in several places once you have Collections whose shape depends on settings, and I suspect the right answer needs someone to look across the ezmsg-org packages as a whole rather than patch the symptom I happened to hit.

The situation

A Collection declares its members as class attributes. CollectionMeta.__init__ harvests them into __components__, and Collection.__init__ deepcopies that into the per-instance _components (collection.py:64). That set is fixed by the time anything else runs.

But network() is free to wire only some of them. A Collection whose topology depends on its settings — an optional processing tail, an optional backend conversion, N device slots of which one is populated — has to declare every member it might use, and then wire a subset.

There is currently no way to express "this member is only part of the graph under these settings." The declaration is unconditional; only the wiring is conditional.

What follows from that

A declared-but-unwired unit is not inert. It is instantiated, added to the process list, and gets setup() → initialize() called like any other. For BaseTransformerUnit that means create_processor() runs, which asserts isinstance(self.SETTINGS, settings_type). A Collection that sensibly calls apply_settings on a member only when it intends to use that member therefore fails at startup:

AssertionError: Settings must be of type <class '...EffectorAssistSettings'>

The message names the type it wanted and nothing else — not the unit, not the Collection. We shipped this bug: an optional shared-control unit, declared unconditionally, configured and wired only if self._use_assist, broke every configuration that left the feature off by default.

configure() is too late to help. The crawl order in backend._setup is gather_edges → network() (backend.py:270), then collect_processes (:297), then configure_collections → configure() (:306). So by the time the documented place to call apply_settings runs, the topology and the process assignment are already fixed. Anything a Collection wants to decide about its own shape has to be decided during construction, before the framework has told it anything.

It costs real startup time. On a synthetic Collection, 48 declared-but-unwired units cost 0.48 s to graph-ready (0.55 s vs 0.07 s pruned). Steady-state cost is nil — nothing routes to them — so this is purely startup, but it scales with how conditional the graph is. In one of our pipelines a single-device run puts 60 of 81 sub-components on the graph dead: 74%.

It pushes Collections into defensive workarounds. Ours currently does this, and the comment says why:

# IntentSource is a static child collection whose default device is
# HUB1. Explicitly idle every slot first so an unused child cannot open
# an unintended CereLink session merely because it exists in the graph.
for hub_number in range(1, 5):
    getattr(self, f"HUB{hub_number}").apply_settings(self._idle_source_settings())

That is a Collection paying, at runtime, to neutralize members it never wanted on the graph. The hazard it is defending against is that an unused child collection has a default device and would otherwise open hardware.

What users end up doing instead

Both workarounds we know of reach into _components, and both have sharp edges:

• Deleting unwired members from self._components during __init__. Wiring something that was deleted fails the graph build with a bare AssertionError: — no message at all. The deleted member's attribute also survives (the setattr in Collection.__init__ already happened), and delattr makes it worse: lookup falls through to the class-level instance, which is shared across every instance of the Collection.
• Registering optional members during __init__ instead of declaring them. This requires calling the private _set_name(); omit it and you get the same bare AssertionError:. It also costs IDE resolution — PyCharm reports Unresolved attribute reference for every legitimate use, which is the same warning it gives for a genuine typo, so enabling one disables the other.

Notably, ezmsg itself uses the second form internally for boundary relays (backend.py:219-221: _set_name, _set_location, components[name] = unit, setattr), so the mechanism is clearly workable — it just is not something a Collection author is given.

Why a holistic look

I have deliberately not proposed a fix. The shape of the right answer probably depends on things I do not have visibility into:

• how the other ezmsg-org packages' Collections handle settings-dependent topology today, and whether they have hit this or quietly worked around it
• whether the fix belongs in the crawl, in Collection, in ezmsg-baseproc's unit lifecycle, or in the ordering of network()/configure() relative to construction
• whether the related problem — that configure() runs after topology is fixed, so a Collection cannot use anything the framework computed when deciding its own shape — is the same problem or a separate one worth solving together
• whether declaration-time introspection (SomeCollection.__components__ describing the full possible topology without constructing anything) is a property worth preserving, since some tooling and documentation depends on it

Happy to supply repro cases, measurements, or test the result against a fairly conditional real-world graph.

Environment: ezmsg 3.9.0.