GPU memory has no ownership, accounting, or release story

[bdero]

flutter_scene has no story for releasing GPU memory. There is no way to ask for a resource to be freed, no way to see what is resident, and the engine's asset caches hold what they load for the life of the process. On a device with a fixed memory budget, an app that streams levels or swaps materials grows until it is killed, and nothing in the API surface lets the author do anything about it.

The obvious request is a dispose() on textures and buffers, and that is worth having, but it would not fix this on its own. Most of what an app allocates is pinned by flutter_scene rather than by the caller, so a destructor would free nothing while the engine still holds the last reference. Ownership has to come first.

Why dropping references does not work

Flutter GPU's Texture is a RefCountedDartWrappable, and neither lib/gpu/ nor lib/ui/dart_wrapper.h reports an external allocation size to the Dart GC. The collector therefore sees a small wrapper object, and a multi-megabyte texture exerts no heap pressure whatsoever.

So reclamation is not merely non-deterministic, it is uncorrelated with GPU pressure. An app can sit at hundreds of megabytes resident with an idle Dart heap and never collect. That is the mechanism behind "null the reference and hope", and it is why the fix has to be explicit rather than a matter of being tidier about references.

What retains memory today

• Loaded textures never drop. _textureCache in lib/src/texture/texture_registry.dart is keyed by asset path for the life of the process. Loading a second level does not release the first level's textures.
• Loaded scenes never drop. _sceneTemplates in lib/src/importer/scene_registry.dart has the same shape.
• fmat shader libraries and sidecars accumulate per material in lib/src/fmat/material_registry.dart.
• Static resources are permanent by construction. Placeholder textures, the BRDF LUT, the memoized studio environment, and the pass shaders and quad buffers held in statics across lib/src/render/. Mostly small and mostly correct to keep, but they are unaccounted.
• ResourceGroup.dispose() does not release anything. It disposes the progress notifier only, which is a trap for anyone who reasonably reads the name as "free what this group loaded".

The _pipelineCache in lib/src/scene_encoder.dart is deliberately not on this list, see the decision on it below.

The one place that already does this properly is the declarative model-template cache in lib/src/widgets/declarative.dart, which hands out refcounted leases and drops the entry when the count reaches zero. That is the pattern to generalize rather than reinvent.

Goals

1. An author can find out what is resident and roughly what it costs, in bytes and by category, without a native profiler.
2. An author can scope a set of loads to a lifetime and release them together when it ends, which is how levels and screens are actually structured.
3. No cache grows without limit for the life of the process.
4. Releasing is safe. Nothing an author can call from Dart should be able to free a resource an in-flight frame still samples.

Proposed work, in order

1. Accounting. flutter_scene creates the textures, buffers, and render targets, so it can count them. A snapshot API reporting resident bytes and counts by category (textures, geometry, render targets, transients) plus what is holding each cache entry. Entirely ours, needs nothing upstream, and comes first because there is currently no way to tell whether any later change helped.

2. Refcounted leases in the asset registries. Generalize the declarative.dart lease pattern to the texture and scene registries so an entry drops when its last holder goes away.

3. Lifetime-scoped ownership through ResourceGroup. It already exists, apps already group loads through it, and its lifetime already matches a screen or a level. Make it own what it loaded and add a release that drops those claims. Fix dispose() so the name means what it says, or rename it. Symmetric API, no upstream dependency.

4. Upstream asks. Two, in increasing size. First, report external allocation size for Flutter GPU resources so GC scheduling correlates with real GPU pressure; this is small, self-contained, and helps every flutter_gpu consumer. Second, a dispose() on textures and buffers with defined semantics, drop the handle's claim, mark it dead, reclaim after the last in-flight frame that referenced it retires. Related, flutter/flutter#178264 covers Impeller-level memory stats and a pressure hook.

Decisions

Refcounted leases, not group-keyed wholesale drops. The registries key on asset path, so sharing across lifetimes is the normal case rather than the exotic one, two levels referencing the same texture. Group keying forces a choice between double-loading the shared asset and freeing it out from under the other holder. The bookkeeping is one increment per load, which is nothing beside a texture upload, and leases give the accounting API its "who still holds this" answer for free.

Explicit release, plus pressure-triggered eviction, never a steady-state budget. Evicting costs a reload, disk plus transcode plus upload, so a budget-triggered evictor thrashes precisely when the app is busiest and converts a memory problem into a frame-time problem. Responding to an OS memory warning is a different trade, since the alternative there is being killed. So explicit release is the API, with an opt-in hook that drops only unreferenced entries under platform memory pressure.

Do not bound the pipeline cache by count. Instrumenting resolvePipeline across the full smoke suite produced 15 pipelines for 14 scenes spanning PBR, unlit, two .fmat materials, splats, particles, shadows, skinned meshes, and a raw shader pair. It grows with distinct shader-and-layout triples, meaning distinct materials, not with draws or frames or time. More decisively, Impeller keeps its own PipelineMap of the real pipeline objects with no bound and no eviction beyond RemovePipelinesWithEntryPoint, so evicting our Dart map would free a map entry and leave the GPU-side object alive. The right model is ownership-keyed eviction, dropping a shader library's pipelines when it is released, which evictPipelinesForShaders already does for hot reload. If pipeline memory ever becomes a real cost, the ask is upstream.

Ship the release API before reclamation is deterministic, and document the gap. Dropping the last reference at a known time is most of the benefit, and the external-size finding above means the current behavior is worse than non-deterministic. Waiting for upstream would leave authors with no lever at all in the meantime.

Non-goals

An immediate synchronous free. A queued frame may still sample a texture, so destroying on call is a use-after-free with a GPU-side crash and no useful stack. Release should mean the engine drops its claim and the resource dies when nothing references it, with a debug check that nothing in the live scene graph still points at it.

Manual pooling APIs. Once the steps above are done the remaining timing gap is upstream, and a pooling surface would be a workaround that hardens into API.

Still open

• What granularity the accounting snapshot reports, and whether it is debug-only. Per-resource attribution is the useful version and also the expensive one.
• Whether release belongs only on ResourceGroup or also per-resource. Per-resource is more flexible and much easier to misuse.
• Whether the static resources (placeholders, BRDF LUT, studio environment) should stay permanent or become releasable once nothing references them. They are small, so this is about honest accounting more than savings.