Draft: layout engine v2 structural crossing refinement

[Delcado19]

Experimental implementation of FlowForge layout engine v2 on a separate branch.

Active implementation:

• SCC-condensed node-level layer assignment
• dummy vertices for long internal edges
• repeated directional median sweeps and local transpose refinement
• port-aware physical-link scoring
• weighted top-level group ordering using physical wire multiplicity
• SCC-aware group flow layers with dummy vertices for long group dependencies
• weighted group medians/transposition with fixed/pinned protection
• compact internal layer placement after v2 ordering
• width-growth guard for node-level candidate selection
• Phase 3 global-flow compactness candidate
• decorative annotations packed above the graph only after structural candidate selection
• decoration-only compaction must preserve crossings, RTL, overlaps and link length
• Phase 4 conservative eligible-ungrouped component compaction with direct group incidence allowed
• eligible ungrouped SCC layers wrapped into a shared horizontal band while groups remain fixed obstacles
• Phase 4 rejects any crossing/RTL/overlap regression, always requires at least 8% width reduction, and forbids area growth
• group-bridged diagnostics confirmed that Jibs and Flux2 VTON are structurally mixed Group/Ungrouped flows
• Phase 5 shared Group/Ungrouped dependency graph with group supernodes and eligible ungrouped vertices
• SCC layering, long-edge dummy vertices, weighted median sweeps, weighted transpose, and monotonic left-to-right mixed-layer placement
• Phase 5 evaluates weighted vs baseline-stable layer order with conservative 100/80/40 px vertical-gap profiles derived from existing layout settings
• Phase 5 preserves group-internal geometry, skips pinned workflows, and uses the same strict no-regression + 8% width + no-area-growth gate
• in-place runtime adapter preserving Workflow/Node/Group identity

81-workflow Optimize + Layout measurements:

• Master: 4320 crossings / 372 RTL
• First v2: 4395 / 318
• Phase 2: 4153 / 318
• Phase 2.5: 4130 / 333
• Phase 3 structural result: 4127 / 339

Phase 3 decorative compaction preserves the 4127 / 339 structural result while materially reducing large workflow dimensions:

• Flux Edit Ultra: 18288x13211 -> 10549x14833
• Z-Image Base Ultra: 18934x12228 -> 10364x14076
• Luneva: 13200x16536 -> 11108x10610
• Qwen Edit: 13024x7967 -> 7985x9527

Remaining width hotspots are now dominated by actual ungrouped flow:

• Amazing Z-Image: ungrouped span 12220 px across 20 layers / 43 columns
• Jibs: ungrouped span 5977 px vs grouped span 3090 px
• several Flux2 VTON workflows have ungrouped spans larger than grouped spans

Current Phase 5 gate:

• first full Phase 5 corpus exposed a port-aware vs node-center metric mismatch (4138 / 331 vs Phase 4 reference 4127 / 339)
• Phase 5 now protects both metric families; the dual-metric 81-workflow rerun passes at 4124 center crossings / 335 center RTL
• targeted rerun confirms Flux2 VTON 6.0.2.7 is now rejected for center-crossing regression (46 -> 60) despite improving port-aware crossings/RTL
• next gate: use tools/report_phase5_diagnostics.py example-workflows --optimize --accepted-only to identify the actual accepted Phase 5 workflows, then visually inspect those layouts before changing placement heuristics
• inspect proposal width/height, crossings, RTL, and exact rejection reason
• if targeted results are promising, run the full 81-workflow Optimize + Layout corpus
• preserve the 4127 crossings / 339 RTL structural reference until new corpus evidence is reviewed
• keep Amazing Z-Image pinned and out of automatic reorganization
• keep this PR draft and do not merge until Phase 5 corpus results are reviewed

Deferred:

• user-facing Undo/Redo for Layout/Optimize by restoring complete pre-action workflow state

• --accepted-only now suppresses routine INFO logging so corpus review output contains only accepted Phase 5 workflows plus the final summary.

• accepted-only corpus review: 81 selected, 37 attempted, 1 accepted, 36 rejected, 44 skipped; the sole accepted workflow is VTON/Flux.1 Kontext-Dev/Flux.1 Kontext dev Virtual Try-On 6.0 (Codex).json using stable-gap-80 (6350x3886 -> 5810x4168, port crossings 43 -> 38, center crossings 53 -> 50, port RTL 10 -> 6, center RTL 5 -> 1).

• visual review of the sole accepted Phase 5 workflow failed despite metric improvements; the layout is visibly fragmented by isolated blocks and large empty bands. Added read-only --geometry diagnostics to measure node density, largest X/Y gaps, empty groups, and moved top-level items before changing the production gate.

• fragmentation root cause isolated: the sole visually rejected Phase 5 result contains three positive-size empty authored groups (Prompt, Sampler, Reference Image 1). These rectangles are outside the mixed graph but were displaced by overlap finalization. Phase 5 now treats unmodeled empty group geometry as a hard scope constraint and falls back to Phase 4.

• targeted validation after the empty-group guard: Flux.1 Kontext dev Virtual Try-On 6.0 (Codex) is no longer attempted by Phase 5 (reason=unmodeled_empty_group_geometry; candidates=0), confirming the visual false positive now falls back to Phase 4.

• Windows redirected diagnostics are now UTF-8 safe: report_layout_quality.py and report_phase5_diagnostics.py reconfigure stdout/stderr to UTF-8 so PowerShell pipelines do not fail on emoji/group names (e.g. 🎨).

• final post-guard validation: accepted-only corpus diagnostic selected 81 workflows, attempted 14, accepted 0, rejected 14, skipped 67. Full Optimize + Layout corpus returned exactly to the Phase 4 reference at 4127 center crossings / 339 center RTL with 0 failures. Phase 5 is therefore currently corpus-safe but a no-op after excluding the visual false positive.

• added --attempted-only plus aggregate attempted-rejection/skipped reason summaries so the next Phase 5 architecture change can target the dominant failure mode among the remaining 14 physical-candidate workflows.

• attempted-only diagnostics clustered the remaining 14 physical candidates into 8 crossing regressions and 6 insufficient-width rejections. Added a fully gated compactx realization that keeps the same mixed layers/Y order but assigns X per real vertex behind true predecessors and vertically overlapping geometry, preventing one wide group from forcing unrelated narrow parallel lanes to inherit the whole layer width. Original layer-X variants remain unchanged.

• first post-compactx accepted corpus candidate: Flux.2 klein 9B Reference Identity 1MP + 2x, variant weighted-compactx-gap-40; size 5810x2774 -> 4850x3308 (~16.5% narrower, ~0.45% less area), port crossings 82->78, center crossings 93->92, port RTL 12->10, center RTL 6->4. Pending visual review before documentation/final corpus acceptance.

• visual review of the first compactx acceptance (Flux.2 klein 9B Reference Identity 1MP + 2x) still failed: FAST MODE, QUALITY MODE, and output regions became distant islands with long cross-canvas wires despite improved width/crossing/RTL metrics. Added diagnostic-only total and maximum port-link Manhattan length to measure this second false-positive mechanism before adding any new gate.

• all-link cohesion diagnostics on the visually rejected compactx candidate changed only modestly (total port-link Manhattan length 57454->58496, +1.8%; max link 3584->3622, +1.1%), so they do not explain the severe islanding well. Added diagnostic-only mixed-graph-level cohesion metrics (real mixed-vertex bounds/density/gaps plus weighted mixed-edge total/max length) to measure dispersion at the abstraction level Phase 5 actually moves.

• mixed-graph cohesion measurement for the visually rejected Flux.2 klein 9B Reference Identity 1MP + 2x compactx candidate: mixed bounds 5150x2774 -> 4190x3308 (~3.0% less bounding area), density 0.449->0.462, max X gap 200->200, max Y gap 100->40, max mixed edge 4135->3421, but weighted total mixed-edge Manhattan length 53466->58988 (+10.3%). This fits the screenshot better: the failure is cumulative top-level wiring dispersion rather than one extreme edge or empty-space gap. No new threshold is added yet; compare this metric across all attempted workflows first.

• corpus-wide mixed-cohesion review shows weighted mixed-edge growth is not unique enough to justify an immediate threshold. Added --variants diagnostics to compare all 12 physical Phase 5 realizations for a selected workflow, including existing gate result and mixed-edge total/max length, so we can determine whether the visual false positive is a candidate-ranking problem before adding another gate.

• 12-way Flux 9B variant review ruled out a simple ranking bug: all shared-layer X variants kept weighted mixed-edge length near baseline but failed crossing/RTL, while all compact-X variants gained width/RTL at ~+10..13% mixed-edge growth; only weighted-compactx-gap-40 passed. Added a third fully gated anchoredx realization that preserves Compact-X's right boundary and constraints but uses horizontal slack to move vertices back toward their Phase 4 X positions, targeting over-compacted weakly constrained branches without giving back the compact width envelope.

• weighted-anchoredx-gap-40 also passes the full Phase 5 gate at the same 4850x3308 size as the compact candidate, while reducing weighted mixed-edge growth from +10.3% to +1.7% (81 port crossings, 92 center crossings, 12 port RTL, 6 center RTL). Production ranking is intentionally unchanged pending visual review. Added diagnostic-only --export-variant <name> --output <path> so an exact candidate can be written to ComfyUI JSON without changing normal selection.

• visual review of weighted-anchoredx-gap-40 showed a substantial horizontal improvement over Compact-X, but still failed overall because QUALITY MODE and FAST MODE remain vertically detached with long vertical/diagonal wiring across large empty bands. This isolates the remaining issue to Phase 5's shared-base stacked Y realization. Added diagnostic-only weighted-anchoredxy / stable-anchoredxy variants that preserve each real mixed vertex's Phase 4 Y coordinate while using Anchored-X. They are excluded from normal Phase 5 selection pending metrics + visual review.

• exact Phase 4 Y anchoring (weighted-anchoredxy) failed on the Flux 9B review case: 6730x2774, port crossings 82->105, center crossings 93->113, weighted mixed-edge +19%. Added diagnostic-only Y-band anchoring (*-anchoredx-yband-gap-*): preserve each dependency layer's internal order/spacing, but shift the whole stacked layer toward its Phase 4 vertical band using the median Y offset. Production selection remains unchanged pending metrics and visual review.

• Y-band diagnostics on Flux 9B produced a strong near-miss: weighted-anchoredx-yband-gap-40 keeps width at 4850 px (-16.5%), improves port crossings 82->78 and center crossings 93->88, keeps RTL at 12/6, and holds weighted mixed-edge growth to +1.5%. It is rejected only for area growth: 5810x2774 -> 4850x3731 (~+12.3% area). This candidate should be visually reviewed before changing either the Y realization or the area gate.

• visual review confirmed full Y-band anchoring is the best-looking Phase 5 result so far, but it still fails area due to 4850x3731 (~+12.3%). Added diagnostic-only compressed Y-band variants at 25/50/75% layer-anchor strength (compact 40 px gap). This creates a measured continuum between stacked Anchored-X (0%) and full Y-band anchoring (100%) so we can search for a height/cohesion sweet spot without changing production selection or loosening gates.

• compressed Y-band sweep showed weighted-anchoredx-yband25-gap-40 is closest to the strict gate: 4850x3414 (~+2.7% area), 81 port crossings, 91 center crossings, mixed-edge +2.4%; 50/75% anchors cost more area. Added diagnostic variants at the existing hard minimum 12 px vertical gap for the same 25/50/75% anchor strengths to test whether tighter within-layer spacing can recover the remaining area excess without loosening the gate.

• minimum-gap sweep brought weighted-anchoredx-yband25-gap-12 to 4850x3351, only ~+0.84% area above the Phase 4 baseline, with port crossings 82->81, center crossings 93->91, and mixed-edge +1.1%. Added finer diagnostic 10/15/20% anchor strengths at the same 12 px floor to search the narrow interval where visual locality may survive while the strict no-area-growth gate passes.

• visual review of weighted-anchoredx-yband10-gap-12 passes qualitatively: the workflow keeps the clearer Y-band grouping while remaining compact at 4850x3292, with FAST/QUALITY branches reading as intentional side flows rather than detached islands. This is the first Phase 5 candidate to pass both the existing numeric gate and visual review. yband15-gap-12 remains worth a final side-by-side check because it costs only +19 px height while improving center crossings from 92 to 90.

• promoted the visually validated minimum-gap Y-band candidates into normal Phase 5 generation: weighted/stable yband10-gap-12 and yband15-gap-12. Hard acceptance gates remain unchanged. Accepted-candidate ranking is now center-crossings first, then weighted mixed-graph edge length for top-level cohesion, then port crossings/RTL and size. This is intended to select the validated weighted-anchoredx-yband15-gap-12 over the visually fragmented Compact-X result when both are gate-safe.

• production-path validation now selects weighted-anchoredx-yband15-gap-12 on the Flux 9B reference workflow (22 standard candidates): 5810x2774 -> 4850x3311, port crossings 82->81, center crossings 93->90, RTL unchanged at 12/6, total port-link length 57454->56458, mixed-edge 53466->53913 (~+0.8%), max mixed edge 4135->3424, max Y gap 200->112, no empty groups. This matches the visually preferred diagnostic candidate and confirms the promoted candidate/ranking path is active.

• final production corpus gate: 81/81 workflows, 0 failures, center crossings 5723->4124 and RTL 568->339. Relative to the Phase 4 reference (4127/339), production Phase 5 improves crossings by 3 while holding RTL exactly at the floor. Phase 5 attempts 14 workflows, accepts exactly the visually validated Flux 9B weighted-anchoredx-yband15-gap-12, rejects 13, and skips 67 by existing scope constraints.

• fixed logger file spam: ordinary setup_logger(__name__) calls are now console-only; file logging is opt-in via log_dir, uses delayed file creation, and no longer emits Fehlerhaftes ... filenames. Added regression tests for no default files, delayed explicit file creation, and handler deduplication.

• documentation cleanup: Current Scope Boundary now leads with the actual production-safe Phase 5 status, while the earlier false-positive and safe-no-op measurements are explicitly labeled as historical development checkpoints. README/CHANGELOG remain intentionally untouched and the PR remains Draft.

• added bounded full-workflow transformation Undo/Redo in the frontend. Successful Layout Only, live spacing-layout, and atomic Optimize + Layout runs record complete ComfyUI JSON snapshots (10-entry limit). New-file load clears history, new transformations clear Redo, Undo/Redo invalidate pending layout work, toolbar buttons and Ctrl/Cmd+Z / Ctrl/Cmd+Shift+Z / Ctrl/Cmd+Y are supported, and manual canvas edits remain outside this first history scope. Frontend README and layout-v2 docs updated; CI green.