Skip to content

Distributed block-table serialization (kv_cache_serde) #125

@inureyes

Description

@inureyes

Context

Disaggregated / decode-node handoff must transfer paged block tables and pool blocks, not dense snapshots, once the unified store is the source of truth.

Tasks

  • Extend src/distributed/kv_cache_serde/ to serialize PagedSequenceState block tables plus the referenced pool blocks (and Turbo sidecars) and restore them via PagedBlockPool::restore_sequence.
  • Validate a prefill-node to decode-node handoff reconstructs identical KV and identical first-token logits.
  • Confirm refcount and block ownership are correct after restore (no double-free, no leak).

Acceptance criteria

  • A disaggregated run reproduces single-node output within tolerance using the paged block-transfer path.
  • Block accounting after restore matches the originating node.

Dependencies

Blocked by Phase 4 (unification).

Part of #116

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:coremlxcel-core: MLX FFI, primitives, KV cache, layersstatus:backlogIn the backlog, not yet readytype:enhancementNew features, capabilities, or significant additions

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions