Store raw blocks in validator

[Mirko-von-Leipzig]

We now support node replication via the full node. However there is always the chance that there is no node that is fully in sync with the sequencer when it crashes. This could occur either because there is no full node near the tip, or because the sequencer crashed after committing the block, but before pushing it to the full node.

So we want a guaranteed backup service. The validator already sees all blocks and a block can only be committed after the validator signs it. So if the validator only signs once the block is store locally, we have a solid backup system.

I suggest we re-use the existing BlockStore from the store. It may need cleaning up.

[Ollie202]

Hi, I'd like to work on this if it's available. I traced the current validator path: SignBlock validates the ProposedBlock and persists its header in �alidator.sqlite3, while the existing store BlockStore already provides file-backed serialized block persistence. I'd add a validator-owned BlockStore, construct the signed block from the validated header/body and signature, and ensure the raw block is persisted before the validator acknowledges a successful signing request. I'd add focused coverage for persistence, replacement blocks, and storage failures, keeping any BlockStore cleanup limited to what this integration needs. Could I be assigned?

[Mirko-von-Leipzig]

We should also add a method for streaming these blocks, similar (identical?) to the RPC block subscription method.

This would allow a full node to be promoted to sequencer by doing the following:

1. Shutdown the sequencer (if not down already).
2. Shutdown the designated full node.
3. Restart the full node but now point its block source at the validator.
4. Full node stops once it reaches the chain tip (part of the stream).
5. Note: blocks won't have proofs so these still need to be commissioned as part of recovery.
6. Restart full node as sequencer.

This can be a secondary PR. Some problems:

• The full node has separate clients for these sync, and the "normal" sync has slightly different semantics e.g. the stream shouldn't end, proofs aren't streamed.
• We should therefore make this a separate miden-node recover --validator.url <..> command to be more explicit. Other name suggestions welcome.

[Mirko-von-Leipzig]

The above recovery should also be added to operator manual as a specific chapter/section. Unsure exactly where, but it should be something about network recovery or something.