Skip to content

Commit 96108c4

Browse files
authored
feat: meta-service: add proposed_at tracking the time when a key is written (#18812)
* feat: meta-service: add `proposed_at` tracking the time when a key is written Add `proposed_at_ms` field to `KvMeta` protobuf to track proposal timestamps for key-value operations in the meta service. Introduce `ReduceSeqV` trait with two methods: `erase_proposed_at()` removes timestamps from metadata for test comparisons, and `reduce()` removes default-valued metadata to minimize storage overhead. Implement the trait for common types including `SeqV<T>`, `Option<SeqV<T>>`, collection types, and domain-specific types (`Change`, `pb::Event`, `pb::StreamItem`). Update all meta service tests to use `erase_proposed_at()` when comparing results, allowing tests to ignore variable proposal timestamps while verifying data correctness. Add comprehensive unit tests covering metadata erasure and reduction behavior. * refactor: update tests for KVMeta proposed_at_ms field Fixed compilation errors and test assertions after adding proposed_at_ms to KVMeta structure. Updated tests to handle the new field: - Updated KVMeta::new() calls to include proposed_at_ms parameter - Applied .erase_proposed_at() for SeqV types in assertions - Updated expected values to match database default (Some(0)) - Replaced legacy KVMeta::new_expires_at() calls - Updated JSON serialization test expectations All 94 raft-store tests now passing. * refactor: rename ReduceSeqV to NormalizeMeta with improved docs * M src/meta/service/tests/it/grpc/metasrv_grpc_kv_read_v1.rs * M tests/metactl/want_exported_v004 * M src/meta/client/src/lib.rs * M tests/metactl/subcommands/cmd_export_from_grpc.py
1 parent 271b7d6 commit 96108c4

34 files changed

+1158
-242
lines changed

Cargo.lock

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -498,7 +498,7 @@ socket2 = "0.5.3"
498498
span-map = { version = "0.2.0" }
499499
sqlx = { version = "0.8", features = ["mysql", "runtime-tokio"] }
500500
state = "0.6.0"
501-
state-machine-api = { version = "0.3.1" }
501+
state-machine-api = { version = "0.3.4" }
502502
stream-more = "0.1.3"
503503
strength_reduce = "0.2.4"
504504
stringslice = "0.2.0"
@@ -663,7 +663,7 @@ openraft = { git = "https://github.com/databendlabs/openraft", tag = "v0.10.0-al
663663
orc-rust = { git = "https://github.com/datafuse-extras/orc-rust", rev = "d82aa6d" }
664664
recursive = { git = "https://github.com/datafuse-extras/recursive.git", rev = "16e433a" }
665665
sled = { git = "https://github.com/datafuse-extras/sled", tag = "v0.34.7-datafuse.1" }
666-
state-machine-api = { git = "https://github.com/databendlabs/state-machine-api.git", tag = "v0.3.1" }
666+
state-machine-api = { git = "https://github.com/databendlabs/state-machine-api.git", tag = "v0.3.4" }
667667
sub-cache = { git = "https://github.com/databendlabs/sub-cache", tag = "v0.2.1" }
668668
tantivy = { git = "https://github.com/datafuse-extras/tantivy", rev = "7502370" }
669669
tantivy-common = { git = "https://github.com/datafuse-extras/tantivy", rev = "7502370", package = "tantivy-common" }

src/meta/client/src/lib.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,9 @@ use semver::Version;
155155
/// - 2025-09-27: since 1.2.821
156156
/// 👥 client: require 1.2.764(yanked), use 1.2.768, for `FetchAddU64`
157157
///
158+
/// - 2025-09-30: since TODO
159+
/// 🖥 server: store raft-log proposing time `proposed_at_ms` in `KVMeta`.
160+
///
158161
/// Server feature set:
159162
/// ```yaml
160163
/// server_features:

0 commit comments

Comments
 (0)