Add callback-based block creation API#166
Open
larsewi wants to merge 10 commits into
Open
Conversation
larsewi
force-pushed
the
callback-block-api
branch
2 times, most recently
from
a940646 to
2a52eae
Compare
Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Tables that omit the `source` key are reserved for the upcoming callback-backed path; the CSV loader is renamed to load_from_csv and bails if its source is unset. Existing CSV tables with `source = "..."` behave exactly as before. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Move the panic guard, null/cstring argument checkers, repr-C buffer and cell types, and the cell-decode helper out of lib.rs into the private src/ffi.rs module. Subsequent work (a callback-based block creation path) needs the same plumbing from a sibling module, and this keeps lib.rs focused on the extern "C" entry points. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Introduce a Rust-side adapter for the upcoming callback-based block creation path: a Callbacks struct wraps the three C hook pointers (table_begin, read_cell, table_end) plus an opaque usr_data, and translates the cell-callback's three-valued return code into a CellResult enum (Cell / EndOfTable / FilterRecord). Table::load_from_callbacks pulls rows in ascending order, requesting cells in canonical column order (primary keys lex-sorted, then subsidiaries lex-sorted) and validating each cell against its declared kind. A new validate_cell helper rejects Cell::Null on primary-key fields and a cell kind that does not match the field configuration. The new module is private to the crate; nothing here is wired into lch_block_create yet -- that comes next. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
State::compute now takes an optional callback bundle. Tables with a configured source are loaded from CSV as before; tables without a source are pulled through the callback bundle, which fires table_begin once, then read_cell for every (row, column), then table_end with a status reflecting whether the table drained cleanly. table_end always fires when table_begin succeeded so the caller's per-table resources can be released on the error path too. Block::create keeps its single-argument shape for the existing ~30 in-tree call sites; Block::create_with_callbacks is the new entry point that the FFI layer drives. A one-time warning fires when filters are configured alongside any callback-backed table, because filters are CSV-only -- the callback owns row inclusion via LCH_FILTER_RECORD. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
lch_block_create() now takes a second argument: a pointer to an lch_callbacks_t, or NULL when every table in the config is CSV-backed. The bundle carries an optional table_begin, a required (when any table is callback-backed) read_cell, an optional table_end, and an opaque usr_data forwarded verbatim to every callback. Two new return-code sentinels are defined: LCH_END_OF_TABLE = 1 (no row at this index, stop iterating) and LCH_FILTER_RECORD = 2 (drop the current row). This is a breaking change to the C ABI: every caller must update lch_block_create(cfg) call sites to lch_block_create(cfg, NULL). The on-disk block format is unchanged, and CSV-backed configs behave exactly as before. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Extend the test config with a second table (`events`) that has no `source` key, and define a callback bundle that synthesises two rows for it. The test now drives lch_block_create(cfg, &callbacks) and asserts: * table_begin and table_end fired exactly once each for `events`, * table_end received status = LCH_SUCCESS, * the callback hooks were never invoked for the CSV-backed table. This exercises the full callback-backed path -- begin / read_cell / end lifecycle, value marshalling through lch_cell_t, the per-table dispatch in State::compute -- in the same binary that already covered the existing CSV-only API. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Cover the new lch_block_create() signature, lch_callbacks_t bundle, callback typedefs, and LCH_END_OF_TABLE / LCH_FILTER_RECORD return codes in the libleech2 man page, including the per-table lifecycle, thread model, and how filters interact with callback-backed tables. Add a README section showing a minimal C callback example and update the existing C snippet to pass NULL for the callback bundle when no table needs it. CONTRIBUTING gains a brief paragraph in Block::create() on the two source paths and a note for the new src/ffi.rs and src/callbacks.rs modules in the source layout. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
The implementation has landed; the plan file has served its purpose. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
Dedents one continuation line inside a multi-line inline code span. Cosmetic only -- markdown folds the whitespace at render time, so the change is invisible in the rendered output. Signed-off-by: Lars Erik Wik <lars.erik.wik@northern.tech>
larsewi
force-pushed
the
callback-block-api
branch
from
2a52eae to
3f782ea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lch_block_create()now accepts an optionallch_callbacks_t *callbacksargument so C applications can supply table rows through a cell-pull callback instead of staging them to CSV files first.sourcekey inconfig.toml. CSV-backed and callback-backed tables coexist in the same config.table_begin,read_cell,table_end) plus an opaqueusr_data; onlyread_cellis required when any table is callback-backed.table_endalways fires whentable_beginsucceeded, including on the error path, so per-table resources can always be released.lch_cell_t(the same tagged union already used bylch_patch_inject); the kind tag must match the field's declared kind.LCH_END_OF_TABLE(no row at this index, stop iterating) andLCH_FILTER_RECORD(drop the current row).[filters]apply to CSV-backed tables only — the callback owns row inclusion for its tables, viaLCH_FILTER_RECORD. A one-time warning is logged when filters are configured alongside any callback-backed table.Breaking change
Every C caller must update
lch_block_create(cfg)call sites tolch_block_create(cfg, NULL). The on-disk block format is unchanged, and CSV-only configs behave exactly as before.🤖 Generated with Claude Code