Support composite natural primary keys (#104) - #105
Merged
Merged
Conversation
A schema declaring more than one primary key field raised MatchError,
because Fields.get_pk_field!/1 matched a single-element list:
[pk_field] = schema.__schema__(:primary_key)
Ten call sites consumed that scalar, covering insert, update, delete,
watch and every read. So the only way to model a compound natural key
was a synthetic key plus a covering index.
That workaround is not free. TPC-C, for one, identifies DISTRICT by
(D_W_ID, D_ID) and ORDER-LINE by (OL_W_ID, OL_D_ID, OL_O_ID,
OL_NUMBER). A synthetic key changes the physical layout, and with it
the write contention a benchmark is trying to measure.
Encoding
Composite key values are spliced into the FDB key tuple as separate
elements:
{adapter_prefix, source, "d", v1, .., vn}
This is the shape the partitioned Versionstamp key already produces,
so the prefix property comes for free. A query constraining a leading
prefix of the key fields resolves to one GetRange with no secondary
index, and the declared field order decides the sort order. Nesting
the values in a sub-tuple would encode them as one opaque element and
lose that.
This matches Apple's fdb-record-layer, whose TupleRange.allOf builds
the same inclusive-both-ends range over a prefix tuple that
Pack.primary_prefix_range/3 does.
A query that constrains key fields which are not a leading prefix
raises Unsupported and names the key order. It previously fell through
to the single-key path, built a key from the one value, matched
nothing, and returned [] with no error.
Values reach the layer as a Fields.CompositePK struct. A struct rather
than a tagged tuple, because a tagged tuple is a value a user could
legitimately store as a primary key, and partition_by: schemas already
use a plain tuple.
Backwards compatibility
A single-field primary key never produces a CompositePK, so it takes
the original code path and its encoded key is unchanged, byte for
byte. The existing doctests pin those exact bytes and still pass.
Data written by earlier releases reads back unchanged. Composite
schemas could not be written before, so there is no stored data to
migrate.
get_pk_field!/1 keeps its single-key behaviour and now raises a
described ArgumentError for a composite schema, instead of a bare
MatchError. Callers that support composite keys use get_pk_fields!/1.
Tests
217 existing tests pass unchanged. Adds 22, covering insert and read
back, key-field distinctness, nil rejection, prefix range scan without
an index, key ordering, update, delete, update_all and delete_all over
a prefix, where-clause field order, trailing-field rejection,
byte-level single-key encoding, negative and large integer ordering,
and prefix isolation between 1, 11 and 111.
Two cover records split across keys. PrimaryKVCodec splits a value
over max_single_value_size across several keys, appending to the key
tuple, which a composite prefix range also covers. A 250_000 byte
record reassembles through a prefix scan and through a full-key
lookup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@fire started the work on this feature in #104, and we'll prepare it for merge to main here. Note: do not squash out fire's commit from the history.
Closes #103