Spike: extensions.compatObjectFormat measured on a Rust-enabled git (backlog 29.5)

[scolladon]

Backlog 29.5 was scheduled as "a spike first, not an implementation", on the grounds that this project had never observed a Rust-enabled git — every dev machine reports rust: disabled, and the only evidence was negative (CI expectations flipping when GitHub's runner image changed). This spike obtains such a build and measures it.

Headline: the extension's stated purpose — cross-algorithm push/pull — does not work at all, in either direction. Two further defects (a BUG: abort and a silent wrong-oid answer) sit on trivially reachable configs. Recommendation is to document the divergence, not implement.

Every claim below is tagged [measured] or [inferred].

Question

• Q0 Can a Rust-enabled git be obtained and run reproducibly?
• Q1 What does it observably do where the non-Rust build refuses everything with exit 128?
• Q2 Is that behaviour stable enough for the prime directive to bind to?
• Q3 What would implementing it cost in tsgit?
• Q4 What does "be git-faithful" even mean when git's behaviour is a property of the build?

Method

Two builds of the same version string, probed with identical scripts:

	build	source
non-Rust	git 2.55.0, rust: disabled	darwin arm64, local
Rust-enabled	git 2.55.0, rust: enabled	ubuntu:24.04 + ppa:git-core/ppa, arm64, in Docker

Probes reproduce the matrix in docs/design/repository-format-acceptance-gate.md §1i (log and config --list against a hand-written .git/config), then extend into on-disk inspection, cross-algorithm transport, and lifecycle. GIT_* scrubbed throughout. Oracle for translation correctness: the canonical SHA-1 of blob hello\n, ce013625030ba8dba906f756967f9e9ca394464a.

Results

R1 · The build flag is the whole story [measured]

Both builds self-report as git version 2.55.0. Only git version --build-options separates them (rust: enabled vs rust: disabled). The version string cannot be used to predict behaviour — confirming the design of the existing GIT_HAS_RUST_COMPAT probe in test/integration/interop-helpers.ts.

R2 · The §1i matrix, both builds [measured]

config	non-Rust 2.55.0	Rust-enabled 2.55.0	tsgit today
version absent + compat=sha1	0 — works	0 — works	accept
= -1 + compat=sha1	0 — works	0 — works	accept
= 0 + compat=sha1	128 v1-only	128 v1-only, identical	v1-only
= 0 + compat + refStorage	128 v1-only (both names)	128 identical	v1-only
= 1, sha1 repo, compat=sha1	128 requires Rust	134 — BUG: repository.c:201: hash_algo and compat_hash_algo match, SIGABRT	refuse
= 1, sha1 repo, compat=sha256	128 requires Rust	0 — works	refuse
= 99 + compat=sha1	128 version ceiling	128 identical	version ceiling
= 1 + subsectioned x.compat	128 unknown extension	128 identical	unknown
= 1 + valueless compat	128 bad config line	128 identical	bad config line

Every row outside the version = 1 band is byte-identical across builds. The Rust flag changes exactly the two rows the non-Rust build answers with requires Rust — and it answers one of them by aborting.

The abort is a real SIGABRT (exit 134 = 128+6, Aborted), not a styled fatal. It fires on compat = sha1 in a SHA-1 repo — i.e. when compat equals the native format. git-config(5) states the value "must be different from the value of extensions.objectFormat"; that precondition is unvalidated and reaches an internal assertion instead of an error message.

R3 · In its intended shape, the feature works [measured]

SHA-256 repo + extensions.compatObjectFormat = sha1, set before any objects exist: init, add, commit, log, fsck (exit 0, clean), gc, and bundle create all succeed.

.git/objects/loose-object-idx appears — plain text, one space-separated pair per line:

# loose-object-idx
2cf8d83d9ee29543b34a87727421fdecb7e3f3a183d337639025de576db9ebb4 ce013625030ba8dba906f756967f9e9ca394464a
b62f2fae32f11031f8fdf4563eb7be5f363e4a2784997b856cdc0b39cbdaf4e3 b4ed918248039b78f24383523fa4e51f80994fac

Verified properties:

• Translation is correct. The blob line matches the canonical SHA-1 oracle exactly.
• Compat oids are addressable. git cat-file -p ce013625… inside the SHA-256 repo returns hello — the "use oids encoded in compatObjectFormat to locally specify objects" half of the documented promise is real.
• rev-parse exposes both. --show-object-format → sha256, --show-object-format=compat → sha1, --output-object-format=sha1 HEAD → a genuine 40-hex oid.
• The table is incremental and survives packing. 4 lines after one commit, 7 after two. After gc (zero loose objects remaining) entries are retained, cat-file by compat oid still resolves, and translation is still correct. The file is not pruned to match its name.
• No version marker. The header is the bare string # loose-object-idx — no format version, no stability handle.

R4 · The stated purpose does not work [measured]

git-config(5) promises "when fully implemented the pushes and pulls from a repository in whose objectFormat matches compatObjectFormat". Both directions fail on the Rust-enabled build:

direction	result
push: SHA-256 + compat=sha1 → SHA-1 bare peer	fatal: the receiving end does not support this repository's hash algorithm, exit 128
fetch: SHA-1 peer → SHA-256 + compat=sha1	fatal: mismatched algorithms: client sha256; server sha1, exit 128

The feature's headline capability is absent. What exists today is purely local: a translation table and compat-oid addressing.

There is also no porcelain to enable it: git init rejects --compat-object-format, and git clone rejects --object-format outright. The only way in is hand-editing .git/config.

R5 · The natural enablement path silently returns wrong oids [measured]

Because there is no init-time flag, the obvious user path is init, then git config. That path is broken. Identical repos, identical commands — the only difference is when the key was set:

	loose-object-idx	rev-parse --output-object-format=sha1 HEAD
compat set before objects exist	created	933b530f1150656bb75b5121777ee850e3a6a24b — real 40-hex SHA-1
compat set after objects exist	never created	b0b5bebd…2bd7590b — the SHA-256 oid, 64 hex, exit 0, no warning

Asked for a SHA-1, git returns a SHA-256 and reports success. No error, no warning, no diagnostic. A caller cannot distinguish it from a correct answer except by measuring the length. Existing objects are never backfilled into the table.

Findings

1. Q0 — solved. A Rust-enabled git is obtainable in ~2 minutes via Docker; recipe below. Local goldens are no longer blocked. [measured]
2. The divergence 29.5 was opened to investigate is real but narrow. It exists only in the version = 1 band, only on a Rust-enabled build, and only for configs a user must hand-edit into place. [measured]
3. On two of the four version = 1 outcomes, tsgit's refusal is safer than Rust-git's behaviour — one aborts the process, one silently returns a wrong-width oid. Refusing is not merely defensible there; it is the better answer. [measured]
4. The prime directive cannot bind to this. Not because the docs warn it may change, but because it demonstrably does not have settled behaviour: a BUG: assertion and a silent wrong answer on ordinary paths. Pinning goldens would pin defects. [measured]
5. Implementing would buy no interop. The reason to want compat is cross-algorithm push/pull; that is exactly the part that does not work. [measured]
6. The finished subset is coherent and small — build the translation table on write, resolve compat oids on read. Local-only, no transport. If 29.5 is ever implemented, this is the slice, and it is the part git has actually completed. [inferred]
7. Q4 — "be git-faithful" is under-determined here. Faithfulness assumes one git; this behaviour is a property of the build, and tsgit has no build flag. Today's refusal is faithful to non-Rust builds and divergent against Rust-enabled ones. This flips from mostly-right to mostly-wrong the day Rust-enabled becomes the default — with no change on our side. [inferred]

Opinion / proposed lift

Document the divergence. Do not implement. UNBACKED_EXTENSIONS = ['compatobjectformat'] in src/repository/read-repository-format.ts stays as-is, and 29.5 closes as a recorded, justified divergence rather than as work.

Reasoning: implementing buys zero interop (F4/F5), the target behaviour includes an abort and a silent wrong answer (F3/F4), and the cost is a dual-algorithm storage subsystem — for scale, 29.3 swept 84 files to support one algorithm. Real-world exposure is capped by there being no porcelain to enable it (R4).

The honest counter, recorded rather than buried: on the working path tsgit refuses where Rust-git succeeds (R3), so this is a genuine divergence, not merely a refusal git also makes. It is being accepted on stability grounds, not correctness grounds.

Decisions left for a future ADR

1. Does the divergence get recorded as an accepted divergence (prime-directive exception with written rationale) or as deferred work? The distinction matters: the former is a stable end state, the latter keeps 29.5 open indefinitely.
2. When git's behaviour is build-dependent, which build is tsgit faithful to — and is that a general policy or a one-off? This is the first time the question has arisen; it will not be the last.
3. If implemented later, is the local-only subset (translation table + compat-oid resolution, no transport) an acceptable partial, given tsgit's usual all-or-nothing stance on write parity?
4. Should the refusal stay generic (ADR-685) or gain a compat-specific reason, given we can now describe precisely what is being refused and why?

Tripwires — signals that should reopen this

Each is mechanically checkable. Ordered by how strongly it should move the decision.

1. Cross-algorithm push/pull starts working. Re-run R4. This is the signal — it is the entire purpose of the extension, and its absence is the core of the recommendation. Everything else is secondary.
2. rust: enabled becomes the default build (upstream Makefile default, or Debian/Ubuntu/Homebrew shipping it on). Makes today's refusal wrong for the majority of users. Note it is already the default on GitHub-hosted runners.
3. The disclaimer moves — git-config(5) dropping "not designed to be enabled by end users", or the description losing its future tense ("when fully implemented").
4. The BUG: abort and the silent-wrong-oid path get fixed. Both are re-testable in one script run; together they are the stability proxy.
5. loose-object-idx gains a version marker, or hash-function-transition.adoc moves from design to specification — either would signal a format worth pinning.
6. A hosting provider ships it (GitHub/GitLab accepting SHA-256 repos with compat interop) — turns this from a lab feature into something a tsgit user can encounter.
7. A user reports it. Someone hits tsgit's refusal on a repository their own git opens fine.

Signals 1, 3, 4 and 5 are checkable by a scheduled job running the probe script against a freshly pulled git-core PPA image and diffing against the results recorded here. This spike does not land that job — specifying it is the deliverable; building it is separate work.

Repro

cat > Dockerfile <<'EOF'
FROM ubuntu:24.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update -qq >/dev/null && \
    apt-get install -y -qq curl gnupg ca-certificates >/dev/null && \
    install -d /etc/apt/keyrings && \
    curl -fsSL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0xA1715D88E1DF1F24" \
      | gpg --dearmor -o /etc/apt/keyrings/git-core.gpg && \
    echo "deb [signed-by=/etc/apt/keyrings/git-core.gpg] https://ppa.launchpadcontent.net/git-core/ppa/ubuntu noble main" \
      > /etc/apt/sources.list.d/git-core.list && \
    apt-get update -qq >/dev/null && \
    apt-get install -y -qq git >/dev/null
EOF
docker build -t rustgit .
docker run --rm rustgit git version --build-options | grep rust:   # => rust: enabled

The three decisive probes:

# 1. the abort
docker run --rm rustgit bash -c 'cd /tmp && git init -q r && cd r &&
  git -c user.email=a@b -c user.name=a commit -qm x --allow-empty &&
  printf "[core]\n\trepositoryformatversion = 1\n[extensions]\n\tcompatObjectFormat = sha1\n" > .git/config
  git log; echo "exit=$?"'            # => BUG: repository.c:201 ... exit=134

# 2. push/pull do not work
docker run --rm rustgit bash -c 'cd /tmp && git init -q --bare peer &&
  git init -q --object-format=sha256 src && cd src &&
  git config extensions.compatObjectFormat sha1 &&
  git -c user.email=a@b -c user.name=a commit -qm x --allow-empty &&
  git push /tmp/peer HEAD; echo "exit=$?"'   # => fatal: the receiving end ... exit=128

# 3. the silent wrong answer (compat enabled AFTER objects exist)
docker run --rm rustgit bash -c 'cd /tmp && git init -q --object-format=sha256 r && cd r &&
  git -c user.email=a@b -c user.name=a commit -qm x --allow-empty &&
  git config extensions.compatObjectFormat sha1 &&
  echo "native: $(git rev-parse HEAD)" &&
  echo "compat: $(git rev-parse --output-object-format=sha1 HEAD)"'
  # => the two are identical; a 64-hex SHA-256 returned as a "sha1", exit 0