SPEC-GIT-IMPORT.md
This Kit feature is not available yet.
In docs/specs/ · At 26cf72cCopy the commit ID
In docs/specs/ · At 26cf72cCopy the commit ID
Status: Normative for the git→mkit import direction. Scope: the mapping from git objects to mkit v1 objects under the importer-signed policy, the refusal matrix, provenance, per-remote state and key binding, canonical remote identity, and pull/divergence semantics. Shared byte-level encodings (hex/base64 forms, ref grammars, chunking parameters) are defined by SPEC-GIT-BRIDGE, SPEC-OBJECTS, SPEC-SIGNING, SPEC-REFS, and SPEC-FASTCDC; this spec only adds the inbound rules.
Import is not the inverse of export. Export round-trips mkit objects losslessly through carrier headers; import is a signed translation across a trust boundary: every imported commit and tag is a new mkit object signed by the importer, attributing the original author and binding the original git bytes as provenance. The result is a downstream fork — its mkit hashes are a function of the importer's signing key, and two importers of the same upstream produce unrelated mkit histories by design (§6).
git blob → mkit blob (≤ 1 MiB) or chunk blobs +
manifest (> 1 MiB, pinned FastCDC)
git tree → mkit tree (re-sorted; names validated)
git commit → mkit commit, SIGNED BY THE IMPORTER
git annotated/signed tag → mkit tag, SIGNED BY THE IMPORTER
git lightweight tag → bare refs/tags/<name> ref
git gitlink (160000) → REFUSED (per-ref)The importer's Ed25519 signature asserts exactly: "I vouch that
these bytes are my translator's faithful output for the named
upstream object." It is never an authorship claim — original
authorship rides in the author/tagger Identity (§3.2), and
SPEC-SIGNING §6 already separates attribution from the verification
key. A git GPG/SSH signature on the source commit/tag is carried in
the retained raw bytes (§5) and is not representable as an mkit
signature.
The mapping is defined for mkit object schema 0x01 and is pinned by
an import-spec version recorded in the per-remote state (§6).
Implementations MUST refuse to incrementally extend an import whose
recorded import-spec version they do not implement (forcing an
explicit re-import) — translation-rule drift between versions would
silently fork hashes for the same key.
Import is a pure function of (upstream git bytes, importer signing key, import-spec version). mkit signing is deterministic (RFC 8032), so re-running an import reproduces byte-identical mkit objects, the sha1↔blake3 map is a rebuildable cache under the same key, and a crashed import is resumed by re-running it.
refs/notes, refs/replace, grafts.Implementations SHOULD obtain upstream objects via a full
git clone --mirror into the per-remote staging repository (§6) and
read objects with git cat-file --batch (git owns wire protocol,
auth, and pack storage; the bridge owns only translation). Commit
order MUST be a topological parents-first order (git rev-list
--reverse --topo-order). The inbound parser MUST be tolerant of
everything git accepts — multi-line continuation headers (gpgsig,
mergetag), unknown headers, the encoding header, malformed
person lines in historic commits — because this is an
untrusted-input boundary; tolerance means parse and carry or refuse
loudly, never crash or silently alter.
objectFormat = sha256 upstreams MUST be refused at clone time.
git blob bytes map verbatim. Content at or below the 1 MiB chunking
threshold becomes one mkit blob; above it, the pinned FastCDC
chunker (SPEC-FASTCDC) produces chunk blobs + a chunk_size = 0
manifest — exactly the writer-side rule, so imported stores are
exportable (SPEC-GIT-BRIDGE §4 refusals never trigger on them).
SPEC-FASTCDC leaves the threshold an implementer choice; for
importers CHUNK_THRESHOLD = 1 MiB is normative (a different
threshold would fork hashes across implementations).
Blobs over worktree::MAX_FILE_BYTES (1 GiB, the per-file cap)
refuse per-ref.
mkit field ← git source
tree_hash ← translated tree
parents ← translated parents, order preserved
author ← Identity::Opaque(author name + " <" + email + ">")
signer ← the importer key (§4)
message ← message bytes VERBATIM (no UTF-8 constraint;
`encoding` header tolerated and dropped)
timestamp ← git COMMITTER epoch seconds
message_hash ← zero
content_digest ← BLAKE3(framed raw git commit bytes) (§5; advisory)
signature ← Ed25519 by the importer under COMMIT_DOMAINPinned rules:
author keyword+space through the last > byte that
is preceded anywhere on the line by a < (interior spacing,
doubled spaces, missing space before <, and stray </> inside
the slice all preserved — no recomposition, so historic
malformations hash identically across implementations). Lines with
no such byte (no < at all, or < with no closing > anywhere
after it) use the bracket-less rule: the remainder after author ,
with one trailing match of the pattern space, decimal seconds,
space, + or -, four digits stripped. A person line from which
no timestamp can be parsed (no token after the identity slice, or
no trailing pattern on a bracket-less line) refuses the ref —
commits and tags always need a timestamp. Payloads that would be
empty or exceed 4096 bytes refuse per-ref.gpgsig,
mergetag, encoding, and any unknown headers are not
represented in the mkit commit; they are recoverable from the
retained raw bytes (§5).MAX_PARENTS).COMMIT_DOMAIN — no new signing domain.
Commits carry no domain marker and verify_commit hardcodes the
commit domain; a separate import domain would make imported commits
unverifiable everywhere. Distinguishability comes from the
git-import/v1 predicate, the DSSE keyid, and the dedicated import
key (§4) — the same principle SPEC-GIT-BRIDGE §11 pins for export.Entries re-sort from git order (directories keyed name + "/") to
mkit byte-lex order. Canonical modes map 100644→0x01, 40000→0x02,
120000→0x03, 100755→0x04. Historic non-canonical mode SPELLINGS
normalize to their canonical equivalent's mkit mode, enumerated
exhaustively: 100664→0x01, 100640→0x01, 100600→0x01,
040000→0x02 (zero-padded directory). Any other mode string refuses
per-ref. Normalization is declared-lossy (warned once per ref) —
except in a state dir whose recorded direction is fork (§6),
where a normalized tree could not reproduce its original sha1 and
the affected ref MUST refuse instead.
160000 (gitlink/submodule) refuses per-ref with an actionable
message. Two structural refusals close gaps git can represent but
mkit cannot decode: entry names that collide byte-equal after the
re-sort (a file and a directory of one name) refuse per-ref, and
trees nesting beyond 128 levels (mkit's MAX_TREE_DEPTH defense)
refuse per-ref. Entry names are validated by SPEC-OBJECTS §4.1
(deserialize-time rules — non-negotiable): names that are .git/
.mkit case-insensitive, end in dot/space, are Windows device stems
(aux.c-class), exceed 255 bytes, or contain backslash refuse the
ref.
Annotated and signed git tags map to mkit tag objects signed by the
importer under TAG_DOMAIN (same vouch semantics as commits; the
original tagger line maps to the tagger Identity by the §3.2 rule
and the mkit tag timestamp is the tagger epoch, negative-refused
like commits; git tag GPG signatures ride in the retained raw bytes
only). Historic tagger-less tags (git v0.99 era) take the PINNED
sentinel Identity::Opaque("(no tagger)") and timestamp 0 — any
other choice would fork tag hashes across implementations. The tag
name must satisfy the mkit single-segment tag grammar
(SPEC-GIT-BRIDGE §7.1) or the tag refuses. Lightweight tags map to
bare refs. Tag targets follow the object mapping; tag→tag chains are
bounded at 16. The mkit tag object has no annotation slots, so tag
provenance lives only in §5's retained bytes + attestation.
Upstream refs/heads/* and refs/tags/* map to
refs/remotes/<remote>/<branch> tracking refs and refs/tags/<name>
respectively (fresh-clone form also sets refs/heads/<default> and
HEAD). Ref names outside the SPEC-REFS §3 grammar (@, +,
unicode, .lock suffix, HEAD final segment) refuse per-ref. The
refusal UX matches export: skip-and-warn per ref, non-zero exit when
everything was skipped.
Three layers, from authoritative to advisory:
git-import/v1 attestation (authoritative): one DSSE/in-toto
attestation per imported ref head per fetch, signed with the
importer's configured signer. predicateType =
https://github.com/officialunofficial/mkit/spec/predicate/git-import/v1.
subject[0] = {name: <full mkit ref>, digest: {blake3: <mkit
head hash>}}. Predicate (JCS key order):
{
"gitCommit": "<40hex sha1 of the upstream head>",
"refName": "<full mkit ref name>",
"remoteUrl": "<canonical remote identity, §8>",
"schemaVersion": 1,
"specVersion": 1
}Because parents are inside signed commit bytes, a head attestation
transitively pins the imported closure. Envelopes are stored like
any attestation (.mkit/attestations/<commit>/…). Importers
SHOULD skip re-minting for a head whose recorded imported state is
unchanged (mirroring SPEC-GIT-BRIDGE §11's guidance).
"<type> <len>\0" + body — stored
under the per-remote state dir, keyed by their sha1 preimage purely
as a lookup key (this framing is normative for both retention
and the content_digest input; body-only would fork digests across
implementations). Small (commits/tags only — trees/blobs are
recoverable from the staging mirror and re-derivable from the mkit
twins). Consistent with SPEC-GIT-BRIDGE §2/§11: sha1 here is a
locator, never a security proof. Keying storage by sha1 does not
make the retained bytes trustworthy on its own — an adversary who
can compute a sha1 collision (practical for chosen-prefix collisions
since 2020) could in principle plant different bytes under the same
key. What actually establishes trust is described in point 3 below.content_digest
= BLAKE3(raw git commit bytes) is carried as an advisory field,
excluded from signing bytes (SPEC-OBJECTS §5.1) and therefore
malleable in isolation — a signature-preserving sibling with a
forged slot is constructible. Verifiers MUST treat content_digest
as a hint for locating/diffing retained bytes, never as something
the signature vouches for.Correspondence audit: a signed head binds the mkit closure but does not by itself prove that a mutable mapping cache pairs each Git object with the correct mkit object. An auditor MUST derive each reachable imported object's unsigned fields from retained Git bytes under the recorded import-spec version, compare all translated fields and edges, and verify its signature with the pinned public key. It MUST validate a required head's attestation claim against the exact subject hash, full ref, Git ID, canonical source, predicate and schema versions. At least one complete matching claim is required; unrelated extra envelopes do not qualify and may be ignored. Native bridge objects' mapped parent/tree edges are checked against the same correspondence graph. Audit requires no private signing material and MUST NOT rewrite objects or caches. See SPEC-GIT-BRIDGE §14.3 for traversal limits and failure behavior.
Collision-resistant hashing (normative requirement): because sha1
here still functions as a lookup key over data an adversary may control
(untrusted upstream Git history), every sha1 computed over upstream
bytes in the bridge/import path MUST use a collision-detecting SHA-1
construction (rejecting known cryptanalytic collision patterns), not a
plain SHA-1 implementation — matching real Git's own practice for
exactly this reason. mkit-git-bridge's current dependency (sha1 =
"0.11") is a plain implementation and does not meet this requirement.
State lives in .mkit/git/<remote>/ (layout implementation-defined,
non-normative except for the binding semantics below):
dest (export-direction dirs)/source: the canonical remote
identity (§8), immutable after first use. One state dir = one
upstream. Fork-direction dirs record dest informationally only
(last push target) — fork pushes lease against a fresh observation
per SPEC-GIT-BRIDGE §14.1, so they are not destination-bound.direction: import, export, or fork (passthrough-enabled).
A state dir MUST NOT serve import and plain export simultaneously;
mixed use is refused at open. (fork combines an import source
with passthrough export per SPEC-GIT-BRIDGE §14.)signer: the pinned importer pubkey (§4).import-spec-version (§1.2).repo.git): for import it is durable
state, not a disposable cache — it is the byte-perfect original
archive backing §5.2 and the passthrough object source. Deleting
it forces a re-clone but loses nothing else.Before translating, an import MUST run two checks:
content_digest ==
BLAKE3(candidate's raw bytes) exists in the store but was signed
by a different key, refuse with guidance ("this upstream is
already imported here under key <hex8>; pull from the designated
importer over mkit transport, or install that key"). A linear
store scan is an acceptable lookup mechanism (imports are rare,
interactive operations); an index is permitted but not required.The probe is best-effort (content_digest is advisory) — it catches the realistic accident, not an adversary.
fetch semantics: git fetch in the staging mirror, translate the
new closure under the pinned key (incremental: map hits skip
subgraphs), advance refs/remotes/<remote>/*. Tracking refs are
not user work: an upstream force-push moves them with a loud
warning naming the rewound ref and old/new tips; map entries for
abandoned commits stay valid forever (determinism).pull = fetch + fast-forward of the current branch from its
tracking ref, reusing the native FF machinery and guards.
Divergence refuses with the executable hint
(mkit merge <remote>/<branch> / mkit rebase); integration is
native — imported commits are ordinary mkit objects.All bindings, guards, and attestation remoteUrl fields use one
normalization, remote_identity(dest):
[user@]host:path rewrites to ssh://host/path
(honesty note: a home-relative scp path and an absolute ssh:// path
can be different repos on a generic server; the conflation only
widens the guard — fail-closed). IPv6 literals keep their
brackets: [::1]:path → ssh://[::1]/path;ssh 22, https 443, http 80, git
9418);/ and one trailing .git stripped from the
path;file:// URLs (file:///p, file://localhost/p)
collapse to the symlink-resolved absolute local path.Equivalence examples (all one identity):
git@github.com:org/repo.git ≡ ssh://github.com/org/repo ≡
ssh://GIT@GITHUB.COM:22/org/repo/. The origin guard
(SPEC-GIT-BRIDGE §14.2) compares canonical identities: plain
(non-passthrough) export MUST refuse a destination whose identity
matches any recorded import source. This is a safety net against
accidents, not a security boundary — mirrors and redirects are
undetectable; the push lease remains the backstop.
Pinned under rust/tests/golden/git-import/ with the standard
MANIFEST convention; each vector records the source git object bytes,
the imported mkit object bytes under the fixed test key, and both
ids. Single exception: vector 9's > 1 MiB content is regenerated
deterministically by the golden test rather than committed; its two
ids are pinned like every other vector (same carve-out as
SPEC-GIT-BRIDGE §13's chunked vector). Refusal vectors (gitlink tree,
reserved tree-entry name, negative timestamp, out-of-grammar tag
name) pin their TYPED refusal in REFUSALS.txt alongside the
success vectors.
encoding header — verbatim bytes.100664 tree — normalize-declared-lossy.Refusal vectors (assert the typed refusal, not bytes): gitlink tree,
aux.c tree name, negative timestamp, v1.0+build ref name,
sha256 repo marker.
Every vector MUST be byte-stable across two runs (determinism) and
MUST verify_commit/verify_tag under the test key.
| Version | Changes |
|---|---|
| 1 | Initial importer-signed mapping: blobs/trees/commits/tags, refusal matrix, dedicated-key + pinning model, three-layer provenance, canonical remote identity, fetch/pull semantics, golden vectors. |
| Invariant | Enforced by |
|---|---|
| Re-running an import reproduces byte-identical mkit objects | import is a pure function of (upstream bytes, importer key, import-spec version); RFC 8032 signing is deterministic (§1.2) |
| Two importers under the same key produce identical hashes; under different keys, unrelated forks — never a silent mix | hashes are a function of the signing key (§1.2); the pinned key refuses imports under any other key (§4) |
| Translation-rule drift can never silently fork hashes for one key | recorded import-spec version; unimplemented versions refuse incremental extension (§1.2) |
| Every imported commit/tag carries a verifiable importer vouch, never an authorship claim | importer Ed25519 under the existing COMMIT_DOMAIN/TAG_DOMAIN; attribution rides in the author/tagger Identity (§1.1, §3.2, §3.4) |
| Historic malformations (person lines, modes, tagger-less tags) hash identically across implementations | byte-exact identity-slice rule, exhaustive mode-spelling table, pinned (no tagger) sentinel (§3.2, §3.3, §3.4) |
| Untrusted upstream input is never silently altered | parse-and-carry or typed per-ref refusal — the §3 refusal matrix; never crash or normalize undeclared (§2, §3) |
| Imported stores are always exportable | normative 1 MiB threshold + writer-side FastCDC rule, so SPEC-GIT-BRIDGE §4 refusals never trigger (§3.1) |
| The translation is byte-auditable after the fact | framed raw commit/tag bytes retained sha1-addressed; head attestation transitively pins the signed closure (§5.1, §5.2) |
A forged content_digest cannot defeat verification | the slot is advisory and excluded from signing bytes; the mkit-side signed attestation (BLAKE3 subject + Ed25519 signature), not the sha1-addressed retained bytes, is the proof surface (§5.3) |
| One state dir tracks exactly one upstream, in exactly one direction | canonical remote identity (§8) immutable after first use; mixed import/plain-export use refused at open (§6) |
| Re-importing an already-imported upstream in the same store is caught | state-dir identity check (structural) + content probe (best-effort) (§6.1) |
| An upstream force-push never silently rewrites local work | tracking refs move with a loud warning; the current branch advances only by fast-forward (§7) |
The content probe (§6.1) is explicitly best-effort — it catches the realistic accident, not an adversary; the pinned-key refusal (§4) is the structural guarantee.