SPEC-TRANSPORT-ENC.md
In docs/specs/ · At edb461cCopy the commit ID
In docs/specs/ · At edb461cCopy the commit ID
mkit-transport-enc is the encrypted-stream sibling of
mkit-transport-ssh. It implements the same seven-verb
Transport trait, exchanging
the same SshFrame protobuf
messages, but over an authenticated, encrypted byte stream provided by
commonware-stream::encrypted
instead of a system ssh(1) child process.
The motivation is to give mkit a self-contained "always-encrypted" transport for environments where shelling out to OpenSSH is awkward (WASM, embedded targets, browser-hosted Workers) while keeping a single source of truth for verb-level framing across SSH and encrypted paths.
The transport has two forms: a deterministic in-process scaffold
(EncSession/from_session) exercised by the in-tree round-trip
test suite, and the real TCP transport used in production — URL
parsing, connect_tcp/serve_tcp, the mkit-cli/enc-transport
feature gate, and the serve_tcp_* listener helpers. The transport is
deprecated: no maintained server hosts a listener (use SSH mkit serve
or Connect). §6 describes the TCP transport's mechanics and its current limitations.
| Scheme prefix | Transport | Notes |
|---|---|---|
mkit+enc://[user@]host[:port][/path]?pubkey=<hex-or-b64url> | mkit-transport-enc | Implemented by mkit_transport_enc::url::parse_enc_url. The user@ and /path components are accepted for round-tripping but not consulted by the handshake; trust is via the ?pubkey= payload only. |
The URL carries the server's static ed25519 public key as a
query-parameter pubkey=<…> in one of two equivalent encodings:
[A-Za-z0-9-_]. The trailing 2 bits of the final base64 character
MUST be zero; otherwise the parser rejects the URL to prevent
two distinct strings from decoding to the same payload.Trust is established by out-of-band knowledge of this key — there is
no fall-back to a TOFU-style first-use cache, and no CA chain. If the
server's actual key during handshake differs from the URL-advertised
key, the dialer aborts with EncInitError::PeerRejected (mapped from
commonware_stream::encrypted::Error::PeerRejected).
The default TCP port when the URL omits one is 9418 (same as git's daemon, reused for operator familiarity).
mkit+enc://h.example?pubkey=0000…0000 # bare host + hex
mkit+enc://h.example:7777?pubkey=0000…0000 # explicit port
mkit+enc://alice@h.example/projects/x?pubkey=… # user + path (round-trip only)
mkit+enc://h.example?pubkey=AAAAAA…AAA # b64-url, 43 charsThe encrypted layer is provided verbatim by
[commonware_stream::encrypted] (pinned to the commonware train fixed in
rust/Cargo.toml, =2026.9.0 as of this revision). mkit does not implement its own handshake or AEAD;
this section documents the parameters mkit configures.
| Field | Value | Rationale |
|---|---|---|
namespace | b"mkit/transport-enc/v1" | Pre-shared transcript binding. Prevents cross-application replay if peers share ed25519 keys with another commonware-stream user. Bumping the /v1 suffix is the lever for a hard cryptographic break in future. |
max_message_size | mkit_rpc::MAX_FRAME_BYTES (1 MiB) | Matches the inner SshFrame framing cap so a maximally-sized verb fits in one encrypted record. Removes an off-by-overhead class of bugs where one limit sneaks past the other. |
synchrony_bound | 30 s | Tolerable clock skew between client and server. Generous, sized to keep flaky-CI failures out of the in-tree round-trip test; a production deployment operating over real, higher-latency networks SHOULD configure a tighter bound (≤ 5 s). |
max_handshake_age | 30 s | Same envelope as synchrony_bound; the same tightening applies for production use. |
handshake_timeout | 60 s | Hard ceiling for handshake completion. Generous for the in-process test scaffold; a production deployment SHOULD tighten to ≤ 10 s. |
Both peers carry an ed25519
static keypair. mkit chose ed25519 (over BLS) for the in-process scaffold because:
commonware-cryptography::ed25519 is already a stable Signer and
needs no application glue.A later revision may add BLS as an optional alternative (the
commonware_stream::encrypted API is generic over Signer), but the
default — and the only form covered by the in-process scaffold's CLI — is ed25519.
Inherited from commonware-stream::encrypted (verbatim from its
documented guarantees):
SynAck is transcript-bound to Syn,
preventing replay across sessions.Not provided (also inherited):
mkit/<version> in the
application-level Hello frame anyway, so nothing is lost.The application protocol on top of the encrypted stream is the same
SshFrame protobuf message set used by mkit-transport-ssh, defined
in ssh.proto. Verb
semantics, error mapping, ref-CAS encoding, and pack-streaming chunk
boundaries are byte-for-byte identical to SPEC-TRANSPORT §4.
The one deliberate departure from the SSH transport's wire is the
inner framing. mkit-transport-ssh wraps each SshFrame in a 4-byte
LE u32 length prefix (defined in
mkit-rpc/src/framing.rs)
because SSH's stdin/stdout pipe is an unframed byte stream.
commonware-stream::encrypted already frames each ciphertext record
(one varint length prefix per Sender::send call), so the inner
SshFrame rides as a single protobuf payload per encrypted record
— no second length prefix. One SshFrame send produces exactly one
Sender::send, and one Receiver::recv returns exactly one
SshFrame's worth of bytes.
Concretely:
┌──────────────────────────────────────────────┐
│ SshFrame protobuf body (verb message) │
├──────────────────────────────────────────────┤
│ ChaCha20-Poly1305 ciphertext + 16-byte tag │ per direction, counter nonce
├──────────────────────────────────────────────┤
│ commonware-stream varint length prefix │ framing
├──────────────────────────────────────────────┤
│ Sink / Stream byte transport │ TCP (production) or mocks::Channel (in-process test scaffold)
└──────────────────────────────────────────────┘The 1 MiB ceiling on MAX_FRAME_BYTES is enforced twice — once by
commonware-stream via max_message_size (see §2.1), once by every
SshFrame consumer that already exists. Either layer will reject a
non-conforming peer.
After the encrypted handshake completes, the client MUST send a
Hello frame with proto = PROTOCOL_VERSION_1 and client_id =
"mkit <semver>". The server MUST reply with a HelloResponse whose
proto matches. If either side disagrees, both close the connection
without further verb exchange. As over ssh (SPEC-TRANSPORT §4.2), a
server answers a first frame that is not Hello with
Error{INVALID_REQUEST, "first frame must be Hello"}, and a Hello for
another version with Error{INVALID_REQUEST, "unsupported proto_version
<n>"}, before it closes; it serves no verb on that connection. (The
former mkit serve --listen-enc closed without replying.)
This Hello is layered on top of the encrypted channel, not inside
the encrypted handshake. It mirrors what mkit-transport-ssh does and
keeps the protocol-version dance in one place.
| Cap | Value | Source |
|---|---|---|
| Max single encrypted record | 1 MiB | mkit_rpc::MAX_FRAME_BYTES |
| Max ref name / prefix | 512 bytes (SPEC-REFS §3) | MAX_REF_NAME in mkit-rpc/src/helpers.rs (re-exported by mkit-transport-enc), equal to mkit_core::refs::MAX_REF_NAME_BYTES |
| Per-chunk pack data | 800 KiB | CHUNK_DATA_MAX; same as mkit-transport-ssh |
| Pack body total | mkit_core::protocol::PACK_BODY_LIMIT | Shared client cap |
The in-tree test suite lives in
mkit-transport-enc/src/lib.rs
under #[cfg(test)] mod tests. Tests run inside a single
commonware_runtime::deterministic::Runner so they exercise the same
async code paths the TCP transport's tokio wiring hits, without
depending on wall-clock time or a multi-threaded executor.
| Test | What it pins |
|---|---|
hello_and_list_refs_roundtrip_over_ciphertext | End-to-end: encrypted handshake → app Hello → ListRefs round-trip → assert two refs decoded. Also captures bytes leaving the dialer's Sink and asserts the literal "refs/heads/" plaintext does NOT appear in the capture — the bytes on wire are ChaCha20-Poly1305 ciphertext. |
handshake_rejection_surfaces_peer_rejected | Server's bouncer unconditionally returns false; client's dial MUST resolve to an error and the server-side outcome MUST be EncryptedError::PeerRejected. |
peer_rejected_error_maps_to_init_error | Pure unit test: EncryptedError::PeerRejected(_) → EncInitError::PeerRejected. Catches regressions in the From impl even if a future commonware release moves which side surfaces the rejection. |
url::parse_enc_url::* (~25 cases) | URL parser: pins accepted forms (mkit+enc://[user@]host[:port][/path]?pubkey=<hex|b64url>), both pubkey encodings, and rejection of bad inputs (missing prefix / pubkey, port overflow, CRLF / NUL injection, .. path segments, b64 trailing-bit ambiguity, duplicate / unknown query params). |
tcp::executor_handles_repeated_block_on | TokioExecutor::block_on is safe to call repeatedly: a task spawned during the first block_on is still alive when a later block_on awaits it, so a refactor that drops and rebuilds the runtime between calls fails the test. |
tcp_e2e::list_refs_round_trip_over_real_tcp (gated on --features tcp) | End-to-end TCP: real TcpListener on a free port, real connect_tcp dialer via an in-test byte-sniffing proxy. Asserts (a) Transport::list_refs round-trip succeeds and (b) the proxy never observes the literal "refs/heads/" prefix the client sent — bytes on wire are ChaCha20-Poly1305 ciphertext. |
mkit_transport_enc::url::parse_enc_url accepts
mkit+enc://[user@]host[:port][/path]?pubkey=<hex|b64url> per §1.
EncTransport::connect_tcp opens a tokio TcpStream, drives
commonware_stream::encrypted::dial against the URL-advertised peer
pubkey, and returns a fully-wired synchronous Transport. The dial is
single-roundtrip; verb calls block_on a per-call future through the
TokioExecutor. Custom Sink/Stream wrappers
(tokio_io::{TokioSink, TokioStream}) adapt tokio's OwnedReadHalf /
OwnedWriteHalf to commonware_runtime::{Sink, Stream} because the
upstream pub(crate) types aren't reachable from outside the runtime
crate.
remote_dispatch::open recognizes the mkit+enc:// scheme behind the
mkit-cli/enc-transport cargo feature; default builds remain SSH-only.
The standalone listener (mkit-server serve --listen-enc) was removed with
the native server, and the CLI's former mkit serve --listen-enc before it;
no maintained server hosts mkit+enc://. The library keeps the pieces a
listener is built from: mkit_transport_enc::serve_tcp_listener, the async
accept loop, runs on the embedder's runtime, and each session can be served
with mkit_server::ssh::serve_session over a pipeline, as the
TransportPeer principal holding the key the handshake authenticated; its
verb replies are therefore the ssh session's (§3). Under multi-repository
addressing such a listener binds every session to one configured
repository (SPEC-TRANSPORT-CONNECT §7.4): nothing in the session's frames
can select another.
A listener MUST be fail-closed (issue #178): it refuses to serve unless
it is given an allowlist of client public keys or an explicit allow-any
development escape. The accept loop consults a PeerPolicy —
AllowAny (dev / the explicit unsafe escape) or
Allowlist(HashSet<[u8;32]>) (one client pubkey per line in the file
form, 64-hex or 43-char url-safe base64; # comments and blank lines
ignored). serve_tcp_with_addr retains AllowAny for the direct e2e
harness only. The allowlist bouncer rejects any unlisted dialer at the
handshake — a rejected peer never receives a HelloResponse,
list-refs, packs, or update-ref.
A listener bounds each client: a handshake deadline (§2.1, default 10 s), a per-frame idle timeout after it (applied to every frame read and write), and the per-connection budgets of SPEC-TRANSPORT §4.4. A listener caps the connections in the handshake apart from the sessions, so clients that connect and never handshake cannot hold the slots of authorized ones. On shutdown it stops accepting and ends each session at its next frame boundary (never inside an upload), within its grace period.
The server identity is a stable raw-32 key loaded, or created on
first run, from an operator-supplied path (there is no default path), so the advertised ?pubkey= is stable across restarts;
only the unsafe allow-any mode without a key file keeps an ephemeral
per-process key. A client can similarly pin its identity via the
MKIT_ENC_CLIENT_KEY environment variable (a user-scoped or
CLI-supplied raw-32 key file) so an allowlisting server can pin the
client across restarts; when unset, the client derives an ephemeral
per-process key, which only an allow-any server will accept. Peer-authorization and identity key paths are CLI-supplied
or user-scoped and are never read from repo-local .mkit/config.
connect_tcp lazily bootstraps a commonware_runtime::BufferPool by
spinning up a one-shot commonware tokio Runner on a fresh OS thread,
then dropping the bootstrap runtime while holding an Arc clone of the
pool; the pool is cached process-wide. This exists because
commonware-runtime does not currently expose a public
BufferPool::new_for_network() constructor.
Server and client identities are stable raw-32 key files on disk
(a listener's server key and MKIT_ENC_CLIENT_KEY), not yet routed through
mkit-keystore the way SSH host keys and signing keys are; keystore
integration would also let the public connect_tcp signature take a
keystore-backed key type instead of a raw one. The allowlist is a flat
authorized-peers file, not a keystore partition.
EncTransport is connection-oriented (a single long-lived encrypted
session), so it follows the same reconnect-before-retry pattern as
SshTransport (SPEC-TRANSPORT §4.5) rather than the request-per-attempt
shape HTTP/S3 use. Orphan rules prevent mapping
commonware_stream::encrypted::Error variants individually, so
stream_err collapses every cipher-layer I/O failure — a send, a
receive, a decrypt — to TransportError::ConnectionFailed; any such
failure can leave the session's per-record nonce counters desynced
between client and peer, so resuming on the same session is never safe
once a failure is observed.
Every verb is driven through [mkit_core::protocol::retrying]. A
ConnectionFailed marks the session dead; the next attempt calls
ensure_connected, which redials through the transport's ReconnectFn
(when one is wired — tcp::connect_tcp wires a real redial closure
that reruns the full dial and commonware_stream::encrypted::dial
handshake) and redoes the application Hello/HelloResponse exchange
before the verb is re-issued against the fresh session. A transport
built directly from an already-established session with no redial
capability (for example the in-process test harness) has no ReconnectFn
wired, so a dead session surfaces ConnectionFailed immediately
instead of retrying — matching pre-retry behavior for those callers.
As with SSH, upload_pack is safe to resend in full on the fresh
session; update_ref is not idempotent across retries per
SPEC-TRANSPORT §7, so a retried CAS write still requires the caller's
read_ref disambiguation on RefConflict.
Application protocol version is the same as the rest of mkit
(PROTOCOL_VERSION_1). The cryptographic-namespace prefix
(mkit/transport-enc/v1) is bumped independently if the handshake or
record layer needs a hard break that the application-level
HelloResponse mismatch can't catch.
| Invariant | Enforced by |
|---|---|
| The peer is the key the URL advertises, or no session exists | ?pubkey= carried out-of-band in the URL; mismatch → EncInitError::PeerRejected; no TOFU cache, no CA fallback (§1) |
| Two distinct URL strings cannot decode to the same pubkey | hex (exactly 64 chars) or unpadded b64url (43 chars, trailing 2 bits zero) — ambiguous encodings rejected at parse (§1) |
| Mutual authentication and forward secrecy | static-key signatures over the handshake transcript; ephemeral X25519 (§2.3) |
| No nonce reuse, no cross-session replay | per-record counter nonces never on the wire; SynAck transcript-bound to Syn (§2.3) |
| No cross-application replay of handshakes | namespace = b"mkit/transport-enc/v1" transcript binding (§2.1) |
| No frame exceeds 1 MiB — enforced twice | max_message_size = mkit_rpc::MAX_FRAME_BYTES at the record layer (§2.1) and by every existing SshFrame consumer (§3.1) |
One SshFrame per encrypted record, no framing ambiguity | single protobuf payload per Sender::send; no second length prefix (§3.1) |
| No verb exchanged before version agreement | post-handshake Hello/HelloResponse with PROTOCOL_VERSION_1; disagreement closes the connection (§3.2) |
| The listener is fail-closed: an unlisted peer gets nothing | binding requires an authorized-peers allowlist (or an explicit allow-any escape); rejected peers never receive a HelloResponse, refs, or packs (§6.1) |
| Verb semantics never diverge from the SSH transport | same SshFrame message set; semantics byte-for-byte per SPEC-TRANSPORT §4 (§3) |
| Application plaintext never appears on the wire | pinned by the byte-sniffing round-trip tests (§5) |
A ConnectionFailed never resumes writes/reads on the same (possibly desynced) session | every verb driven through retrying; a dead session is redialed and re-Hello'd before the next attempt, never reused (§6.3) |
Explicitly not guaranteed, inherited from the stream layer: anonymity (peer identities exchanged in cleartext) and length padding (message sizes leak) (§2.3).