README.md

Server conformance

The protocol contract is the normative specifications plus conformance. Any server implementation can run these suites. Reference-server storage layouts are private to its test binding.

• Tier 1: wire. Ordinary transport HTTP requests, a base URL, and a declared wire profile. No server control is required.
• Tier 2: server control. The wire URL and profile plus the version 1 test-only HTTP binding below, at the same or a separate origin. Cases combine wire operations with controlled restart, authority changes, barriers, and atomic observations.
• Optional storage SPI suite. Enable storage-spi to implement storage::KvHarness and invoke storage_suite!. This tests the reference server's storage API; other implementations do not have to implement it.

The default library and runner expose Tier 1, Tier 2, and their message types without a dependency on mkit-server. The optional storage-spi, stubs, and test-host features add reference-server support. An out-of-tree server needs only the conformance dependency, its base URL and profile, and its own test-control binding; it need not implement a Rust trait or adopt the reference storage SPI.

Out-of-tree runner

This crate is prepared for the next lockstep release; it is not published by this change. Until that release, use a pinned git dependency. After release, add mkit-server-conformance at the same released version as any other mkit dependencies, or install its binary with cargo install mkit-server-conformance. The published package includes its fixtures; no workspace paths are required.

use std::collections::BTreeMap;
use mkit_server_conformance::{controlled, wire::{Profile, WireTarget}};

# async fn check(profile: Profile, expected_skips: BTreeMap<String, String>) {
let target = WireTarget {
    base_url: "http://127.0.0.1:8080".parse().unwrap(),
    profile, // the server's declared signed-write profile
};
let wire_report = mkit_server_conformance::wire::run(&target, None).await;
let control_report = controlled::run(&target, &expected_skips, None).await;
assert!(!wire_report.failed(), "{}", wire_report.tap());
assert!(!control_report.failed(), "{}", control_report.tap());
# }

mkit-server-conformance wire --base-url http://127.0.0.1:8080 \
  --auth auth-v2 --audience http://127.0.0.1:8080 --repository default \
  --random-signer --atomic-advance
mkit-server-conformance controlled --base-url http://127.0.0.1:8080 \
  --profile conformance.toml --declared-skips declared-skips.json

Both commands print TAP and exit 1 on a failed case, 2 on usage errors. --list-cases prints each tier's registry. --filter selects case names. The wire profile declares authentication, optional features, and fixture limits; wire::ProfileSpec documents all TOML fields. Controls never enable production capabilities. Run Tier 2 only against an isolated disposable test instance.

Standalone run and differential commands

Install the portable runner at the same immutable revision as your server:

cargo install --git https://github.com/officialunofficial/mkit --rev <revision> \
  --locked mkit-server-conformance

The run command runs Tier 1 and Tier 2 without linking a Rust dependency. Specify the transport URL and the test-control binding separately when needed:

mkit-server-conformance run --base-url http://127.0.0.1:8080 \
  --control-url http://127.0.0.1:8081 --profile conformance.toml

Copy the packaged example runner profile to conformance.toml. Edit its origin, repository, limits, features, controls, and known gaps to match your disposable server, and set CONFORMANCE_SIGNER_SEED to a 64-character hex test seed. Installing the binary does not copy fixture files into your working directory.

The version 1 runner profile contains the existing wire profile under [wire] and test-control declarations under [control]. These declarations describe an already isolated test host; they never enable production routes.

version = 1

[wire]
auth = "auth-v2"
audience = "http://127.0.0.1:8080"
repository = "default"
signer_seed_env = "CONFORMANCE_SIGNER_SEED"
atomic_advance = true
run_id = "standalone"
features = ["tickets", "-authority-fencing", "-authority-setter", "-fork"]
relay_lag_bound_ms = 0

[control]
controls = ["clock.pin", "observe", "restart.graceful"]

[control.declared_skips]
"acceptance.kill_recovery" = "This test host retains process-local memory only."

Use --tier wire or --tier controlled to select one tier, and --filter to select case names. Every selected case appears in the report. Missing optional features are not applicable; missing controls require a concrete declared skip for the applicable case. Negotiation errors fail setup. Even --tier wire negotiates the test binding. The optional [control] field url sets its origin; --control-url overrides it.

Install the embedded reference test utility for differential runs:

cargo install --git https://github.com/officialunofficial/mkit --rev <revision> \
  --locked --features reference-runner mkit-server-conformance
mkit-server-conformance diff --base-url http://127.0.0.1:8080 \
  --control-url http://127.0.0.1:8081 --profile conformance.toml

diff starts a fresh reference server on loopback and drives the same canonical fixtures against it and the target. Both test bindings pin business time; fixed keys, nonces, and timestamps make each logical operation repeatable. Run against disposable instances with no other writers. Differential writes use an Ed25519 test key whose public seed is 32 bytes of 0x75, independently of the profile's Tier-1 signer seed. A multi-repository target must admit that key's owner namespace and the repository named differential-v1; a single-repository target uses the configured repository. The reference host supports its declared test profile; unsupported profiles fail setup explicitly. Indexed differential profiles declare multi-repo, indexed-mode, tickets, auth v2, and atomic advance. The native CLI regression exercises that profile without dropping its features; the small canonical packs complete verification within the inline bound. This does not establish scheduled-verification equivalence.

Normalization excludes allocated server identifiers, timing fields, and Retry-After values. Refusal comparison uses protocol error codes. Replay fingerprints are first checked against each exact signed operation, including its target audience, then normalized to that operation's logical role. Each server must replay its original receipt bytes. Receipt issuance, signatures, and statement fields are outside these differential fixtures; they require separate receipt conformance coverage. Atomic snapshots retain ref pairs, ticket and reservation order, membership, terminal result multiplicity, and quota counters. The report names each differing field. Two failed executions never count as passing equivalence.

Differential results establish decision equivalence for these deterministic fixtures. They do not establish crash, recovery, or concurrency behavior; Tier 2 covers those properties separately. Cloud Build's all-feature workspace suite exercises two native reference hosts over HTTP.

Both commands write a versioned JSON report to stdout by default; run also supports --format tap. JSON contains case results and differences, without credentials or signing material. Exit codes are 0 for a passing run, 1 for a failed case or differential mismatch, and 2 for usage, profile, setup, negotiation, or runner execution-limit errors. Existing wire and controlled commands retain TAP output.

Differential decisions have a 45-second execution budget. The runner honors valid pending-verification polling hints clamped to 1–60 seconds. If a mandatory wait cannot fit, the JSON case status is runner_error and the command exits 2; this is inconclusive, not a protocol failure or a match. An already observed protocol violation remains failed with exit 1 even if a later step or the other target reaches this budget. Rerun against a disposable fixture whose verification resolves within that budget. The runner never polls early to avoid the wait. Ordinary protocol refusals retain their failure result.

Version 1 test-control binding

Protocol requirement (test interface only). This interface is optional test infrastructure, outside the public transport wire. A production build MUST NOT mount these routes. The reference binding requires test-host. Optimized builds additionally require the explicit reference-runner test utility feature. Production artifacts never enable these features. A server under test implements the binding in its own test host; it need not use Rust or any reference storage layout.

GET /__mkit_test/v1/capabilities returns JSON with version: 1, controls (an array of supported names), durability_profile (the actual failure promise), and unsupported (an object mapping unavailable control names to reasons). Unknown versions, malformed responses, contradictory declarations, and missing negotiation routes fail the run. An unavailable capability is a coverage gap, never a passing assertion.

POST /__mkit_test/v1/control accepts the JSON encoding of control::Request. The operation discriminator and fields are:

Success is HTTP 200 with control::Response JSON. Unsupported controls return HTTP 501 without mutation. Rejected requests return a 4xx response without mutation. The response has optional authority_namespace, authority_statement, observation, now_ms, generation, fork_fixture, fork_observation, publication_observation, reliance_fixture, reliance_observation, and gc_drop_result, plus paused. New optional fields are omitted when absent. Clock controls do not advance monotonic execution budgets or simulate durability.

The four boundaries come from SPEC-SERVER §18.1:

These names identify protocol decisions, not engine steps or storage layouts. A binding advertises only boundaries it can reach. The optional decision_class defaults to ref_acceptance; fork cases select fork_acceptance, fork_progress, or fork_source_proof. For the latter, before_acceptance is after all destination membership effects are staged and before obtaining the final source proof. Its capability is decision.fork_source_proof; fork.final_source_proof asserts that the target implements that complete source proof, not merely per-pack checks. The boundary does not prescribe construction steps. fork_destination_claim selects the decision that first atomically excludes competing destination creation or writes. writer_authority_ready selects an ordinary signed write immediately before acceptance, after it has acquired reusable write authority; resumption exercises that retained authority. These classes require decision.fork_destination_claim and decision.writer_authority_ready, respectively. They expose semantic decisions, not leases or storage rows. The registration race lets a competing writer claim the destination while fork's claim is paused. The acceptance race proves fork exclusion defeats a new writer. The retained-authority race lets a previously ready writer defeat fork registration. Each asserts exactly one winner; it does not require an impossible reverse ordering after authority is already usable. The failed-destination case additionally advances beyond the reference job and lease lifetimes, restarts, and repeats signed read/write probes.

The optional occurrence is one-based and defaults to the first occurrence in that operation's selected decision class. It permits fencing every advancing commit without prescribing a job's internal phases or number of commits. Pause/resume cannot move guard checks earlier.

At after_durable_acceptance_before_response, setting on_resume: "uncertain_storage_result" requests an uncertain storage error after release, while retaining the committed state. It requires fault.uncertain_result. The public error may be internal or unavailable; neither proves abort. The runner verifies the full committed observation, retries the original signed identity, and checks that settlement did not repeat. The after-response restart case first receives the response at the client, then requests restart; it does not need a server-side after_response barrier.

An observation query selects a full repository, head ref head and its implicit paired packmap, optional replay identity (audience, repository, signer, nonce), ticket_ids, reservation_ids, packs, and optional quota window (signer, optional window_start_ms; null selects the current window). Object/pack/ticket IDs and signed hexadecimal fields use lowercase hex; reservation IDs remain opaque. Times use Unix milliseconds. It accepts no storage keys. The observation returns live head/packmap IDs, semantic replay state and saved result class, ticket states, reservation states and terminal-result counts, local live membership, and quota counters. Every returned value MUST coexist at one authoritative observation point. A binding MUST reject queries outside its declared atomic domain rather than combine independent snapshots. Arrays preserve query order. Missing tickets with no closure evidence are reported as null. A binding can report consumed or expired from retained terminal evidence; absence alone MUST NOT be interpreted as consumption.

With empty reservation_ids, infer the reservations bound to ticket_ids in order, followed by the ref reservation bound to replay. The test fixture grants a separate ref reservation and advertises fixture.ref_reservation; its actual reservation names stay private. Terminal counts exclude repeated sink delivery. Fixtures retain replay, ticket closure evidence, and terminal responsibility for the case; timers must not prune asserted evidence during an observation. Result classes include advance_committed, update_ref_committed, and conflict or rejection classes. A replay fingerprint binds the original signed request.

The acceptance fixture uses opaque packs and arbitrary ref targets, advertised as fixture.opaque_refs. An indexed-only test configuration cannot advertise that fixture; its missing capability is a visible coverage gap. The reference control host supports unsharded opaque and indexed modes. The reference indexed configuration requires Multi addressing. Indexed profiles use real signed uploads and advances for publication-control fixtures. Ordinary indexed wire cases remain in Tier 1.

A fork fixture query contains destination, signer, and source_head. prepare_fork returns fork_fixture with the source repository and head, tip, packmap, ordered packs and one objects entry per pack, and advancing_commits. The source is public and readable by the fixture signer. It may have a distinct owner so a visibility change can revoke the forking principal's read access. Its tip/packmap pair must pass content validation once the destination activates. Repeated preparation for fresh destinations yields the same source contents and preparatory-commit count; acceptance is excluded from that count. The count selects observable commit occurrences and imposes no orchestration.

A fork observation query contains destination, the original replay identity, and packs. observe_fork returns fork_observation: accepted, failed, writable, the destination head/packmap, ordered staged_memberships and visible_memberships, progress_commits, the saved result, replay, and reservation outcome. These values coexist at one authoritative observation point. Internal staging may exist while the destination remains unreadable and unwritable. Fork itself creates no destination ref. A saved result includes the source identity, source publication sequence, tip/packmap, pack count and bytes, object count, sorted-pack-set digest, and membership generation.

change_fork_source takes the same query as observe_fork and one change: moved publishes a different head/packmap pair; unpublished removes the admitted publication; visibility_revoked commits a visibility change that makes the source unreadable to the forking principal; denied denies each inherited pack explicitly selected by the nonempty query.packs list. Denial cases use fresh forks and target every imported pack separately, so checking only a final pack cannot establish conformance. The binding MUST commit the selected source change before returning success, and MUST NOT modify the destination. The final-proof cases pause after every queried membership is staged, commit that change, resume, and require not_found / source not found with an inert destination. The post-acceptance cases commit the same changes after durable acceptance and inspect accepted result, replay and terminal responsibility before a replay can repair or hide loss. They then require the original replay to preserve them. Source denial does not waive independently applicable destination denial; these cases assert acceptance stability, not immunity from later takedown.

fork.invoke declares the test-only signed bridge at POST /__mkit_test/fork, which uses the canonical body and auth-v2 procedure described in SPEC-SERVER §9.9. The bridge calls the server's ordinary authenticated fork path. Fixture setup and snapshots do not replace the cases' destination read/write probes. This bridge is outside the public transport, like the other controls.

Publication observations include the original acceptance snapshot plus published_head, published_packmap, live_sequence, published_sequence, deletion_boundary, and published_memberships in query order. These values MUST coexist at one authoritative observation point. The snapshot reports published membership itself, rather than an eventually consistent projection. Eligibility controls affect future prepares; clearance driving only releases outstanding work for the selected branch. A server can use separate clearance and leave combined-publication undeclared.

The reliance fixture MUST seed its base through a valid accepted operation and keep the candidate branch absent. The query declares purpose: gc or inspection; a binding must not infer that choice from a ref name. For gc, on return from setup the base has no live or retained publication root, ticket pin, or hold that independently forbids its prepared GC drop; any required GC mark and wait have already completed at the pinned clock. Its accepted membership remains usable. Fixture setup may construct the required valid history before returning, but must restore the pinned clock and must not fabricate accepted state. The fixture may delete its seeding ref through a valid operation to establish this state. Its returned pack contains an external delta base available only in base_pack; head identifies the candidate object. packmap_pack is an independent canonical packlist containing the candidate and base, with ID packmap. The inspection purpose needs no GC mark or wait history. The verdict case uploads and accepts the candidate through signed wire requests. GC cases instead create a paired ref using the already accepted base_head and base_packmap, without tickets: a candidate ticket would itself root its external bases and prevent a genuine GC drop. Both original members remain unrooted before that ticketless attempt. Removal controls never supply an acceptance result.

The runner pins the business clock throughout these races so ticket age and the declared membership-lag permission are deterministic. The prepared GC attempt retains its original recheck proof across the competing acceptance; a later fresh recheck is outside this race. Hiding the unique base supplies no replacement and does not change the repository generation. Fixture bytes are bounded to 1 MiB each. gc declares repository membership GC; async-inspection declares verdicts or holds that can hide relied-on members. Neither is inferred from the unrelated lease or synchronous-inspector flags.

A case requiring an unsupported control is a declared skip only when its stable case name appears with a nonempty reason in the runner's expected-skip map. An undeclared skip fails; a declared case that passes also fails, so stale gaps must be removed. Missing wire features report not applicable instead of a declared skip. A known gap is never a conformance certificate. CI profiles that require a capability reject skips for it.

Reference-server behavior. test_host::ControlledHost exposes retained process-local memory, real core handlers, a manual clock, test admission, and local authority fencing. It supports the first two boundaries for ref_acceptance, occurrence one, and the uncertain-result action. It rebuilds the pipeline and cancels old request futures on restart. It declares restart.kill unsupported because retained memory does not prove process, disk, host, or zone recovery. It also declares pause.after_response unsupported because the client observes receipt. Indexed profiles support pause.before_published_clearance, publication observations, eligibility, and scoped real clearance driving. Opaque profiles declare these controls unsupported. Indexed profiles advertise combined-publication because eligible acceptance already includes publication. Repository GC and asynchronous inspection are not integrated in this host: gc and async-inspection remain undeclared, and their removal controls return HTTP 501 without mutation. The original TestHost remains available without these control routes.

The reference control host also provides an indexed, leased fork pipeline on its isolated stores. Its fork fixture seeds signed canonical contents, sealed inventories, indexes, and published source membership without creating the queried destination. Fork requests and ordinary destination probes use the same authenticated pipeline. The wrapper pauses at source proof, preparatory commits, and guarded acceptance, and observes all partitions under its mutation gate. Sixteen fork cases exercise this binding without declared fork skips; results are established by running the suite, not by the capability declaration.

The HTTP binding above is normative for black-box Tier-2 runs. Every case uses these routes; an optional in-process trait cannot be required by a case.

The profile declares optional features using features: transport-writes (enabled by default; use -transport-writes for a read-only target), authority-setter (both wire getter and setter), authority-fencing, and fork. Absent feature names mean the feature is not offered. relay_lag_bound_ms declares the membership visibility bound, defaulting to zero. Immediate membership visibility is conforming; a zero-bound target never returns MembershipNotVisible. Controls and their gaps are negotiated separately. A missing feature reports not applicable with its name, distinct from a declared skip for a known gap in an offered feature. A declared skip for a non-applicable case fails rather than hiding a stale declaration.

Protocol coverage and remaining work

Classification applies to the rule, not to reference mechanisms used to test it. The table tracks SPEC-SERVER and SPEC-TRANSPORT-CONNECT requirements. Case means executable portable coverage. Declared skip names an executable case unavailable in the reference profile. Planned means a portable independent assertion still needs implementation; a reference layout or orchestration audit is supporting evidence, not a portable conformance case. A planned entry is not a skip and certifies no requirement. Conditional rules apply when a server offers the feature.

In-process test host

Contract tests that need real loopback HTTP can use the debug-only test-host feature. It runs the production core Connect handlers over fresh MemoryKv and MemoryBlobStore instances and exposes the same profile shape as the wire suite. This is an unstable test-support API. Optimized builds reject the feature unless the standalone test utility opts into reference-runner.

Pin the feature-enabled dependency to the same mkit revision as the other contract-test dependencies:

[dev-dependencies]
mkit-server-conformance = { git = "https://github.com/officialunofficial/mkit", rev = "<same-revision-as-mkit-dependencies>", features = ["test-host"] }

use mkit_server_conformance::test_host::TestHost;
use mkit_server_conformance::wire::{Profile, WireAuth, WireTarget, run};

let profile = Profile::new(WireAuth::AuthV2 {
    audience: String::new(), // replaced with the allocated canonical origin
    repository: "default".into(),
    seed: [0x5e; 32],
});
let host = TestHost::start(profile).await?;
let target = WireTarget {
    base_url: host.base_url().parse()?,
    profile: host.profile().clone(),
};
let report = run(&target, None).await;
// Run the consumer's shared contract macro/report assertions here.
host.shutdown().await;

TestHost::start_with_hooks accepts a consumer's core HookSet, including scripted admission and outcome hooks. start_with_hooks_factory supplies the allocated server origin and manual clock so callers can connect RemoteAdmission to stubs::hook::FakeHook through LoopbackHookChannel. clock() returns a deterministic ManualClock; advance it explicitly and call drain_core_timers() with a partition and budget when a case needs timer delivery. drain_timers() accepts a custom core TimerRegistry for tests that need to control the handler set. Drains have a finite executor-poll budget. A handler or outcome sink that stays pending returns an unavailable error; its uncommitted timer remains queued. Each host owns its stores and listener. Dropping it aborts the listener task.

Portable file and takedown contracts

The shared registry includes health.deadline_headers, which exercises absent, finite, paired and malformed headers over Connect and gRPC; the existing runtime probe also covers zero timeouts, configured defaults and all supported wasm dispatch entry points. Native stalled-body deadline tests remain in mkit-server.

files.empty_readback, files.deep_tree, files.long_path, files.byte_distinct_names and files.path_limits publish canonical packs and paired refs through the ordinary transport. Names stay in memory as bytes; checkout filesystem normalization and case sensitivity cannot change them. A 100-directory tree round-trips over the ref route. A 2048-byte core path round-trips through disclosure lookup and object-ID HTTP reads; its ref URL returns 400 under the specified 1024-byte HTTP limit. Separate boundaries cover 128/129 authenticated core steps, 255/256-byte components and 1024/1025-byte HTTP paths. embedding.multipart_file_readback retains the existing binary payload and multipart upload, adding GET/HEAD, metadata, conditionals and exact ranges. Committed advances can precede published indexes. Fixture setup uses bounded HEAD readiness for each positive HTTP selector, then runs immediate exact byte/metadata/range assertions. Negative bounds and all takedown denial checks remain immediate. Private token setup also waits for published membership before minting the token and checking its HTTP selector; an owner live ref is insufficient. The Worker observer rejects zero-length R2 ranges; the core HTTP spy also checks backend range geometry and absence of an empty payload fetch.

takedown.contract requires takedown, admin, signed-reads, indexed HTTP objects and ticketed multi-repository transport. Use a fresh private-by-default fixture with the synthetic operator key used by the runtime probe. It publishes the same content into public and private repositories, proves public/ref reads, private URL-token reads and signed owner downloads work, accepts signed admin Takedown, and asserts their denial plus restricted admin status. Failure to provide the enabled capability fails the test. It never silently skips setup failures. The token stays in an exclusive, mode-0600 scratch file and never enters TAP.

Restart the server over the same stores, origin and configuration, then run takedown.persisted_denial with the original run ID and signer seed. This checks the original token, owner download, public/ref denial and admin identity/selectors, then deletes the scratch capture. It fails when the producer capture is missing. The native test host mounts the two signed admin routes for this profile and restart_with_default_hooks() rebuilds its listener, pipeline and admin engine over the same metadata, serving and preservation stores. This fixture does not assert preservation completion; the full admin rehearsal covers acquisition, legal hold, retention and audit separately.

python3 scripts/connect-deadline-runtime.py --portable

This pinned local workerd run prints all nine selected wire cases (eight new, one extended), rejects missing/skipped cases, restarts its own process over persisted state, and checks the R2 observer ran. Hosted Workers CI invokes it. It proves local runtime behavior, not cloud placement or production timing.

The coverage audit found existing one-winner and atomic-advance races, replay.expired_retry_rejected, nonce/different-operation rejection, admission.concurrent_duplicate_during_admit, repository isolation, private owner/grant/non-owner reads and URL-token visibility revocation. These seeds add no duplicate race/replay/privacy-only variants, URL staging or product policy.