Worlds Provenance Sidecar — Specification v1

Status: v1.0.0-phase-b — stable for Phase B core libs; subject to revision before Phase F public rollout Spec URL (canonical): worlds.pub/spec/v1 (planned; currently authored in repo) Authored: 2026-05-09 Reference implementation: src/lib/provenance/{merkle,sign,sidecar}.ts in the Worlds monorepo License: this specification is licensed under CC BY 4.0 — anyone may implement a verifier without permission. Reference TypeScript implementation MIT-licensed (see tools/verify-cli/).

This document specifies the Worlds Provenance Sidecar format (*.provenance.json) — a self-contained, cryptographically verifiable artifact shipped alongside an exported manuscript. A reader who possesses both the manuscript and its sidecar can verify, offline and without trusting Worlds's servers, which paragraphs of the manuscript were personally signed by the named human author and which were generated or assisted by an AI model.

The format is the operational substrate of Worlds's signet — the per-paragraph cryptographic stamp uniquely assigned to one human author (R7 prior-art research, resources/notes/duende/product/08-human-signet-prior-art.md).

1. Conventions

The keywords MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119 and RFC 8174.

All hash values, public keys, signatures, and fingerprints in the sidecar JSON are encoded as lowercase hexadecimal strings with no 0x prefix and no internal whitespace. Implementations MUST reject sidecars containing uppercase hex or 0x prefixes — strict equality is required for canonical comparison.

All timestamps MUST be ISO 8601 / RFC 3339 in UTC with millisecond precision (e.g., 2026-05-09T14:23:11.000Z).

JSON files described by this spec MUST be UTF-8 encoded, with no BOM, with Unix line endings (\n).

2. Threat model and goals

A verifier of a Worlds-signed manuscript is asking three questions:

1. Was each paragraph in this manuscript actually signed by the author whose key fingerprint is named? (Per-record signature check.)
2. Is each paragraph's record actually part of the manuscript's published Merkle log, or has the sidecar been forged or tampered with? (Inclusion proof check.)
3. Did Worlds's audit infrastructure attest to the Merkle root for this manuscript at this size, or is this an entirely fabricated log? (Root signature check.)

The spec answers all three offline. A verifier needs only:

• The manuscript text
• The sidecar JSON
• The Worlds audit public key (out-of-band, published at worlds.pub/audit/key)

The author public keys themselves travel inside the sidecar — the audit-key signature on the root is what binds the included author keys to a trusted authority.

Out of scope for v1:

• Revocation. If an author's key is compromised, future records can be re-signed with a fresh key, but historical signatures remain mathematically valid; an external revocation list (Phase D) is the planned remediation path.
• Time attestation beyond the author's claimed created_at_iso. RFC 3161 trusted timestamps land in v2.
• Privacy of the per-paragraph text content. The sidecar contains hashes of paragraph text, not the plaintext — but the manuscript itself does. The Phase F "redacted public journal" mode (v2) will allow signatures + hashes without the manuscript text alongside.

3. Cryptographic primitives

All cryptographic operations MUST be performed via constant-time implementations. Verifiers MUST compare hash values and signatures using constant-time equality routines to avoid timing oracles.

4. The provenance record

Each verifiable text unit (paragraph, in the default Worlds segmentation) is represented as a provenance record. The record is what gets signed by the author and hashed into a Merkle leaf.

4.1 Schema

{
  "manuscript_id": "string (opaque, ≤128 chars, RECOMMENDED UUIDv4)",
  "unit_id": "string (opaque, ≤128 chars, unique within manuscript)",
  "author_type": "HUMAN-ORIGINAL | HUMAN-EDITED | HUMAN-TRANSCRIBED | AI-GENERATED | AI-SUGGESTED-HUMAN-ACCEPTED",
  "model": "string | null  (e.g., 'claude-sonnet-4.5'; MUST be null for HUMAN-* types)",
  "text_hash_hex": "64-char lowercase hex (SHA-256 of normalized text)",
  "author_key_fingerprint_hex": "64-char lowercase hex (SHA-256 of author's raw Ed25519 public key)",
  "parent_unit_id": "string | null (for HUMAN-EDITED, the unit this was edited from)",
  "created_at_iso": "RFC 3339 UTC timestamp"
}

4.2 Field semantics

• manuscript_id — opaque manuscript identifier. Same value across all records in a single manuscript.
• unit_id — opaque text-unit identifier. Unique within a manuscript. Implementations SHOULD use a stable identifier that survives editor round-trips (e.g., a content-derived ID, not array index).
• author_type — exactly one of:
  • HUMAN-ORIGINAL — text written from scratch by the named human author.
  • HUMAN-EDITED — text the human modified after AI suggestion or import. parent_unit_id MUST point at the prior unit.
  • HUMAN-TRANSCRIBED — Whisper or equivalent ASR output of the author's own recording.
  • AI-GENERATED — text generated by an AI model with minimal or no human modification. model MUST be set.
  • AI-SUGGESTED-HUMAN-ACCEPTED — AI suggestion accepted verbatim by the human. model MUST be set. Counts as human-attested for word-share purposes per the Phase B reference implementation; subject to revision pending IP counsel Q2 (Copyright Office derivative-work doctrine, see resources/notes/duende/legal/ip-counsel-brief.md).
• model — the AI model identifier (e.g., claude-sonnet-4.5, gpt-4o-2024-11). Implementations SHOULD include a date-pinned model identifier to support future audits.
• text_hash_hex — SHA-256 hash of the text after normalization.
• author_key_fingerprint_hex — SHA-256 of the author's raw 32-byte Ed25519 public key. The full public key MUST appear in the sidecar's author_keys array.
• parent_unit_id — for HUMAN-EDITED records, the unit_id of the prior version. null for all other types.
• created_at_iso — author's claimed creation time.

4.2 Text normalization

Before computing text_hash_hex, text MUST be normalized as follows:

1. Apply Unicode normalization form NFC.
2. For each line (split by \n), strip trailing whitespace (matching the regex \s+$).
3. Re-join with \n. Do NOT add a trailing newline.
4. UTF-8 encode the result.
5. SHA-256 the UTF-8 bytes.

This normalization preserves semantic equality across editor round-trips (different editors disagree on trailing spaces) without erasing meaningful whitespace inside the line. Reference implementation: hashText() in src/lib/provenance/merkle.ts.

4.3 Canonical encoding

To produce a deterministic byte sequence for hashing or signing, a record MUST be encoded as follows:

1. Construct an object with exactly these keys, in lexicographic order:
   • author_key_fingerprint_hex
   • author_type
   • created_at_iso
   • manuscript_id
   • model
   • parent_unit_id
   • text_hash_hex
   • unit_id
2. JSON-serialize using minimal output: no whitespace, no trailing newline, ASCII-only \u escapes for non-printable characters as per JSON.stringify semantics.
3. UTF-8 encode the serialized string.

The byte sequence produced by step 3 is the canonical record bytes. These are:

• Hashed (via §5 leaf hashing) into a Merkle leaf.
• Signed (via §6 author signature) by the author's private Ed25519 key.

Reference implementation: canonicalEncode() in src/lib/provenance/merkle.ts.

Determinism is non-negotiable. A single byte's difference between encoder implementations breaks every signature and inclusion proof. Implementations MUST validate against the test vectors in §13 before claiming v1 conformance.

5. Merkle tree

The provenance log is a binary Merkle tree following RFC 6962 §2.1 (Certificate Transparency).

5.1 Hashing rules

• Leaf hash: SHA-256(0x00 || canonical_record_bytes) — domain separation prefix 0x00 prevents second-preimage attacks.
• Internal node hash: SHA-256(0x01 || left_child_hash || right_child_hash) — domain separation prefix 0x01.
• Empty tree root (zero records): SHA-256("").
• Single-record tree root: the single leaf hash itself.

5.2 Tree shape

For n > 1 leaves, let k be the largest power of 2 strictly less than n. The tree splits as:

• Left subtree: leaves [0, k) (recursively constructed)
• Right subtree: leaves [k, n) (recursively constructed)
• Root: SHA-256(0x01 || left_subtree_root || right_subtree_root)

This shape is identical to RFC 6962 §2.1.

5.3 Inclusion proof format

An inclusion proof for leaf at index i is an array of (sibling_hash, side) pairs, ordered from leaf level upward to root level. Each step's side indicates which side of the concatenation the sibling occupies.

A verifier reconstructs the root by:

current = leaf_hash
for step in proof:
    if step.side == "right":
        current = SHA-256(0x01 || current || step.sibling_hash)
    else:  // step.side == "left"
        current = SHA-256(0x01 || step.sibling_hash || current)
return constantTimeEquals(current, claimed_root)

Reference implementation: verifyInclusionProof() in src/lib/provenance/merkle.ts.

6. Author signature

The author's Ed25519 private key signs the canonical record bytes (§4.3) directly — no pre-hash, no envelope, no domain separation prefix. This is per RFC 8032 §5.1 PureEd25519, which already absorbs the message into its internal hash.

signature = Ed25519.sign(author_private_key, canonical_record_bytes)

Verifiers MUST:

1. Look up the author's public key in the sidecar's author_keys array by fingerprint_hex matching the record's author_key_fingerprint_hex.
2. Recompute the fingerprint as SHA-256(public_key_bytes) and verify it matches — the named fingerprint is not authoritative; it must derive from the named public key.
3. Verify Ed25519.verify(public_key, canonical_record_bytes, signature).

Reference implementation: verifyProvenanceRecord() in src/lib/provenance/sign.ts.

7. The sidecar JSON

7.1 Schema

{
  "spec_version": "1.0.0-phase-b",
  "manuscript_id": "string",
  "generated_at": "RFC 3339 UTC timestamp",
  "tree_size": 0,
  "merkle_root_hex": "64-char lowercase hex",
  "root_signature_hex": "128-char lowercase hex (64-byte Ed25519 signature)",
  "worlds_audit_key_fingerprint_hex": "64-char lowercase hex",
  "author_keys": [
    {
      "fingerprint_hex": "64-char lowercase hex",
      "public_key_hex": "64-char lowercase hex (32-byte raw Ed25519 public key)",
      "algorithm": "Ed25519",
      "label": "optional human-readable label, ≤64 chars"
    }
  ],
  "entries": [
    {
      "unit_id": "string",
      "record": { /* a ProvenanceRecord per §4 */ },
      "signature_hex": "128-char lowercase hex",
      "inclusion_proof": [
        { "sibling_hash_hex": "64-char lowercase hex", "side": "left | right" }
      ]
    }
  ],
  "summary": {
    "total_units": 0,
    "by_author_type": {
      "HUMAN-ORIGINAL": 0,
      "HUMAN-EDITED": 0,
      "HUMAN-TRANSCRIBED": 0,
      "AI-GENERATED": 0,
      "AI-SUGGESTED-HUMAN-ACCEPTED": 0
    },
    "human_word_share": 0.0
  }
}

7.2 Field semantics

• spec_version — exact string "1.0.0-phase-b" for this version of the spec. Verifiers MUST reject sidecars with unknown spec versions or surface a clear warning.
• tree_size — number of entries. MUST equal entries.length.
• merkle_root_hex — root hash of the Merkle tree built from entries[].record in array order.
• root_signature_hex — Ed25519 signature by the Worlds audit key over the bytes described in §8.
• worlds_audit_key_fingerprint_hex — fingerprint of the audit key. The verifier MUST independently obtain the audit public key (out-of-band, e.g., from worlds.pub/audit/key) and check its fingerprint matches.
• author_keys — every author public key referenced by any entry MUST appear here exactly once. Verifiers MUST reject sidecars with dangling fingerprints or duplicate entries.
• entries — sequential. Index i in the array MUST correspond to leaf index i in the Merkle tree.
• summary — informational; verifiers SHOULD recompute and compare for consistency, but disagreement is a warning, not an error (the Merkle root is authoritative).

8. Worlds audit-root signature

The Worlds audit key signs a deterministic byte string derived from the root, tree size, and manuscript ID:

audit_payload = UTF-8(
    "worlds-provenance-v1\n" ||
    manuscript_id || "\n" ||
    decimal(tree_size) || "\n" ||
    hex(merkle_root_bytes)
)
root_signature = Ed25519.sign(worlds_audit_private_key, audit_payload)

Verifiers reconstruct audit_payload byte-for-byte and verify the signature against the audit public key obtained out-of-band.

Why this binds tree size: without it, a malicious sidecar could selectively omit later entries and re-publish a smaller tree with a still-valid root signature. Including tree_size in the audit payload prevents truncation attacks.

Why out-of-band public key: if the audit key were named in the sidecar without external anchoring, a forged sidecar could substitute a forged audit key. The verifier's confidence ultimately rests on the channel by which they obtained Worlds's audit public key — verifiers SHOULD pin a specific audit key fingerprint and refuse to verify against unknown keys.

9. Verification recipe

Given (manuscript text, sidecar JSON, trusted Worlds audit public key), a conformant verifier MUST perform the following checks in this order. Failures MUST be surfaced; partial successes MUST NOT be reported as full validity.

Step 1 — Spec version gate. Reject if spec_version != "1.0.0-phase-b" (or surface as warning if forward-compatible behavior is implemented).

Step 2 — Audit key match. Compute SHA-256 of the trusted audit public key bytes; require it equals worlds_audit_key_fingerprint_hex.

Step 3 — Author key consistency. For each entry in author_keys: compute SHA-256 of public_key_hex bytes; require it equals fingerprint_hex.

Step 4 — Per-entry signature. For each entry:

• Locate the author public key by entry.record.author_key_fingerprint_hex. Reject if not found in author_keys.
• Compute canonical record bytes from entry.record per §4.3.
• Verify Ed25519.verify(public_key, canonical_bytes, hexToBytes(entry.signature_hex)).

Step 5 — Per-entry inclusion proof. For each entry at index i:

• Compute leaf_i = SHA-256(0x00 || canonical_record_bytes_i).
• Walk the inclusion_proof per §5.3 starting from leaf_i.
• Require the result equals hexToBytes(merkle_root_hex) (constant-time).

Step 6 — Recomputed root sanity. Build the full Merkle tree from entries[].record per §5.1–§5.2. Require root equals merkle_root_hex.

Step 7 — Audit-root signature. Construct audit_payload per §8. Verify Ed25519.verify(audit_public_key, audit_payload, hexToBytes(root_signature_hex)).

Step 8 — Manuscript text consistency (RECOMMENDED). For each entry whose unit_id is also locatable in the manuscript text: normalize the manuscript paragraph per §4.2, hash, and compare to entry.record.text_hash_hex. Disagreement is an error — it indicates the manuscript was modified after signing.

Step 9 — Surface summary. Report:

• Per-author-type counts.
• Human word share (recomputed from manuscript word counts; the sidecar's summary.human_word_share is an unverified hint).
• Any signed paragraphs that no longer appear in the manuscript (warning: text has been deleted post-signing).
• Any manuscript paragraphs without a corresponding signed entry (warning: text was added post-signing or by an unauthenticated party).

10. Author types and human-share computation

A "human word" for the purpose of summary.human_word_share is a word in a record of type:

• HUMAN-ORIGINAL
• HUMAN-EDITED
• HUMAN-TRANSCRIBED
• AI-SUGGESTED-HUMAN-ACCEPTED (provisional — see §4.2)

AI-GENERATED words are excluded.

The reference implementation computes human_word_share as human_words / total_words. Verifiers SHOULD recompute this from manuscript text rather than trusting the sidecar's claimed value.

Forthcoming spec change: the inclusion of AI-SUGGESTED-HUMAN-ACCEPTED in the human numerator is provisional and may change in v1.1 pending IP counsel guidance on Copyright Office derivative-work doctrine (Q2 of the Worlds IP counsel brief). Implementations SHOULD report the raw counts so a downstream consumer can recompute under either policy.

11. Versioning and compatibility

This spec uses semver-flavored version strings: <major>.<minor>.<patch>-<phase-tag>.

• Patch (1.0.X) — clarifications, test-vector additions. Existing sidecars remain valid.
• Minor (1.X.0) — new optional fields, new author types. Existing verifiers may surface unknown fields as warnings but MUST continue to verify cryptographic integrity correctly.
• Major (X.0.0) — breaking changes to the canonical encoding, hashing, or signature scheme. Verifiers MUST reject sidecars with unknown major versions.

The -phase-tag suffix (-phase-b, -phase-d, etc.) signals which Worlds phase a spec version is operational for. Reserved tags:

• -phase-b — JSON-only (current).
• -phase-d — adds COSE-binary representation embedded in C2PA text manifests for KDP / EU AI Act Article 50 compliance.
• -phase-f — adds the public author journal mode and the redacted-journal mode (signatures + hashes, no plaintext).

12. Security considerations

12.1 Key compromise

If an author's private key is compromised, all past signatures with that key remain mathematically valid. Worlds publishes a key rotation event at worlds.pub/audit/keys with the timestamp of the compromise; verifiers SHOULD treat any record with created_at_iso after that timestamp signed by the compromised key as untrusted.

If the Worlds audit key is compromised, the entire system's trust collapses for sidecars signed after the compromise. Worlds MUST publish a key rotation announcement immediately and MUST re-sign existing sidecar roots under the new audit key for a documented re-issuance period.

12.2 Truncation attacks

Including tree_size in the audit payload (§8) prevents an attacker from publishing a smaller-tree subset of the log with a different valid root signature. Verifiers MUST reject any sidecar where entries.length != tree_size.

12.3 Substitution attacks

A malicious sidecar could substitute a different worlds_audit_key_fingerprint_hex and a self-generated audit key. The verifier's protection is the out-of-band trusted audit key: if the sidecar's claimed audit fingerprint doesn't match the verifier's pinned fingerprint, the sidecar MUST be rejected (§9 step 2).

12.4 Timing oracles

Verifiers MUST use constant-time comparison for hash and signature equality. Reference: constantTimeEquals() in src/lib/provenance/merkle.ts.

12.5 Defamation exposure (deferred to IP counsel Q6)

A public verifier (worlds.pub/verify) that reports manuscripts as "tampered" or "AI-generated" carries some risk of defamation claims if the report is misused. The MIT-licensed reference implementation includes a "no warranty, informational only" disclaimer; downstream verifier authors SHOULD carry the same. This is being formally reviewed by Worlds's IP counsel; see resources/notes/duende/legal/ip-counsel-brief.md Q6.

13. Test vectors

The canonical fixture is at docs/spec/v1/test-vectors.json (planned canonical URL: worlds.pub/spec/v1/test-vectors.json). It pins:

• inputs — a deterministic author keypair, audit keypair, and a 5-record manuscript covering all five spec §4.2 author types with stable timestamps.
• expected — every intermediate value the spec produces from those inputs:
  • Per-record canonical bytes (5)
  • Per-record leaf hashes (5)
  • Merkle root
  • Canonical audit payload (per §8)
  • Per-record Ed25519 signatures (5)
  • Audit-root Ed25519 signature
  • Full sidecar JSON

13.1 Conformance procedure

Implementations claiming v1 conformance MUST pass the following procedure against the canonical fixture:

1. Read inputs.author_keypair.{private_key_hex, public_key_hex} and inputs.audit_keypair.{private_key_hex, public_key_hex} and rehydrate as 32-byte Ed25519 keypairs (no Web Crypto generation; use the pinned bytes directly).
2. Fingerprint check — compute SHA-256(public_key_bytes) for each keypair; confirm it equals the pinned fingerprint_hex.
3. Canonical bytes — for each record in inputs.records, compute the canonical encoding per §4.3; confirm it equals expected.canonical_record_bytes_hex[i].
4. Leaf hashes — for each record, compute SHA-256(0x00 || canonical_bytes); confirm it equals expected.leaf_hashes_hex[i].
5. Merkle root — build the tree per §5.1–§5.2; confirm root equals expected.merkle_root_hex.
6. Audit payload — compute the canonical audit payload per §8 using inputs.manuscript_id, inputs.records.length, and the computed root; confirm it equals expected.audit_payload_hex.
7. Record signatures — for each record, sign the canonical bytes with inputs.author_keypair.private_key; confirm the resulting 64-byte signature equals expected.record_signatures_hex[i]. (Ed25519 signatures are deterministic per RFC 8032 §5.1, so byte equality is the correct test.)
8. Audit signature — sign the audit payload with inputs.audit_keypair.private_key; confirm it equals expected.audit_signature_hex.
9. Sidecar wrapper — confirm expected.full_sidecar.{spec_version, merkle_root_hex, root_signature_hex, tree_size} match the computed values from steps 4, 7, and the records list.

A conforming implementation MUST pass all 24 byte-equality checks.

13.2 Reference checker

The Worlds reference CLI implements this procedure:

cd tools/verify-cli
npm run verify -- --vectors ../../docs/spec/v1/test-vectors.json
# Conformance: ✓ CONFORMANT (24/24 checks passed)

The checker source is at tools/verify-cli/src/vectors.ts. Implementers porting the spec to other languages SHOULD mirror its check structure so failures surface granularly (a canonical-encoding drift fails at step 3 rather than cascading through subsequent checks).

13.3 Re-pinning policy

The fixture is pinned once and considered immutable. The following events MAY trigger a re-pin (with a test_vectors_version bump):

• Spec changes that alter canonical encoding (canonicalEncode), text normalization (hashText), Merkle hashing rules, or the audit-payload format.
• Spec changes that add new author types (the existing fixture's records cover all current types; new types would warrant adding records).
• Spec major-version bumps (X.0.0 in §11) — fresh fixture for the new spec version.

Patch-level (1.0.X) and minor-level (1.X.0) spec changes that don't touch the bytes-on-the-wire MUST NOT re-pin the fixture. A drift between the fixture and a conforming implementation is a spec-implementation bug; the fixture is the source of truth.

14. Reference implementation

The reference implementation is pure-TypeScript using the Web Crypto API. It runs on Node 22+ (Node 25 confirmed working) and modern browsers without external dependencies.

Important implementation note. The Worlds Web Crypto sign path requires both the private key (d) AND the public key (x) on JWK import — Node 25 strictly enforces JWK Ed25519 conformance and rejects an empty x. Implementations using runtimes with looser JWK validation MAY skip this, but reference compliance MUST plumb both halves of the keypair through the sign primitive.

15. Acknowledgments

The Merkle tree construction is identical to RFC 6962 (Certificate Transparency, Laurie / Langley / Käsper, IETF). Ed25519 follows RFC 8032 (Josefsson / Liusvaara). The architectural decision to use an off-chain Merkle log instead of blockchain anchoring is documented in Worlds R6 research. The first-mover positioning rationale is in Worlds R7 research.

16. Change log

• 2026-05-09 — v1.0.0-phase-b initial draft. Authored as Phase F.4 deliverable.
• 2026-05-09 — Implementation hardening alongside the CLI work:
  • Library: buildSidecar() and verifySidecar() callbacks renamed signRoot → signAuditPayload and verifyRootSignature → verifyAuditSignature. The library now computes the canonical audit-payload bytes per §8 and hands them to the caller; callers cannot pick their own encoding. Closes a spec/implementation drift between the smoke-test JSON-of-{root, tree_size} format and the §8 canonical encoding (which binds manuscript_id and includes the version-tag prefix worlds-provenance-v1\n).
  • Library: signBytes() and signProvenanceRecord() now require publicKey alongside privateKey for Node 25+ JWK Ed25519 import compatibility (see §14 implementation note).
  • CLI: full eight-step verification pipeline wired and tamper-tested across four mutation classes (manuscript word change / signature zeroed / wrong audit key / forged Merkle root) — all four correctly rejected.

17. Sources

• RFC 2119 — keywords for use in RFCs to indicate requirement levels
• RFC 8174 — ambiguity of uppercase vs lowercase in RFC 2119 keywords
• RFC 3339 — date and time on the internet
• RFC 6962 — Certificate Transparency
• RFC 8032 — EdDSA signature schemes (Ed25519 in §5.1)
• FIPS 180-4 — Secure Hash Standard (SHA-256)
• Web Crypto API — W3C
• C2PA Specification — Coalition for Content Provenance and Authenticity (Phase D integration target)