These procedures choose the tentative authority model, peer contract and transactional index. They are reviewable algorithms, not implemented endpoints. The high-level syncing guide introduces the design.
One ingest procedure for every source
Local publication, uploads and peer downloads converge on the same procedure:
Enforce interest/custody policy and bounded admission before requesting bytes. Sender identity, statement signer and resource reader are distinct.
Verify the full CID against original bytes, then signatures and structural rules where the format is known. Parse into provisional facts; do not advertise unknown-authority or invalid data.
Resolve trusted root and permanent admin evidence first, then revocable grants/revocations, then node statements, state dependencies and required content. Request missing evidence by named dependency, not global blob enumeration.
For every new content association, prove its origin: it is already authorized in that resource, already public, actually supplied as full bytes by the publisher, or accompanied by a verifiable source-read context and an explicit structural publication into the destination. Merely naming a private CID held elsewhere on the receiver does not disclose it into the new resource. Deduplication of bytes must not skip this association check. Ordinary references never propagate read access.
In a short writer transaction, store verified bytes and facts, resolve pending revocations, recompute affected authority, select current statements, update active associations/metadata, invalidate affected views and append index events. Persist incoming receipts/outgoing work in that transaction. Missing items enter the bounded stash with exact dependency rows; no partial accepted facts escape a rolled-back savepoint.
Commit before notifying watchers or acknowledging indexing. Retry newly satisfiable stash entries through the same pipeline. Full state replay may run outside the writer lock; install its result with a compare-and-swap against the selected node CID and authority revision. If either changed, discard/recompute rather than committing stale state.
A verified raw file is not automatically authorized content. Conversely, an unknown apply implementation does not invalidate its CID or silently trust the supplied utility snapshot. Report unsupported materialization and retain validated evidence.
Cold read
Bob opens R on an empty device using a signed invitation bundle with the expected root and intended hosting principal. The bundle is evidence, not bearer authority: another key cannot use Bob's grant.
sequenceDiagram
participant UI as Bob's app
participant D as Local daemon
participant P as Hosting peer
UI->>D: Open R as Bob
D->>D: Validate root/invitation/source custody evidence
D->>P: Hello + Authenticate on libp2p connection
P-->>D: Principal-bound authentication token
D->>P: GetControl(R, expected root, seed evidence)
P-->>D: Allowed grant/revocation evidence + observation token
D->>D: Verify and commit authority
D->>P: OpenScope(R, control/state/metadata)
P-->>D: Frozen data view + watch cursor + selected node CID
D->>P: ReconcileBlobs(view, local allowed set)
P-->>D: Allowed difference ranges
D->>P: FetchBlobs(view, missing CIDs)
D->>D: Verify, index and materialize
D-->>UI: State/version/materialization and freshness status
D->>P: WatchScope(after snapshot cursor)Before sending a private local fingerprint, Bob's daemon verifies that P's proven principal may read R. Server and client independently filter sets. On receiving the selected node, fetch the immutable apply implementation and all change dependencies before claiming computed state. Fetch visual content next and bulk attachments last. A utility snapshot may accelerate presentation only after replay verification or clearly labeled provisional display; it never replaces history truth.
The version token is the selected signed node statement's CID. It commits its kind, heads/apply or snapshot declaration. A computed state CID authenticates local canonical output bytes; it is not a new signed authority statement. Return both separately. A valid newest statement with missing dependencies remains the newest statement, with missing state; an old cached rendition may be shown only with its old version and stale/incomplete label.
Warm read and offline display
The daemon reads under the active principal, checks current local authority revision and retrieves the selected version and computed state. A matching local cache can render immediately. Then refresh control evidence and resume the watch or reconcile if the connection is absent, cursor expired or policy changed.
The response distinguishes locally verified state, incomplete/unsupported materialization, and authority knowledge stale relative to the last checked source. Offline display of previously retained bytes is a client policy, not a fresh peer read grant. No network data is sent using a stale authority cache. Do not let the active user's read query accidentally inherit another logged-in identity's rights.
Publishing locally, offline and to a host
Carol edits an existing resource R. Under the tentative policy, writer implies read; only its root may create a new resource in the first implementation.
Obtain current known authority and state. For a new resource, create a root-signed initial node and initial ResourcePolicy, then issue any required service reader grant. Establish trust using the expected root, not the newly received policy alone.
Construct the change DAG and ordered operations, or an authoritative snapshot. A change-bearing node names the exact immutable apply CID. Its heads cover the intended DAG. It has no prev; snapshot/tombstone/redirect may have their optional breadcrumbs. The unresolved republish variant is not silently given payload fields.
Sign immutable blobs and a new node statement referring to the original node CID. Validate against current local authority and commit local bytes, facts, selected state and outbox together. A utility snapshot mismatch is discarded. Local success does not imply remote publication.
Offline, show the locally committed version as pending delivery and authority freshness as stale. Persist the outbox. On restart, reset abandoned running jobs to queued and reuse stable request IDs/body digests. Do not silently mint an admin grant to make an upload work.
Authenticate the intended hosting principal and refresh control. Verify its reader/custody authority before sending private manifest IDs or evidence. The host also checks its own rights and configured interest.
The publisher daemon mints a short-lived export token for that exact recipient and CID set. Send OfferBlobs; the host records wanted work, pulls bytes from the publisher and commits STORED, WAITING, INDEXED or REJECTED receipts. A relay follows the same procedure without becoming the author.
Resume with GetOfferStatus after a lost acknowledgement. Clear outbox work only when its durable indexed receipt has been recorded locally. An indexed receipt may still report unsupported materialization. A refusal or stale writer grant keeps a visible failed/pending publication state; it does not count as success.
sequenceDiagram
participant C as Carol's daemon
participant H as Hosting daemon
C->>C: Sign + index + commit outbox
C->>H: Authenticate and refresh control
C->>H: OfferBlobs(stable request ID, manifest, export token)
H->>H: Commit wanted work/idempotency digest
H-->>C: WANTED receipts
H->>C: FetchBlobs(export token)
C-->>H: CID-framed bytes with live recipient checks
H->>H: Hash/signature checks + index transaction
H-->>C: Durable post-commit receipt
Note over C,H: Lost receipt: repeat status/request, do not create another publicationOffer retries with the same body are idempotent. A request ID reused for different content is rejected. Already-held bytes still need association-origin validation. A host crash after commit but before reply returns the stored receipt on retry; a crash before commit yields no durable indexed acknowledgement. Progress counters alone are insufficient.
First synchronization of a node
Start with local keys and explicit subscription/invitation inputs, not an indiscriminate crawl. Inventory each identity's requested scopes independently. Contacts and follows supply interest only. Resolve owner sites/devices and relevant configured peers, with public discovery separate from private invitation evidence.
For each identity and resource: validate bootstrap; authenticate to appropriate sources; control-refresh; open and reconcile the permitted scope; fetch authority/structure first, then rendition content. Deduplicate verified bytes across identities by CID, but retain the per-resource associations and per-principal permission context. Sharing a byte store must not share a read decision.
For recursive interest, evaluate every child separately. A grant on R does not grant its descendants under this tentative exact-scope policy. Listings can show only permitted child metadata; hidden ancestor names must not be reconstructed into paths. A directly shared child can be opened by CID without granting its parent listing. Add concurrency and byte quotas so one large media tree cannot starve text/control synchronization. Persist follow/pin scopes; demand scopes can expire when no view needs them.
Initial synchronization completes only relative to the sources, facets and view boundary actually reconciled. It is not a claim that every network replica agrees or that all offline revocations have been learned.
When something changes
A local edit or incoming verified blob updates the index through the same transaction procedure. Append a durable index_events row only after all source facts and synchronous authority fences are ready; commit the row with those changes. Watch delivery is a filtered projection of this log, not an alternate authority database.
A client watches from the snapshot cursor obtained by OpenScope. Events arriving during download remain replayable. On each event, durably enqueue required control refresh/reconciliation work, then advance the stored cursor. Repeated events are harmless. A state event is a hint to reconcile; its supplied head CID is checked against signed statements. It does not authorize a blob by itself.
On reconnection, authenticate again and ask after the persisted cursor. If retained events cover the gap, replay them with current access checks. Otherwise return RESET_REQUIRED, refresh control and perform RBSR against a new view. A slow consumer or compacted log gets the same reset path. Periodic reconciliation remains a repair fallback even with watches; do not use a sleep to resolve index visibility races.
Administrator revocation
Alice has valid permanent admin authority over R and revokes Bob's exact reader grant. She signs a revocation with the required admin proof and publishes it through the control pipeline. A root uses a provisioned self-admin grant as that proof.
sequenceDiagram
participant A as Alice's daemon
participant H as Hosting daemon
participant B as Bob's daemon
A->>H: Offer signed revocation + permanent admin evidence
H->>H: Verify target/proof/signer/scope
H->>H: Commit revision bump, rights rebuild, view invalidation, event, receipt
H-->>A: INDEXED receipt after commit
H-->>B: Personalized ACCESS_CHANGED without private head metadata
B->>H: GetControl(own revoked grant)
H-->>B: Exact revocation + required admin/root evidence only
B->>B: Invalidate live scope/cache, preserve retained bytes according to local policyWithin the hosting transaction, bump R's authority revision and invalidate all R decisions, including denied/missing rows; adding a grant can create a route that no positive dependency recorded. Recompute permanent admin proofs without revocation lookups, validate exact reader/writer revocations, union remaining grants, reevaluate node candidates, update active associations and invalidate affected exact/recursive session policy stamps. Append the event and receipt before commit. Async tree maintenance can rebuild later, but an old tree may never bypass the new live access predicate.
A revocation received before its target is stashed; fetch the named target through the permitted control path. When it arrives, resolve both in one transaction before publishing rights. Two grants mean one revoke may leave access. A writer revoke also reevaluates current node selection as described in the authority model; retained dependencies adopted by surviving statements remain available.
If revocation arrives during a page/transfer, the next authorization fence returns stale/unavailable rather than exposing another fingerprint, metadata item or block. An already-authorized in-flight frame is a past disclosure. Offline peers may remain stale; delivered bytes and old keys cannot be recalled. Bob can fetch his narrow control status even after losing content access, without seeing the whole roster.
Live republishing: a tentative application procedure
Recommend first implementing live republishing as an explicit copy subscription, without pretending the closed republish placeholder already supports it. A dedicated automation principal holds reader on source S and writer on destination R. It watches S, validates/materializes a source version, supplies the copied bytes and signs an ordinary destination snapshot or change statement. Source CID/version provenance is recorded as an ordinary reference in application state. Destination readers get the deliberately copied content, not authority over S.
A source change triggers another destination publication after source control validation. A watch gap reconciles before copying. Loss of source read rights stops future copying; prior destination copies remain visible under destination policy. Source writer revocation/rollback triggers reevaluation and may produce a new copied rendition. Destination writer revocation stops publishing. Audience changes require explicit disclosure intent, not automatic propagation from a bare CID. Persist the recipe/last source version and outbox locally; no signing key or secret belongs in public provenance.
This is operationally implementable with the proposed methods, but does not settle the eventual native republish payload or a transparent live reference's semantics. A native kind needs a separately reviewed schema and audience/provenance rules.
Failure cases to validate
Case | Required behavior |
|---|---|
Bad hash/signature or malformed structure | Reject; no structural rows, scope membership or authority cache entry |
Valid foreign node with no trusted root | Unknown/refused; no new root trust inferred from sender |
Valid writer node before its grant | Bounded waiting state; retry when evidence arrives |
Grant arrives after already-known exact revocation | No transient access published between them |
Referenced secret CID already in the receiver's store | No new resource association without origin/publication evidence |
Missing block or interrupted fetch | Retain verified completed blocks; retry unresolved CIDs through an authorized source |
Revocation during frozen-view paging | Invalidate/recheck; never treat the view as an access lease |
New grant to formerly hidden child | Recompute recursive visible membership and wake/reset affected views |
Unsupported apply implementation | Report unsupported; do not promote utility snapshot to truth |
SQLite/index maintenance crash | Replay durable outbox/events; discard/rebuild derived caches; never trust stale cache revisions |
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime