Part of Stem. This page defines the Fetch method of the sync RPC specification. It replaces Bitswap as the only way blob bytes move between peers.

The formal schema is attached as the schemaDefinition of this page: a struct {key, input, output} with key pinned to Fetch.

Serves blobs by CID to a peer that may read them, ordered so that authority arrives before the data it authorizes. Replaces Bitswap: every served blob is checked against the caller's access and recorded in the disclosure ledger.

Input

field	type	required	meaning
`scope`	data/scope	no	The scope the caller is syncing; used to order the response authority-first.
`offer`	string	no	The transfer id returned by an Offer the caller is answering; blobs of that offer are served back to the offering peer under the `offer` basis.
`cids`	list of cid	yes	The blobs wanted.

Output

field	type	required	meaning
`blobs`	list of rpc/type/blob-envelope	yes	The blobs the caller may read, in authority-first order: Groups, Grants and Revocations, then Nodes, then Snapshots and Changes with dependencies before dependents, then files.
`missing`	list of cid	yes	CIDs the server does not hold. A CID the caller may not read is reported here too, so denial and absence are indistinguishable.

Rules

Access. For each requested CID the server finds the nodes the blob belongs to (blob_access) and serves it only if the connection may read at least one of them, or the blob is readable by everyone, or the blob is being fetched back under an open Offer from this caller. A CID that fails this test is listed in missing exactly like a CID the server does not hold. The response gives no other signal.

Order. Served blobs are sorted into bands, and within a band dependencies come before dependents:

Groups, then Grants, then Revocations (the authority that validates everything else; a Grant before the Grant named in its proof is avoided by ordering on the proof link);

Node blobs, parents before children, prev before successors;

Snapshots and Changes, in topological order of dep and prev links, genesis first;

file and media blobs.

If the response would exceed the message limit the server stops at a band boundary or a dependency boundary and the caller asks again for the rest.

Integrity. The caller recomputes each blob's hash and discards a mismatch, counting it in the transfer.

Ledger. Every served blob writes one disclosure with the basis that allowed it: public, grant (with the account and the grant), site, or offer.

Limits. At most one thousand CIDs per call; blobs are at most 2 MiB each.

Today (HM24)

HM24 fetches with Bitswap: one session per discovery task, render-priority order (DAG-CBOR first, then icons, inline images, bulk files), a per-(peer, CID) callback that refuses unauthorised serves, and no record of what was served. Bitswap also accepts whatever a peer sends. Stem's Fetch is an ordinary RPC, serves in authority-first order so the receiver can validate as it goes, fails closed on everything, and writes the disclosure ledger.

Example

{"key": "Fetch", "input": {"scope": {"space": {"/": {"bytes": "7QEA…"}}, "node": "bafyrein6a6wfhym6l3vfz5zfkkibj5j6wjibagi3mnbqnspuq2idw52ijb", "depth": "subtree"}, "cids": [{"/": "bafy…13"}, {"/": "bafy…12"}, {"/": "bafy…99"}]}}

{"blobs": [
  {"cid": {"/": "bafy…12"}, "data": {"/": {"bytes": "omR0eXBlZUdyYW50…"}}},
  {"cid": {"/": "bafy…13"}, "data": {"/": {"bytes": "omR0eXBlZE5vZGU…"}}}
], "missing": [{"/": "bafy…99"}]}

Input

Output

Rules

Today (HM24)

Example

See also