Introduction
I'm deliberately leaving "runtime", "protocol", "HM26", and other debate-inducing terms out of the title, but let’s settle the terminology first.
Any doubts about naming things go away with one simple test question: must two independent peers agree on it to work together? If so, it's the protocol, otherwise it's the runtime.
Protocol: the things for which every peer on the network must compute the same answer. Even if it looks like indexing: when every peer must derive the same results from the same blobs — the rules for these derivations are part of the protocol. Each peer still stores those results in its own index.
Runtime: everything else. Search, embeddings, frontend-backend APIs, local data formats, GUI, anything application-specific, etc.
The Protocol has three layers:
Formats (physical): how data is stored and exchanged as bytes: Change, Ref, Comment, Capability, UnixFS, etc. This layer can hold many formats and gain new ones over time.
Resource Model (logical): one uniform reading of all those formats: resources, principals, access boundaries, links and permissions (!), plus the rules that derive them from each format. Every peer must compute the same results here.
Sync: which blobs move between which peers and how.
This is a living document, until it stabilizes with every concern addressed. It also supersedes Private Document Hierarchy and Authorization.
Problem
Links leak access. Blob visibility spreads along blob links, so knowing a CID is enough to get access to a private document.
Private documents are broken. Any writer to a space, even with a scoped access, gets access to all private documents.
Document identity is fragile. Pretty paths as the only document identity becomes a problem when documents are moved, renamed, and organized into hierarchies. Especially when you start mixing private and public documents.
The Vision
Spaces are the roots of authority for their content.
Principals (users, devices, agents, etc.) can participate in the space via delegated access, following the Keyline semantics.
Resources have flat opaque IDs as their canonical identity. It survives any moves and reorganizations.
Groups define access boundaries within a Space. Kind of like a subspace.
Blob accounting ensures that every blob belongs to a resource, and every resource belongs to a space or is public. Knowing a CID must not grant any access.
Resource hierarchy is defined by parents naming their children. Names, hierarchy, and URLs are separate concerns.
Sync is generic and works with any formats, thanks to the common logical resource model.
More or less this is the Web of names, hashes and UUIDs of Joe Armstrong.
Phase 1: Access Control, Sync, and Legacy Formats
Before trying to come up with new physical formats, I think we should first adopt the legacy formats to the new logical model, double down on Keyline for enforcing permissions, fix private documents, and change sync to be more uniform and efficient with the new access and resource model.
Keyline
I think we should treat Keyline as a blackbox — you feed it with nodes, edges, and revocations for the authority graph, and then you ask it questions whether some principal can have access to some resource.
I've already began implementing this, and it shouldn't take too long, because we have a reference implementation from Ink & Switch.
Resource Identity
Resources have flat opaque canonical identity.
The canonical URL format is: hm://<space-authority>/:r/<optional-slug-><space-unique-resource-id>.
The :r is used to disambiguate the URL from user-defined pretty paths. We use this trick a lot now.
The optional slug allows to make the opaque URLs somewhat human-friendly. It's up to the client application to use it or not. The normalized resource URL only takes the final slug segment as a resource ID.
Resource IDs only need to be unique within a space, because the fully-qualified identity already includes the space ID.
Named children can be addressed and traversed by adding more path components to the URL. This is how pretty-paths could work now.
For legacy formats, the resource ID is derived from Ref blobs with Hash(ref.space, ref.path).
We should prefer using canonical opaque URLs everywhere now, unless the user explicitly wants a pretty URL.
Resource Paths and Hierarchies
Valid legacy Ref blobs still contribute the pretty-paths they define. We use them as aliases and redirect to opaque resource IDs.
In addition to that, resource hierarchy is built from links in the resource. These links are just names, no access is granted through them.
We currently build the File Browser from pretty paths from Refs. And we do some tricks to materialize the children as embeds on parents, unless parent is a collection.
Now we'll make it explicit — the file browser is built from embed card links on a resource, and pretty paths from Refs are just extra information for the URL resolution — they don't affect the file explorer hierarchy. See New File Browser for more details.
TBD
Agent chats as hypermedia resources.
Users send signed messages to the agent server.
Agent server publishes its state — the queued messages, the processed messages, any other results and artifacts as a prolly tree data set.
Comment moderation.
Someone (the hosting server or the space writers) publish a set of vetted comments as a prolly tree.
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime