Hypercore

Hypercore (formerly Dat) is a protocol for distributing and synchronizing append-only logs. It combines content addressing with mutable, real-time data streams, and is the foundation of the Holepunch ecosystem.

Append-Only Logs

A Hypercore is a single-writer append-only log:


    Each entry is signed by the author's Ed25519 identity key.

    A Merkle tree over entries enables efficient verification of any subset—you can prove entry #1000 is authentic without downloading entries #1-999 (sparse replication).

    The log is identified by the public key; readers verify integrity without trusting the source.

    Entries are immutable once appended. The log head is mutable (the latest signed entry pointer).

This structure makes Hypercore inherently offline-first: the author appends locally, and peers sync when connected.

Higher-Level Data Structures


    Hyperdrive: a POSIX-like filesystem built on Hypercores. Each file change appends to the log; directory listings are derived from the log.

    Hyperbee: a B-tree key-value store over a Hypercore, supporting sorted queries, range scans, and efficient diffs between versions.

    Autobase: multi-writer collaboration by merging multiple single-writer Hypercores with causal ordering, approaching CRDT-like semantics. This bridges the single-writer limitation.

Hyperswarm

Hyperswarm is the networking layer (analogous to libp2p but more focused):


    A Kademlia DHT specialized for topic-based peer discovery. Peers announce and lookup by topic hash.

    Built-in UDP and TCP hole punching for NAT traversal, with relay fallback.

    Noise protocol encryption for authenticated connections, binding each connection to a key pair.

Holepunch and Keet

Holepunch is the company/platform building on Hypercore:


    Keet: a P2P video/chat/file-sharing application built entirely on Hypercore and Hyperswarm, with no servers. Demonstrates the edge-first pattern in practice: all data lives on user devices.

    Pear Runtime: a platform for building P2P applications on Hypercore primitives.

Dat Legacy

Hypercore descends from the Dat protocol, originally designed for scientific data sharing and archiving. The lineage emphasizes:


    Data integrity and versioning via content addressing.

    Efficient sync of large datasets with sparse replication (download only what you need).

    Unix-philosophy simplicity compared to IPFS's broader scope.

How Hypercore Connects to Everything Else

Hypercore's append-only logs are a foundation for State Sync (log shipping, anti-entropy, Autobase multi-writer). It shares the log-centric philosophy of SSB. Hyperswarm competes with libp2p as a P2P networking stack. Security relies on single-writer signing and Merkle verification. Privacy benefits from the offline-first model (less network exposure). Storage supports sparse replication and efficient disk layout. Compliance faces the tension between append-only logs and data erasure (though Hypercore supports truncation).

Do you like what you are reading? Subscribe to receive updates.

Unsubscribe anytime

Connections0

Connections show where other documents or comments link to this document.

Related documents

Loading connections…