The Need for Explicit Branches and Other Issues

This is my take on our current (as of 2024-05-07) issues, and some of the fundamental flaws that I think are impeding us from using our own system, and other users from having a clear understanding of it.

While I was writing this we've had some in-person conversation during the past few days where we already talked about most of these issues. I hope this document could serve as summary of those problems, although it turned out much more messy than I wanted.

TL;DR

Lack of explicit branches affects the system on many levels, in addition to being very confusing for most people.

Collaboration sucks.

Sometimes documents have an implicit "parent context" which is not conveyed very well.

Concurrent publishing to the same pretty-path in groups is lossy.

Details

Implicit Variants Are Messy

I argue that more often than not, whenever you edit a document you don't expect it to fork, unless you are choosing to do it explicitly. Often times you don't want to create a new "timeline" for the document, you just want to add some changes.

Currently, if I edit a document which I'm not the owner of, my edit will create a separate version/variant/branch, and most of the time it's not what I want.

Implicit "My Documents" Collection

Currently we have an implicit collection that shows all of my documents that I've ever written, and people who follow me are getting all of those documents.

If I write a document that I only intend to be published in some group, this document will still appear in my collection, and I have no way to express my real intent, which is to make this document be part of the group.

Profile Documents might help to mitigate this problem, and allow me to curate my public collection of documents, but there's a slight limitation in that there's only one such profile document. Another limitation of Profile Documents is that they don't let you define pretty-paths for your documents.

Even though organizing information with links is very "hypermedia-native", it might be hard to automate, while doing those links by hand is not something that people want to do either.

The Problem With Root Documents

Following the previous point, we started using this idea of a root/collection document for organizing information in Groups as well. Both Groups and Accounts now have such documents, which serve as an entrypoint for the general public to discover and navigate the published information.

The problem with those documents is that they can "escape" from their context, and lose their meaning and purpose. One way or another someone might receive such a root document without discovering the relationship between its de-facto parent (Account or Group), and there's no way for the user to know what they are missing.

Imagine Alice linking the root document (R) from some group (G) inside of her own totally separate document (D). Because you are subscribed to Alice you receive D, you follow the links and discover R, but you don't discover the inherent relationship between R and G, which is a problem, because looking at R without knowing about G is not very useful. You might find out about G, if Alice linked the document including the group's variant, but it's very easy to screw it up, and currently our "Copy Link" button doesn't always behave the way we want.

Collaboration Sucks

We claim that our system is good for collaborating, but it currently sucks big time at that. And I think that's one of the main reason we don't use it ourselves.

In order to collaborate on a Document you'd probably need a Group, but because Group Publishing is so brittle now (more in the next section), there's no way to properly merge the changes from the different collaborators, and there's no good way to suggest changes to other authors.

There's no way to express the intent for a document to be collaborative, and for its group version to be the main version, which means that too often you'd be openning the document in its owner's variant instead of the group variant. This could happen if you open it with the launcher, or from the document list page.

Our implicit variants approach also doesn't help here much. At least for the documents we write for our own group we'd want to treat the group's version as the main version. Implicit variants created for each editor of this document causes a lot of confusion, and goes quite a bit against the whole point of a group document, which is to collaborate and to share a common perspective on a topic.

Group Publishing Is Brittle

Publishing a Document to a Group currently has many different steps:

Create a Change for the document you want to publish. Normally. Sometimes you publish already existing document.

Create a group Change to add the document with a pretty-path to the group.

Create a Change in the group's root document to include the published document in the front page "collection". Sometimes you update the document that's already "linked" in the root front page, so this step could be skipped if the document was referenced with the "latest" specifier.

Create another group Change to include the new root document version into the root pretty path of the Group. This is necessary because currently pretty-paths point to exact versions of documents.

The steps 2 and 4 potentially could be combined into one, but there's still a more dangerous problem: documents are assigned to pretty paths in a last-writer-wins manner. So if several people concurrently publish their group changes touching the same pretty-path, they will collide and only one of them will be respected.

This is especially bad for the root front document, because every new publication would touch the root document to create an embed, which will cause a lot of contention in case of concurrent changes. This is a very important and fundamental concern with our data model for groups.

Comparing Versions Is Almost Impossible

We don't have a diff view, but it's quite crucial for any workflow related to managing branches and changes, not to mention many other places where it would be useful.

Latest Links Need a Point of Reference

We all love the idea of a "latest" link: a link that always points to the most recent version of the document.

Without having a way to define branches explicitly, those "latest" links present a problem in that there's no way for someone to create a change in isolation, without affecting all those "latest" links.

Pretty Paths

Only groups have pretty paths now. But sometimes you don't need a pretty path for a document that you want to use inside of a group.

You might want to "anchor" a third-party document to a particular version inside of the group, but you don't need a pretty path for it, because it's not published anywhere, it's just used as a reference in some other group documents, using the "latest" specifier.

You then want to update the version of this referenced document in a single place once, without having to update all those other group documents that link to it.

Normal documents that are not published to groups don't have pretty paths though, but it might be useful.

Potential Solutions

This section needs more work and is very messy.

Explicit branches.

We need users to be able to explicitly create branches when they want to. Be it to collaborate with someone, or to experiment alone on some ideas before making them "officially" published.

Unless I'm explicitly choosing to create a separate timeline I'd want for my changes to become part of the main branch even if the original owner never "officially" merges it. To me, and maybe to anyone who trusts me enough, my change should become part of the main branch.

It would help if we can clearly show in the UI what parts are coming from my change, and indicate that they are not included in the official main branch, but they are shown to me because my intent wasn't to fork the document timeline, but to fix something in the main timeline.

I'd still want to receive new changes from the owner, and for them to be combined with my own change. Almost like an automatic rebase, but without actually rebasing my change, because there's no need for that, they all can be merged using CRDT rules.

My change shouldn't be shown to other people (including the original author) who happen to have received it, because the document owner didn't authorize me as an editor of the document. My change could appear as a suggested change for those people though, because I didn't explicitly create a separate branch for this change, I just wanted to include it in the main timeline. Users are willing to ignore my change of course, or accept it, even if the original owner never accepts it.

Whenever I want to create a separate timeline, I should be able to, by explicitly creating changes as part of a separate branch, isolated from the main branch.

Branches could become a nice primitive for collaboration.

Create a branch, add collaborators, make some changes in isolation so that "latest" links are not affected by those changes, and so on. When you're done, merge those changes into the main branch. Because collaborators were defined on a separate branch, your main branch is not affected: those collaborators can't keep editing your document, you're done with it, maybe until the next time.

Ink & Switch published some nice prototypes for one possible branching UX and workflow which I find quite neat.

They say branch is a great concept and a good word for describing it. They conversations with writers seem to reveal that people understand the concept very quickly and it makes sense to them. I think we shouldn't be affraid of the word nd concept, and maybe try to embrace it in our UI.

Maybe some documents need an explicit parent?

Especially root documents that serve as a collection.

Maybe we need to resurrect our old idea of a canonical location?

Collections as a snapshot?

The problem with our current approach for groups is that it has a lot of overhead, because groups grow to have lots of changes very fast, and users are required to sync all of those changes before they can do anything with the group.

Maybe groups should be snapshots rather than a sequence of patches? Most of the time we don't care about the group history either.

Being able to create multiple collections whenever is necessary also could be a plus.

Groups as document owners?

I want to publish a document to a group without it polluting my main public collection of documents.

I want to create a document in which I clearly indicate that it's meant for a certain group, and let those group members edit this document right away. Group docs currently always have at least 2 variants/branches (mine and group's), but most of the time people want only one.