January 2024 was a milestone for ACM-published papers as they will no longer be automatically available in print, i.e. non-digitally. Yet the long shadow of paginated print remains and in ways not necessarily helpful to consumption in emerging digital environments. A useful provocation is the XR (Extended Reality) 1 environment that offers the possibilities of decomposing/deconstructing complex texts, but only if the source documents have suitable data to allow this. Academic papers, such as that which you are reading now, are generally constrained by the fixed nature of typeset print—at least within PDF format—thus limiting the reader's ability to explore the document's structure, linked data, and references.

This potential exploration involves a dynamic remediation, a reconstruction of the source in the virtual space. Beyond rendering a mere facsimile of a paper document, meaningful re-use of the source material needs to give accessible clues as to its internal structure and linkages—both internal and external. Today, these enhancements are generally lacking, or effected outside the document, e.g. on a host web page. Given the newness of XR, the tools for generating such environments are immature and still evolving. However, it is surprising how little our writing (i.e. text creation) tools have matured to service this new environment for consuming text.

Here, ‘remediation’ draws upon Bolter & Grusin's Remediation [22], extending the scope of the term. They were broadly considering transition between established, distinct, media—e.g. a work moving from book to film. Bolter's 2001 updated Writing Space [21, (Ed.2) Ch.3] also considered ‘Hypertext And The Remediation of Print’ as well as graphics (Ch.4) and the early eBook (Ch.5)2. He saw that (hyper)text, once freed from the shackles of fixed pagination, could be used more flexibly.

Earlier, Joyce—Bolter's co-creator of the hypertext authoring system Storyspace—noted [46] that a hypertext tool could be constructive even for non-literary creative use. The malleability of presentation in the XR environment is a natural provocation to on-the-fly remediation of sources previously only accessible in the fixity of paginated print. Sect. 3.1 discusses remediation further.

We are now quite used to the term ‘document’ and (computer) ‘file’ being used interchangeably, but that change is quite recent (c.1967) [64]. This semantic drift also plays into the notion that (primarily) textual works should be understood in terms of their printed form, i.e. as seen on paper—be it literal or digital.

Pertinent here to ‘paginated-print-think’, is McLuhan's notion of ‘rear-view thinking’ when faced with the new:

The past went that-a-way. When faced with a totally new situation, we tend always to attach ourselves to the objects, to the flavor of the most recent past. We look at the present through a rear-view mirror. We march backwards into the future. [56, pp.74–5]

Indeed, now we need to look beyond the familiar conventions of (paginated) print. Happily, Hypertext's forward-looking community has always acknowledged a wider context to the understanding of text than as (paginated) print alone. From inception the Hypertext Conference, eschewing Snow's ‘Two Cultures‘ [74, pp.3–4], has not perceived a strict divide between formal writing and the looser creative and literary use of text: literary hypertext was never an ‘other’ but a peer to documentation and academic writing [5, Sect. 2.4], etc. As such, this community is well placed to consider how text creation tools may improve usefully, to move beyond the legacy constraints of print as their primary output.

Consideration has been given to given to ‘digital reading’, but focus tends to remain on the book metaphor [50, 67], and less on reading outside the page(-derived) form [51]. In exploring new interactions with text, we should not simplistically assume that they apply to all existing use of text. As will be seen, some contexts still need an immutable copy of record3 (see Sect. 3.1.1).

To explore the implications of these new opportunities, we will start by reviewing some of the contributory background elements—documents, files, paginated (paper) print, remediation—before considering how hypertext's past work may contribute to better text creation tools and address (unintentional) limits of our current tools and interchange formats. Due to limited space, the following contingent issues are explicitly not addressed, including: annotation of documents, application coding and design, and application UI. These are relevant but require their own proper, discrete, exploration.

XR is also just one example of the challenge ahead. Herein lies the provocative idea: what of the print era is useful as opposed to merely familiar?

2.1 What is a ‘document’?

2.2 The long shadow of PDF and paginated print

Printing of books, magazines and newspapers continues, but digital consumption of text ‘print’ is declining [11, 24]. This change will be gradual, but for those moving away from printed media14 there are already challenges in making best use of digitally native print, not least when remediated into short texts.

A further factor is the increase in variety of digital text display contexts. The replacement of early CRT monitors with lighter, flatter, devices has seen display embrace many sizes and formats: smart watches, mobile phones, tablets, TVS, desktop displays—and XR.

3.1 The future will be remediated

3.2 Writing for whom: author, reader or parser?

3.3 Additional considerations for academic documents

Information overload, and the challenge of recording it effectively, is nothing new: consider Blair's Too Much to Know [19], Linnaeus’ ‘slips‘ [77], and Luhmann's zettelkasten [49]. Early in the use of personal computers, Nelson noted in Computer Lib/Dream Machines:

Hierarchical and sequential structures, especially popular since Gutenberg, are usually forced and artificial. Intertwingularity is not generally acknowledged—people keep pretending they can make things hierarchical, categorisable and sequential when they can't. [61, p.‘DM31’]

and, expanding on this, he notes:

EVERYTHING IS DEEPLY INTERTWINGLED. In an important sense there are no "subjects" at all; there is only all knowledge, since the cross-connections among the myriad topics of this world simply cannot be divided up neatly. [60, p.‘DM45’]

However, the ability to provide metadata to the narrative text of document, to indicate the internal links and invisible structures makes the latter now less challenging than in the past, and requires the author's attention to detail—something new tools ought to assist. Yet, this ability to honour a text's underlying intertwingularity has been widely overlooked in most popular text creation software.

4.0.1 Typeset text and remediation. Storing the source text of a document with hard-coded typesetting (e.g. ligatures20, soft hyphens, etc.) privileges print-based viewing and passes, when extracted, ‘dirty’ text into any remediation.

Documents written with remediation in mind need—or gain significantly from— the ability to extract and expand the compressed narrative within. Whether this is to ‘decompress’ the document and see the main narrative in the context of its sources and peers, or to allow the information to be envisioned from differing perspectives, a task potentially aided by the potential malleability of an XR space.

4.0.2 Humans and computers read differently. Bush was early to contrast human associative thought, with the (then) strictures of erstwhile (computing) technology [26, Sect. 6, paras 1–2]. Despite the latest wave of interest in AI21 it is not clear that algorithms can understand human thought patterns and how we infer meaning. This leaves work for the author, especially if the primary reader is non-human (the choice of reader is rarely as chosen by the author).

Even with the Web's sub-set of wider hypertext, we have a fair appreciation of links between documents. Internally, this is less the case22 and where our text creation tools hide this ‘messiness’ from the text creator. Yet, intra- and inter-document links are essentially the same, if differing in purpose and use. Hiding the internal structure of the document may simplify the writing experience, though not necessarily to the author's benefit.

Remediation, especially when effected as decomposition of a document in to discrete parts, benefits from rich metadata as to the document's structure (q.v. Sect. 4). For academic papers, such structure is normally not shown in output documents beyond outline headings and a few elements like a reference list. Much of the rest is inferred from the narrative and reader's prior knowledge of the domain. For enabling remediation, writing tools need to be able to assist authors to record the structure of their narrative even though it is not part of the visible text output.

5.0.3 Exposing content. Engelbart's NLS/Augment had the notion of ‘viewspecs’ [32] [33, Sect. 7b3]. These are easily-invoked, differing, visualisations of the same source data. Today, the notion of diverse viewspecs within a program can best be seen expressed in Eastgate's Tinderbox [16]. More widely, viewspecs are generally underused at present23. Sadly so, as a single UI (viewspec) fosters a singular view of the ‘shape’ of the document and its narrative/content.

Text need not be constrained to orderly print. Brath's Visualizing with Text [25] shows text to be tractable in presentation style without losing coherence—it can increase the meaning imparted. For those who balk at abstract presentation, the ‘card’ metaphor best known from Atkinson's Hypercard24 [38] offers a less challenging but equally flexible approach to exploring the structure of a text25.

5.0.4 Spatial Hypertext. An overlooked aspect of visualising (hyper)text is Spatial Hypertext (SH), which can help exploration of a document's structure and internal/external links. SH has seen parallel evolution as an algorthmically-mediated forms and as a manually generated map. The former, has lineage from Aquanet26 [52], via VIKI [53] and VKB) [73]. Current work in this form is best seen in Atzenbeck's Mother system [7, 8, 9]. The non-algorthmically mediated SH started with Storyspace [15, 23] where it still remains in use; Eastgate's Tinderbox [16, 17] builds on this for more general writing use.

SH does not demand (visible) text, larger maps can be accommodating via abstractly styled notes (nodes) using simple shape, colour, etc. Link display can be (temporarily) suppressed if it adds visual ‘noise’ to the map. The spatial layout also lends itself to recognition of negative space: what is not mapped, linked, or simply unknown. Bernstein's Can We Talk About Spatial Hypertext? [18] provides a useful summary of the vocabulary and affordances of SH.

5.0.5 High Resolution Addressing. Effective remediation also needs good link addressing so that the disparate objects arising still know their relation to the whole and each other. Accurate addressing of content was a concern of the NLS/Augment project from outset [33, Sect. 6], and carried though to the early web by the notion of ‘purple numbers’ [47]27. HTML allows targeting of headings and (unique within-page) ID anchors, though these affordances are not foregrounded by word processors, the most widely used text creation tools.

Within the paper/document-based metaphor, ‘page’, ‘paragraph’ and ‘sentence’ give some accuracy but are brittle if text is re-flowed, although close addressing should be robust to re-flow. The Kindle e-reader has the notion of a #loc every 150 bytes 28 [31, p.99]. However e-readers seem designed more for basic linear reading where the primary need is the last-read-to place in the piece, whilst still acknowledging the source pagination. The more complex structure of academic communication warrants better addressing tools. These are not unexplored—q.v. initiatives such as the long-lived TEI [29, 76] and JATS (Journal Article Tag Suite) [62, 80].

5.0.6 ‘Block-level’ addressing. In the area of PKM29 software there is the Web/HTML-derived notion of addressing paragraph ‘blocks’30 as the object unit. PKMs like Roam and Obsidian31 use this approach. Roam also re-surfaces the pre-Web concept of links being two-way—see Sect. 5.0.5 below.

5.0.7 Links and linkbases. As well as visualising the structure within the document, and being able to accurate address content within the document, it is useful to be able to record (explicit) and explore (implicit) links within a document. Assistive (AI) processes can help with the former, but the latter benefits from being able to visualise the document and surface links not yet explicit. Before the Web set our general perception of hypertext and embedded links, the emergent Open Hypertext Systems (OHS) included the notion of a linkbases that stored links separately from the content. Normally this was via character offsets in the text stream. At the time text was generally simple unstyled ‘plain’ text, but if the text's style is stored discretely32, this method can still be use for today's more styled text. An interesting new approach is ‘standoff metadata’ [58, 59] based on JSON use and addressing a need in Digital Humanities for overlapping link anchors and targets.

Unlike the Web's HTML-embedded links, earlier hypertext systems embraced discrete link storage: the linkbase. Linkbases could be multiple and contextually applied as in Microcosm [36, 42], or federated as seen in Hyper-G [55]. Grønbæk and Trigg's From Web to Workplace [40] also usefully summarises OHSs.

5.0.8 Link types. Important to the expressing a document's inner relationships is to capture the purpose(s) represented by a link. Early hypertext explored link types in close relationship to structuralised argumentation, most fully expressed by Trigg [78]33. Sadly, distaste for the effort of setting link types led to disfavour, and link types were not actively adopted by the Web. However, they remain a strong potential affordance for remediation by categorising the relationships of content within a document.

5.0.9 Footnotes and other non-linear inclusions. The footnote arises mid-16c.34 allowing branching from the primary linear narrative. A fully linear narrative can be a textual straitjacket, as glosses or insertions intrude. Thus in print, typesetting costs often shunted these to being ‘endnotes’. A different presentational gloss, expensive typographically (and in paper paper use), is the marginal side note as is well displayed by Tufte's books (e.g. The Visual Display of Quantitative Information [79]). A large page margin might be wasteful of space in print form but can still elegantly allow space for text expansion36, yet such a need is moot in a remediated environment like XR—marginal space being only as context requires.

The sidenote concept sits well with remediation: if a (foot/side/ end) note knows its place in the text, in a remediated space its co-presentation as a sidenote seems an obvious provision. Similarly, figures, tables, and formulae are often inserted far from their associated text. Given metadata, representation of supporting content alongside text is an obvious effective re-use to aid presentation alongside its relevant text, even if there are multiple such pieces of text. Indeed, associated but discontinuous runs of text could be remediated as a continuous piece, alongside supporting material.

5.0.10 Structured abstracts. An innovation aiding granularity of addressing is the structured abstract [44]. This concept arose not in hypertext, but in the medical community in the late 1980s, and uses mandated labelled sections in abstracts. These guide both the human reader and offer tractable structure for computer readers. However, they could also, for remediation, use embedded metadata to indicate the purpose of sub-headings and link to relevant parts of the document.

In some areas of academic writing this structure could be extended further into document, though it is hard to generate generic heading/sections that are useful across all domains. But building on the structural notion it could inform document metadata and embedded anchors to aid both addressing and parsing of the whole.

5.0.11 Temporal aspects. Rosenberg's Cartographies of Time [71] shows visualising the temporal axis of information and text has a rich past. Others explore this through into the digital age [2, 25]. A remediation challenge is to capture dates meaningfully: not all dates have the same pertinence, so some authorial metadata hiting is useful. Anderson and Millard [4, p.103, Sect. 5 incl. fn. 21–23] explored the temporal axis of citation links within the hypertext Conference corpus37 and such methods could be usefully expanded for documents that are collections or have large numbers of references. Time has a natural axis but temporal relationships can also be seen as a proxy for other exploitable sets of terms (code-accessed metadata?), such as people, organisations, or domains of study.

5.1 Text and data

Naturally-arising limitations of print place constraints on current tools for creating text. The advent of the likes of XR does not mean 500 years experience of printed text is suddenly of lesser value. However, print's conventions and constraints are not ideally suited these newer forms of textual presentation. This is the challenge for writing tools to address. A lesson from PDF is to not use a valid format outside its design intent without due consideration. It is too early to judge whether existing writing tools will evolve to embrace the needs of remediation by aiding the recording of significantly more structural metadata. Nor does it matter unduly, whether it is new or existing tools—or both—that take up the challenge.

The provocation of the likes of XR remediation is not simply a technical challenge. Rather it leads to (re-)consideration of our teaching and practice of writing. No longer are we only writing linear text. The latter can contain complex and multi-stranded narrative, but its understanding often (assumes) implicit communication direct from author to reader, disadvantaging computer-mediated exploration and consumption of text.

A brief essay leaves little room for depth but it is to be hoped that the challenges and opportunities here are clear. The Hypertext community has much to offer in this context. Some ideas may appear old but that shows the forward thinking nature of the original work, that often ran beyond the technological capabilities of the day: yet they still hold value.

Dave Millard, Peter Wasilko, Dene Grigar, and Mark Bernstein kindly commented on drafts of this paper. Ideas here draw on the Future of Text group's [37] XR research (Sloan grant [39]), data re-use for UK's PSDI [68] and over two decades’ use of Eastgate's Tinderbox [16] and Storyspace [15] tools and supporting their users.

1A collective term for immersive Virtual Reality (VR) and AR (Augmented Reality). AR is reality overlaid with projected VR elements.

2Then, Web (hypertext) was far less capable and its use less widespread than nowadays.

3Immutable, even if only by controlled access to the original (digital document).

4‘What You See Is What You Get’. The origin is disputed, but it appeared in the late 1970s. The PARC ‘Bravo’ system is generally held to be the first WYSIWYG text editor.

5Cites Saenger The History of the Book in the West: 1455–1700, Vol.II [72, p.403].

6For more detail on the evolution of the paginated print book see The Book [45].

7By Warnock and others (at Adobe) from 1982–84. Also: Thinking in PostScript [70].

8This begs the question of what ‘the text’ comprises.

9Aspects of this issue were also reported in [4, Sect. 4.4].

10Note this mode has limited hardware and software support, and is tied to a single vendor as the PDF must be ‘converted’ for this type of use [1].

11Adobes credits a ‘Sensei’ AI. Liquid Mode does not alter the PDF but tries to detect structural elements to assist content re-flow for presumed small-screen use.

12Seen down the right edge of a column of text in PDF as unusual word breaks.

13The latter is currently only offered for papers generated using ACM's TAPS system.

14Either physical media or digital ‘printed’ media.

15Confusingly the term ‘eBook‘ is used to apply to both books set in fixed PDF typography and to those stored/rendered as (X)HTML.

16Ironically, PDF does support two-way intra-document link navigation.

17Text tools supporting such enhancement are missing or immature.

18Generally, these (peer-reviewed) papers are formally-published, long-form, essays or reports on a topic.

19For provenance, replication, and as a bulwark against corruption or malicious alteration of facts.

20Note: in some languages ligatures are structural and not merely stylistic artefacts.

21AI has been through a number cycles of (over-)promise and failure to deliver.

22A fair exemption being documents expressly written as hypertexts.

23Even MS Word has 4 viewspecs (View menu) but most only use the default ‘Print’.

24Hypercard's origin is interesting [6], reflecting on the interconnectedness of things.

25Scrivener's ‘corkboard’ view uses a card metaphor for exploring narrative order [20].

26Aquanet itself drew on Notecards [41] and gIBIS [27]: also see SEPIA [75].

27Looking further back, in 1965 before displays were common or ‘hypertext’ was defined, Licklider was reflecting on issues of citations (and the implicit links) [48, p.56]

28N.B. This is not a standard. Also see: MobileRead wiki article‘Page numbers ’ [57].

29PKM: Personal Knowledge Management.

30A line or paragraph being all the text between two literal line-breaks (or start/end).

31See: Roam (https://roamresearch.com) and Obsidian (https://obsidian.md).

32Thus separating pure textual content from semantic/stylistic data.

33This work was part of the first PhD awarded for Hypertext.

34Zerby The Devil's Details [83, Ch.2] to give less intrusive glossing. Of course, footnotes can have footnotes35.

35Hypertexually-informed narrative can support branching. See: Sect. 3.1 and [14].

36Body text expansion into margins in Even-Ezra's Lines of Thought [34].

37Partly inspired by Citeology’s [54] temporal visualisation, and by arc diagrams.

38Also see national data sharing initiatives in USA [28], UK [68], and Germany [30].

This work is licensed under a Creative Commons Attribution-Share Alike International 4.0 License.

HT '24, September 10–13, 2024, Poznan, Poland

© 2024 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-0595-3/24/09.

DOI: https://doi.org/10.1145/3648188.3678215