Figure 2. Annotation view with links to annotated segments of documents
Gene Golovchinsky
FX Palo Alto Laboratory, Inc.
Spatial hypertext has arisen from overview “map” representations of conventional node-and-link hypertexts (Shipman and Marshall, 1999). These spatial representations (e.g., Streitz et al., 1989; Marshall and Shipman, 1995; Shipman et al., 2001), can be used to model abstract relationships, encoding them using a variety of visual attributes such as relative or absolute position, size, color, shading, etc. Each node in such a data structure represents one or more objects.
Such interfaces can be effective tools for managing, organizing, and categorizing large quantities of information, but tend to require considerable cognitive effort to generate useful structures. Research in this domain has focused on representation and computation over inferred structure of the hypertext.
At least one important aspect, however, remains unexplored. In many cases, users may not be creating knowledge representations “from scratch,” but rather managing references to existing documents or concepts. Although VKB (Shipman et al., 2001), for example, allows users to associate URLs with objects in the hypertext, it limits the ways in which these resources may be manipulated. It is possible to follow a link to display the resource in a web browser or other relevant application; alternatively, content may be embedded directly into the workspace. Each possibility, however, has its limitations. Copying content into the workspace reduces it to plain text or image format; retaining it in the parent application makes it difficult to take advantage of spatial hypertext manipulations.[1] Yet it should be possible to combine the display of and interaction with documents while providing manipulation capability typical of spatial hypertexts.
In this paper, we will describe XLibris, a system that permits spatial manipulation of document fragments, and provides a hybrid spatial and node-and-link hypertext interface to a collection of documents with or without conventional hypertext links. We begin with an overview the system and its interface, and then focus on its hypertext features. The conclusion identifies several directions for further exploration.
XLibris is an application for annotating documents with freeform digital ink (Schilit et al., 1998). It unites the flexibility and natural and familiar expression of marking with a pen on paper with the computational capacity of a conventional computer for the purposes of reading and annotating documents. It is designed to reduce cognitive and interactional overhead of transitions among retrieval, reading, organizing, and writing, activities typical of many white-collar jobs.
In XLibris, a reader marks documents with a stylus or other pointing device (see Figure 1). The system records these freeform marks in a persistent manner. The reader can then retrieve, re-retrieve, manipulate, organize, and link documents based on these marks. We implemented several kinds of links in XLibris (Price et al., 1998), including conventional one-to-one manually-authored static links, many-to-many links based on matching ink shapes, and one-to-many links based and on statistical similarity between passages (Golovchinsky et al., 1999). These links either connected a pair of points (perhaps in the same document) directly, or used an intervening selection among alternatives to mediate navigation.
|
|
Figure 2. Annotation view with links to annotated segments of documents |
The annotation view (Figure 2) made effective use of this technique. Through it, a reader could bring together annotated passages from different documents, filter them by color, potentially sort them by time of annotation, and use this list to move among documents. This was one of the features that made it worth-while to read and annotate on the computer.
Testing with users, however, revealed a flaw in the design: because the system computed passages to be displayed based on the presence of annotations, it was not always easy to find a desired passage if the annotation was removed, or if other marks were added. Our field work (Marshall et al., 2001) suggested a need to create persistent, ad hoc references to document segments that could be manipulated by the reader to record key passages, ideas, facts, etc. They also needed a way to organize ideas and document references in preparation for writing.
These constraints, coupled with the notion of a legal notebook, lead us to design a spatial hypertext interface for collecting and organizing clippings from documents. Passages displayed in the annotation view (Figure 2) could becopied to the notebook (Figure 3) where they could be juxtaposed with other clippings (perhaps from other documents), organized thematically, annotated further, etc. Any number of “notebook” pages could be created to hold the clippings, and arbitrary annotations could be added, as shown. Each clipping could be resized and repositioned to reveal exactly the desired document fragment, regardless of the presence of annotations.
Figure 3. Notebook with clippings forming a spatial hypertext. Clipped document segments can be resized, repositioned, and annotated.
Links to the source page for each clipping were provided, so that the reader could move back and forth between organizing and cataloging on one hand, and reading and collecting materials on the other. This interface added organizational capabilities of spatial hypertext to the navigational capabilities of conventional hypertext. The reader could now simultaneously read and make sense of a passage, examine it in its context in the document, and compare it to other passages in other documents. The annotation capability made it easier to review and re-read, while simultaneously providing alternate means of organization and re-retrieval.
Navigational hypertext has come to form the substrate of our interactions with electronic documents. Spatial hypertext has developed from an overview mechanism into a fully-fledged form of expression, capable of representing structure from atomic objects to whole collections. The distinction between the two, however, is arbitrary from a user’s perspective. Users don’t care how information access is mediated – they want to have the right information at hand at the right time, to be able to read and write, to be able to make and record observations and inferences, to retrieve and re-retrieve.
The challenge to user interface and document designers is to convert these often latent needs into coherent design specifications. Spatial hypertext is just one tool in the designer’s toolbox. It is good at organizing semi-structured information, but may lack rhetorical structure that people can use to understand the space. This flexible means of expression may help a person capture knowledge, but this very flexibility may make more difficult for others to understand it. Spatial hypertext leaves much unstated, for better or for worse. We need to have much more practical experience with such interfaces to understand when they are appropriate, and when they get in the way. We need to understand how to merge space and narrative in a flexible manner appropriate to the opportunistic nature of human thought. How can we let readers combine and recombine fragments to create new documents? How can we unite space and time, sequence with juxtaposition, to create interactive experiences that reflect the capricious nature of human thought?
I would like to thank Laurent Denoue, Cathy Marshall, Morgan Price, and Bill Schilit for the collaborations that lead to the work described here. I would like to thank Joe Sullivan and Jim Baker for supporting this research.
Golovchinsky, G., Price, M.N., and Schilit, B.N. (1999) From Reading to Retrieval: Freeform Ink Annotations as Queries. In Proceedings of SIGIR’99 (Berkeley, CA, July 15-19). ACM Press. pp. 19-25.
Marshall, C.C., Price, M.N., Golovchinsky, G., and Schilit, B.N. (2001) Designing e-Books for Legal Research. In Proceedings of JCDL 2001 (Roanoke, VA, June). ACM Press. pp. 41-48.
Marshall C.C., and Shipman, F.M. (1995) Spatial Hypertext: Designing for Change. Communications of the ACM, 38, 8 (August 1995), 88-97.
Price, M.N., Golovchinsky, G., and Schilit, B.N. (1998) Linking by inking: Trailblazing in a paper-like hypertext. In Proceedings of Hypertext ’98 (Pittsburgh, PA, June 20-23). ACM Press. pp. 30-39.
Schilit, B.N., Golovchinsky, G., and Price, M.N. (1998) Beyond Paper: Supporting Active Reading with Free-form Digital Ink Annotations. In Proceedings of CHI98 (Los Angeles, CA, April 19-26). pp. 249-256.
Shipman, F.M. III and Marshall, C.C. (1999) Spatial Hypertext: An Alternative to Navigational and Semantic Links. ACM Computing Surveys.
Shipman, F.M. III, Hsieh, H., and Airhart, R. (2001) Analytic Workspaces: Supporting the Emergence of Interpretation in the Visual Knowledge Builder. Available on the Web at http://www.csdl.tamu.edu/~shipman/vkb/vkb.html. Downloaded July 7, 2001.
Streitz, N. Hannemann, J., and Thuring, M.> (1989) From Ideas and Arguments to Hyperdocuments: Travelling through Activity Spaces. In Proceedings of Hypertext ’89 (Pittsburgh, PA). ACM Press. pp. 343-364.
[1] Manipulations are performed on the proxy object, when the corresponding content may not be visible or accessible.