Digital dissertation defended!

Just a quick post to note that I successfully defended my digital dissertation on April 14th, 2015 and completed my Literature Ph.D. Check out the finished project at! (And I’m now on the job market for alternative academic positions, so let me know if you’re looking to hire someone with 8+ years of professional interface design experience on a strong research background!)

In addition to links to the Infinite Ulysses participatory digital edition and public code/design repo, the project site links out to a whitepaper covering the process and product of my project, an abstract for the whole project, a manifest listing and describing all the pieces of work that went into the project, and of course all my research blog posts during the project here on the blog. If you haven’t been following along, the dissertation is worth checking out even if you’re not interested in the digital humanities, participatory interface design, or Ulysses: I use a novel approach combining design, code, user-testing, site analytics, and blogging, without the traditional 4-5 chapters of proto-monograph. Tweet @Literature_Geek if you have any questions about the process or my research.

Huge thanks are due to my dissertation committee — Drs. Matthew Kirschenbaum (chair), Neil Fraistat, Melanie Kill, Kari Kraus (also Dean’s Representative), and Brian Richardson — for their intellectual and personal generosity. In particular, their willingness to learn about, support, and refine my project’s unique format and methodology; meeting with me as a team and always being available for discussion throughout the course of the project; and thoughtful evaluation of unusual deliverables for a literature dissertation.

I’m also grateful to the University of Maryland English Department, UMD Graduate School and Dean Charles Caramello, and the UMD Libraries Digital Repository for their support of my unique format and methodology, as well as the Maryland Institute for Technology in the Humanities (MITH) and its stellar staff for introducing me to the digital humanities; intellectual stimulation, community, and funding; and mentorship in diverse digital skills and staff roles.

My Digital Dissertation: Public Humanities, Participatory Design, and Infinite Ulysses

Update on 3/26/2015: is now a live site. Visit and read!

A transcript of my talk on digital humanities interface usability and my dissertation from the recent Nebraska Digital Humanities Forum is below, and it’s a great overview of the entire dissertation. For an even quicker overview of my dissertational research, please check out this three-minute video on my dissertation project!

“What if we build a digital edition and everyone shows up? Public Humanities, Participatory Design, and Infinite Ulysses”

(transcript of talk delivered at Nebraska DH Forum 2014)

Draft of logo for Infinite Ulysses site

Draft of logo for Infinite Ulysses site

What if we build a digital edition and everyone (millions of scholars, first-time readers, book clubs, teachers and their students) shows up and annotates the text with their infinite interpretations, questions, and contextualizations? The “Infinite Ulysses” project pursues this speculative experiment, and today I’m going to talk about how this unlikely hypothetical is helping me study and improve DH interface usability.

Textual scholar Morris Eaves conceives of editions not just as vessels for textual content, but as “problem-solving mechanisms”; and I’m using this speculative edition experiment to both

  • produce something practically useful, designing and coding an actual digital edition of James Joyce’s Ulysses with a number of experimental interface features, and then
  • conducting use and usability testing with real readers and researchers to focus on three interrelated lines of questioning:

1. First, a question about how such an experiment can benefit the humanities: How can we design digital archives and editions that are not just public, but invite and assist participation from both trained academics and the lay person in our love for the nuances of a text’s materiality, history, and meaning?

2. Second, a question about how we make such a thing as my speculative idea work: How can we borrow successful social mechanics from existing online communities (things like upvoting & tagging) to create reading and research experiences that adeptly handle not only issues of user-generated contextual annotation quantity but also quality?

3. And third, a question about how this interface work affects the texts at its core: What happens to complex texts—especially those purposefully authored to be hypertextual, chaotic, and encyclopedic, like Ulysses—when a participatory digital edition places them under heavy and thorough annotation and conversation?

Some Quick Definitions

I’ll be discussing how scholarly digital edition development helps us look at these three questions, and I’ll also address how the overall Infinite Ulysses project functions as an unusually shaped humanities dissertation. Before I go any further, though, I want to define some words I’ll be using a lot:

The “digital editions” I’m talking about are literary or other texts that have been prepared for online reading and research by a scholarly editor according to a carefully constructed interpretation and methodology. These are digital editions like those on the Whitman or Shelley-Godwin Archives, rather than the digitized/scanned and non-interpreted texts you find on Google Books.

I’ll also be talking about “annotations” a lot. When a reader “annotates” my digital edition, they’re highlighting a word, phrase, or other section of the literary text, then typing in a comment on that highlighting. This annotation then appears in the margin of the text depending on some other factors.

Finally, I’ll be talking about “speculative design“, a design approach that imagines future challenges and opportunities, and builds design solutions both imbued with theoretical innovation and the possibility of future utility. It’s a way of using hypotheticals to make connections you might not otherwise make. My designing a digital edition that could handle a hypothetical public wildly interested in annotating a complex text is a speculative experiment that’s helping me think about customizing annotations to individual researchers, and about how heavy annotation changes an unwieldy text like Ulysses.

Participatory, Public

Scholarly editor Gary Taylor has asked : “How can you love a work, if you don’t know it? How can you know it, if you can’t get near it? How can you get near it, without editors?” Scholarly editors are an integral part of the continuum that keeps the stories of the past available to and understood by the present—but just as important in Taylor’s formulation is the you, that public of not just scholars, but also readers in and beyond the academy whose interest keeps the humanities alive and relevant.

Lindsey Thomas notes that “making something freely available online does not necessarily guarantee that thing is public, or that it will be used as a public resource”1. I’m interested in building tools and digital editions that help both textual scholars and the lay person participate in our love for the nuances of a text; and I’m especially interested in how we can use interface design to encourage not just consumption of, but participation in the humanities.

One way of thinking about participatory platforms is as a classroom. What’s the connection between teaching and editions?

“[Nobel-Prize-winning physicist Richard Feynman] prided himself on being able to devise ways to explain even the most profound ideas to beginning students. Once, I said to him, ‘Dick, explain to me, so that I can understand it, why spin one-half particles obey Fermi-Dirac statistics.’ …Feynman said, ‘I’ll prepare a freshman lecture on it.’ But he came back a few days later to say, ‘I couldn’t do it. I couldn’t reduce it to the freshman level. That means we don’t really understand it.‘” — David L. Goodstein, in Feynman’s Lost Lecture: The Motion of Planets Around the Sun 

I like how this Feynman quotation expresses that teaching is knowing, that we learn something more intimately when compelled to make our knowledge declarative for others, and if we’re not able to teach a thing, then we might not know it as well as we thought. When I think about designing welcoming digital editions and other digital humanities sites, I often think about how we can make them more like a really good graduate seminar experience, where scaffolded discussion helps you think and discuss better.

The flip side to the metaphor of a classroom as a participatory space is that many digital editions are kind of participatory once removed. By aiming their content at teachers with things like teaching guides and sample assignments, they do a great job at humanities outreach, but the learning and discussion usually happens off the actual digital project site, and thus doesn’t feed into the experience of other users.

Participatory Design

I’ve just discussed a more loose way of thinking about “participatory” design. There’s also a more rigorous definition for participatory design that comes from the field of human-computer interaction (or HCI). HCI, a field focused on interfaces between user and computer (including interfaces of reading and research), provides this useful approach to a humanities that simultaneously benefits the public and scholars. Participatory design is an approach not just centered on the scholar or just on the public, but finessing the “third space” of discussion and shared learning that exists when the two groups come into conversation; Michael Muller has found that results can include “challenging assumptions, learning reciprocally, and creating new ideas, which emerge through negotiation and co-creation of identities, working languages, understandings, and relationships, and polyvocal dialogues across and through differences”2.

Some sites and projects are intended “primarily with scholars in mind”, and these are necessary and important contributions to the academy and human knowledge. On the other hand, I feel like non-designers are too quick to repeat the cliched “you can’t design for multiple audiences”, instead of thinking “how do I design for this space where we learn together?” For example, in a formal usability study of public use of several key digital archives, I found that it was often very simple interface changes users desired in order to be able to understand and use a DH site. When we bring the public into our scholarly conversations, everyone can benefit, and it’s important to remember that designing for interactive participation and pedagogy is itself a scholarly activity that also benefits the public.

Infinite Ulysses: What’s It Going to Look Like?

Let’s turn to the Infinite Ulysses digital edition and how I’m designing it to be participatory. Here’s the simple version of what my edition will let scholars and readers do: read the text online on their laptop, iPad, or tablet; when they run into something in their reading, whether that’s a question, or an interpretation or context they wish to share, or an answer for a question left by another reader, they’ll be able to highlight that piece of the text and add that as an annotation, which will then show up to the side of the text. Readers can also add tags to these annotations and up- or down-vote them, as well as have the option to fill out site profiles with fields for research interests, whether you’ve read the text before, and other details to aid in site customization.

The site lets readers add notes and otherwise discuss the text, then uses those notes to make a more customized experience for each reader.

The site lets readers add notes and otherwise discuss the text, then uses those notes to make a more customized experience for each reader.

I’m not going to demo the site because I’ve been focusing on coding the interface behavior and not on the final theming design yet. The final site won’t look like this, but it’s a general sense of the layout:

Old wireframe capturing general layout of site (but not current design work).

Old wireframe capturing general layout of site (but not current design work).

  • a central column of reading text
  • a drawer on the right for creating and reading annotations
  • a drawer on the left for customizing your view of the text

Fun Potential Issues with which I’ll Be Dealing!

This project combines its speculative design approach with the scholarly primitive of curation (dealing with information abundance and quality and bias), imagining scholarly digital editions as popular sites of interpretation and conversation around a text. Of course, with our hypothetical millions of users and annotations, we can’t display all of them next to the text, and you’d quickly see the entire text covered in a single block of yellow highlighting if we left everything highlighted that ever got annotated. By drawing from examples of how people actually interact with text on the internet, such as on the social community Reddit and the Q&A StackExchange sites, I’m creating a digital edition interface that allows site visitors to create and interact with a potentially high number of annotations and interpretations of the text.

The big questions I’m trying to answer with this interface design are:

  • If everyone’s submitting annotations to a digital edition, how can we automate the massive task of curation and moderation so that it occurs in a timely and unbiased manner?
  • And once we’ve separated the wheat of critical yet diverse annotations from the chaff of repetitions, spam, and under-substantiated suggestions, how do we make the still-plentiful remaining material accessible to the users it would best serve?
  • That is, how do we separate the signal from the noise when the “signal” of pertinent contextual annotations means different things for different reader needs?

My project is testing out to what degree using tags, upvoting, and profile information can help customize the reading experience for each reader. The hypothetical here is that we’re able to filter the annotations to match the level of help a person needs, or their research interests; and, when there are still too many annotations available for a given word or phrase, at least cycle randomly through better subsets of annotations than the entire available pool. I’ve been coding to follow what seems to work best for existing successful online communities, including by visiting these communities’ spaces for meta-analysis such as the subreddit “theory of reddit”, but we won’t know how well this works with a digital edition until I begin my usability testing this fall.

Site Features

I won’t read through all the examples of functions I’ve been coding for (you can read a full list in this post), but here are a few examples of things you should be able to do with the edition:

  1. filter the annotations to only show those on particular areas of interest (e.g. annotations that translate foreign words, or annotations aimed at first-time readers)
  2. filter out annotations the reader doesn’t want to see (e.g. spoilers about something that only becomes clear later in the book)
  3. easily create a curated “edition” of Ulysses showing only annotations aimed at a specific topic (e.g. an edition meant for use by a class focusing on the place of books, letters, newspapers, and other written material in the Modernist novel), and/or only see annotations added by specific users, such as members of a class or book club

So, will we get enough annotations, tagging, and voting to make the site work? Even if the site doesn’t have many more annotations and tags than the base set provided by me and several contributing scholars, it’ll still be a useful public site. Some years back, I built a prototype non-participatory digital edition of Ulysses that was much simpler, but dealt with similar questions of interface design for ease of Ulysses first-time reading; unfortunately it’s now very technically dated and has a frightening color palette; I still hear from random people around the world that they’ve read its two chapters of Ulysses and want more, though, so I do think there is already a desire for a certain way of presenting Ulysses online with annotation. I’m really hoping to be able to support a resource that is not only read, though, but also participated in, so that we can hear many more voices in our discussion of Ulysses.

Will Readers Tag?

The LibraryThing blog has a great article on how people tag things for themselves, not just because a site offers it, for personal reasons like aiding in memory and socializing. LibraryThing has found success with users tagging books, while Amazon’s book tagging feature has failed (I didn’t even know you could tag books!). I choose Ulysses for my test text in part because it occupies a special position: difficult enough that people welcome help in reading it, yet still very much in the popular consciousness as something that can and should be read (as opposed to, say, Finnegans Wake). People tweet about trying to read Ulysses all the time, hold reunions with book clubs with whom they read the text in the past, and celebrate the Ulysses holiday of Bloomsday every June. There’s an audience out there to support with such a site, and I’m expecting that people will use the site’s features in a way that benefits themselves but also others, whether they choose to annotate and tag and vote to interact with people they know offline like book clubs, or want to create a customized annotated edition for teaching or research use, or want to help track clues throughout the book, or otherwise serve the same purpose as sticky notes and penciled marginalia in print books.

Digital Humanities Use and Usability

I’ll be conducting both use and usability studies to understand and build from user feedback. Usability is concerned with whether the site lets users simply and intuitively do what the site is meant to do—that is, read Ulysses and participate in a discussion around it. To study this, I can for example see how much time or how many clicks it takes a user to get to the text and add a question or other annotation. Use is the equally interesting study of how people actually use the site; do they want to tag annotations? do they want to do unexpected things like literary role-playing using the annotations? To some extent, I’ve tried to design for expected “non-official” uses, such as realizing that people would use the annotation feature to ask questions and supporting it both by allowing filtering for questions and answers, and creating a link to a feed of all unanswered questions.

Literary Texts under Participation

The third question this project is helping me explore is about Ulysses itself. What happens to complex texts—especially those purposefully authored to be hypertextual, chaotic, and encyclopedic—under heavy and thorough annotation and conversation?

My project builds on previous Joycean contextualizing work: such as Blamire’s Bloomsday Book and Gifford’s Ulysses Annotated. Previous contextualizations (especially those on a higher, more macro-level of understanding the basics of Ulysses rather than specific allusions) carry an increased risk of authorial bias because of their individual authorship (e.g. the Bloomsday Book is very interested in charting Bloom’s development as a Christ figure); while the sources I’ve named are all excellent and useful pieces of scholarship that have improved my enjoyment of the novel, I’m very interested in seeing how a participatory edition might moderate among a variety of higher-level contextual interpretations and whether we might thus prevent a canon of interpretation from calcifying around the text, or at least multiple popular key streams of thinking around the text within the same digital space. Ulysses is also an interesting contextualization problem in that there are many sets of contextualizations floating around the web from both scholars and amateurs (in the old and new senses of the word); I’m interested in bringing diverse sets of interpretation together in one place and seeing how they overlap or contend with one another.

Some key print precedents for this project are variorum and other editions that deal with a large quantity of variants, marginalia, or other annotation, such as the Shelley-Godwin manuscript notebooks and Hans Gabler’s synoptic Ulysses edition; these last two examples have convenient digital counterparts in Michael Groden’s never-finished Ulysses in Hypermedia prototype, and the developing Shelley-Godwin Archive.

Ulysses + Hyperannotation = ???

While there isn’t a complete critical digital edition of Ulysses yet published, that hasn’t kept Joycean scholars from anticipating issues that might arise with the eventual migration to digital space. Where the limitations of print space have in the past kept annotations of the text in check, what will happen when a digital platform allows the addition and navigation of infinite annotations? Can we migrate complex print hypertexts such as Ulysses to a digital space with socially multiplied annotations without, as Mark Marino wonders, “diminish[ing] the force of the book”? Marino asks, “Assuming it were possible, would the creation of a system that automatically makes available all the allusions, unravels all the riddles, and translates foreign languages normalize Joyce’s text?”

Derek Attridge similarly sees a risk in Ulysses‘ hypertextualization and hyperannotation, saying “[Ulysses‘] cultural supremacy, and the scholarly efforts which reflect and promote that supremacy, have turned it into a text that confirms us in our satisfied certainties instead of one that startles and defies us and thus opens new avenues for thought and pleasure. It now provides a spurious sense of rich complexity by reducing differences and distinctions”.

Yet Attridge also sees space for promise in the digital development of the text, saying “The very magnitude of the encyclopedic Joycean hypertext can itself be unsettling… and it may be possible to produce a hypermedia version of Ulysses that is anything but reassuring—one that revives, in new ways, the provocations and disturbances of the original publication… The best teachers (like the best critics) are those who find ways to sustain the disruptive force of Ulysses even while they do their necessary work of explaining and demystifying”. I’m hoping that by going back to my earlier definition of a participatory edition—one that functions like a classroom—we might be able to collectively make Ulysses‘ chaotic wonder accessible to more people, without also dampening that wonder.

So, despite there being no full digital edition of Ulysses against which to test these fears and assumptions about hypertextualization, we already have some questions about what happens to a complex Modernist text when “everyone shows up”, and I’m hoping that by creating a site that experiments with allowing “infinite” contextual annotation of Ulysses, we can get a more realistic picture of what extreme annotation actually does to our experience of the text. When even Ulysses editor Hans Gabler says he still learns new things about Ulysses from time to time, I’m confident that Ulysses will persist as always partially unfixed, always giving back more.

How does this work as a dissertation?

The Infinite Ulysses digital edition is a dissertational project. In addition to the Drupal edition site, which will include the theme design, module coding in JS and PHP, and incorporated user feedback, the overall project has four other deliverables:

  1. Public code repository and documentation on using my code to build a similar digital edition
  2. Formal usability study
  3. Regular blogging on the technical and theorizing processes
  4. Scholarly digital article built from the blog posts and the usability study

So why this format and these methodologies? My primary field of interest is textual studies, a sub-field within literary studies that concerns itself with the scholarly acts of editing and edition-making, as well as preserving literary texts; more specifically, I’m interested in digital editing, which migrates these traditional scholarly practices to the online world, opening up possibilities for new ways of reading and studying literary texts. Much of the recent activity in textual studies as a discipline has followed this path, as an examination of the program for the annual Society for Textual Scholarship conference would confirm.

I feel that the most productive and efficacious (in terms of scholarly impact) way to push the current research, reading, and teaching capabilities of the digital edition further is to actually demonstrate them in practice, through the hands-on work of designing, building, and testing a digital platform for new editing practices. In many respects, my project is not much different from the previously accepted practice of producing for the literature dissertation a scholarly edition, complete with commentary and editorial apparatus.

In a field concerned with scholarly editing and edition-making, we need editions that experiment boldly not only with the editing of texts, but also with the crafting of those textual frames, treating the edition interface as an argument and meta-text worthy of experimentation and innovation.

If you enjoyed this post, you might also enjoy the feed of my regular dissertation blogging, including posts on the public digital humanities via participatory interface design, precedents for scholarly experimentation as critical work, and how I’m demonstrating my intellectual values via this project.

This was a Digital Humanities Now Editors’ Choice post. Logo for DHNow Editors' Choice Post

  1. Making something freely available online does not necessarily guarantee that thing is public, or that it will be used as a public resource…’Openness,’ as we tend to understand the term in the Digital Humanities, might not necessarily involve ‘making public’ as we are used to thinking about it.” — Lindsey C. Thomas, “Open Access and the Digital Humanities“ []
  2. Michael J. Muller. “Participatory Design: The Third Space in HCI”. In The Human-Computer Interaction Handbook. 1051-1068. []

Infinite Ulysses: Planned Features for a Participatory Edition

Update on 3/26/2015: is now a live site. Visit and read!

Yesterday’s post introduced some of my research questions with the “Infinite Ulysses” project; here, I’ll outline some specific features I’ll be building into the digital edition to give it participatory capabilities.

This project combines its speculative design approach with the scholarly primitive of curation (dealing with information abundance and quality and bias), imagining scholarly digital editions as popular sites of interpretation and conversation around a text. By drawing from examples of how people actually interact with text on the internet, such as on the social community Reddit and the Q&A StackExchange sites, I’m creating a digital edition interface that allows site visitors to create and interact with a potentially high number of annotations and interpretations of the text.


So that readers on the beta-site aren’t working from a blank slate, I’ll seed the site with annotations that offer a few broadly useful tags that mark advanced vocabulary, foreign languages, and references to Joyce’s autobiography so that the site’s ways of dealing with annotations added by other readers can be explored. Readers can also fill out optional demographic details on their account profiles that will help other readers identify people with shared interests in or levels of experience with Ulysses.

On top of a platform for adding annotations to edited texts, readers of the digital edition will be able to:

1. tag the annotations

For Stephen’s description of Haines’ raving nightmare about a black panther, a reader might add the annotation “Haines’ dream foreshadows the arrival of main character Leopold Bloom in the story; Bloom, a Jewish Dubliner, social misfit, and outcast from his own home, is often described as a sort of ‘dark horse’“. This annotation can be augmented by its writer (or any subsequent reader) with tags such as “JoyceAutobiography” (for the allusions to Joyce’s own experience in a similar tower), “DarkHorses” (to help track “outsider” imagery throughout the novel), and “dreams”.

2. toggle/filter annotations both by tags and by user accounts

Readers can either hide annotations they don’t need to see (e.g. if you know Medieval Latin, hide all annotations translating it) or bring forward annotations dealing with areas of interest (e.g. if you’re interested in Joyce and Catholicism)

Readers can hide annotations added by certain user accounts (perhaps you disagree with someone’s interpretations, or only want to see annotations by other users that are also first-time readers of the book).

3. assign weights to both other readers’ accounts and individual annotations

As with Reddit, each annotation (once added to the text) can receive either one upvote or one downvote from each reader, by which the annotation’s usefulness can be measured by the community, determining how often and high something appears in search results and browsing. Votes on annotations will also accrue to the reader account that authored those annotations, so that credibility of annotators can also be roughly assessed.

3. cycle through less-seen and lower-ranked editorial contributions

To prevent certain annotations from never being read (a real issue unless every site visitor wishes to sit and rank every annotation!)

4. track of contentious annotations

To identify and analyze material that receives an unusual amount of both up- and down-voting

5. save private and public sets of annotations

Readers can curate specific sets of annotations from the entire pool of annotations, either for personal use or for public consumption. For example, a reader might curate a set of annotations that provide clues to Ulysses‘ mysteries, or track how religion is handled in the book, or represent the combined work of an undergraduate course where Ulysses was an assigned text.

I’m expecting that the real usage of these features will not go as planned; online communities I’m studying while building this edition all have certain organic popular usages not originally intended by the site creators, and I’m excited to discover these while conducting user testing. I’ll be discussing more caveats as to how these features will be realized, as well as precedents to dealing with heavy textual annotations, in a subsequent post.

First Wireframe

In the spirit of documenting an involving project, here’s a quick and blurry glance at my very first wireframe of the site’s reading page layout (a higher-res, readable wireframe with a walkthrough should follow in the next few weeks). I thought of this as a “kitchen sink wireframe”; that is, the point was not to create the final look of the site or to section off correct dimensions for different features, but merely to represent every feature I wanted to end up in the final design with some mark or symbol (e.g. up- and down-voting buttons). The plan for the final reading page is to have a central reading pane, a right sidebar where annotations can be authored and voted up or down, and a pull-out drawer to the left where readers can fiddle with various settings to customize their reading experience (readers also have the option of setting their default preferences for these features—e.g. that they never want to see annotations defining vocabulary—on their private profile pages).

I’m looking forward to finessing this layout with reader feedback toward a reading space that offers just the right balance of the annotations you want handy with a relatively quiet space in which to read the text. This project builds from the HCI research into screen layout I conducted during my master’s, which produced an earlier Ulysses digital edition attempt of mine, the 2008/2009

UlyssesUlysses does some interesting things in terms of customizing the learning experience (choose which category of annotation you want visibly highlighted!) and the reading experience (mouse over difficult words and phrases to see the annotation in the sidebar, instead of reading a text thick with highlightings and footnotes). On the downside of things, it uses the Project Gutenberg e-text of Ulysses, HTML/CSS (no TEI or PHP), and an unpleasant color scheme (orange and brown?). I’ve learned much about web design, textual encoding, and Ulysses since that project, and it’s exciting to be able to document these early steps toward a contextualized reading experience with the confidence that this next iteration will be an improvement.


Because code modules already exist that allow many of these features within other contexts (e.g. upvoting), I will be able to concentrate my efforts on applying these features to editorial use and assessing user testing of this framework. I’ll likely be building with the Modernist Commons editing framework, which will let me use both RDF and TEI to record relationships among contextualizing annotations; there’s an opportunity to filter and customize your reading experience along different trajectories of inquiry, for example by linking clues to the identity of Bloom’s female correspondent throughout the episodes. Once this initial set of features is in place, I’ll be able to move closer to the Ulysses text while users are testing and breaking my site. One of the things I hope to do at this point is some behind-the-scenes TEI conceptual encoding of the Circe episode toward visualizations to help first-time readers of the text deal with shifts between reality and various reality-fueled unrealities.

Practical Usage

Despite this project’s speculative design (what if everyone wants to chip in their own annotations to Ulysses?), I’m also building for the reality of a less intense, but still possibly wide usage by scholars, readers, teachers, and book clubs. This dissertation is very much about not just describing, but actually making tools that identify and respond to gaps I see in the field of digital textual studies, so part of this project will be testing it with various types of reader once it’s been built, and then making changes to what I’ve built to serve the unanticipated needs of these users (read more about user testing for DH here). To sign up for a notification when the site is ready for beta-testing, please fill out the form below (or visit the form here if it’s slow in loading onto this page). Tomorrow and Thursday, I’ll be discussing precedents for the project’s use of the edition as an experiment, as well as some caveats and unknowns I’ll be facing as I continue this work.

Update on 3/26/2015: is now a live site. Visit and read!

Infinite Ulysses: What if we build a digital edition and everyone shows up?

Update on 3/26/2015: is now a live site. Visit and read!

Digital humanities productions are increasingly sites of broader participation in textual interpretation, with how to evoke and harness “meaningful” crowdsourcing becoming an increasingly urgent question to scholars seeking a more public humanities.

First slide from my presentation on the Infinite Ulysses project.

(CC BY-NC Amanda Visconti)

In a previous post, I discussed how borrowing the HCI idea of “participatory design” can help DH practitioners and their public audiences to mutual intellectual gain. As we use the Web to open the texts we study to a wider community of discussion, bringing in diverse knowledges and interpretive biases, I’m interested in how we can usefully structure the overabundance of information proceeding from public contextual1 annotation of literary texts. That is: my “Infinite Ulysses” project is conceived as an idealistic speculative or thought experiment: what if we build an edition and everyone shows up and adds their annotations to the text? When this quantity of voices is combined with an unusually complex text such as James Joyce’s Ulysses, how might we create a critical experience that adeptly handles not only issues of contextual annotation quantity but also quality?

I’ve previously blogged about the overarching plans for my digital dissertation: about how I’m empirically testing both current and personal theories about textuality through code and design, and how I’m designing this building-as-scholarship towards helping everyone—textual scholars and the lay person—participate in our love for the nuances of a text’s materiality, history, and meaning. This post is about the first of my dissertation’s three coding projects: designing and coding an interface that allows a participatory, vibrant, contextualizing conversation around complex Modernist texts, with Ulysses being my main focus. Packed into one sentence, my research question for this first project is: What might we learn from crafting an interface to usefully curate quantity and quality of contextual annotation for complex Modernist digital editions (such as Ulysses), where the critical conversation is opened to the public? That’s a long sentence covering both more abstract and more concrete critical work, and I’ll be breaking it into manageable chunks with the rest of this post and those to follow.

“Infinite” annotations.

While there isn’t a complete digital edition of Ulysses yet published, that hasn’t kept Joycean scholars from anticipating issues that might arise with the eventual migration to digital space. Where the limitations of print space have in the past kept annotations of the text in check, what will happen when a digital platform allows the addition and navigation of infinite annotations? Can we migrate complex print hypertexts such as Ulysses to a digital space with socially multiplied annotations without, as Mark Marino wonders, “diminish[ing] the force of the book”2:

Assuming it were possible, would the creation of a system that automatically makes available all the allusions, unravels all the riddles, and translates foreign languages normalize Joyce’s text? (Marino)

In Joyce Effects: On Language, Theory, and History, Derek Attridge similarly sees a risk in Ulysses‘ hypertextualization:

[Ulysses‘] cultural supremacy, and the scholarly efforts which reflect and promote that supremacy, have turned it into a text that confirms us in our satisfied certainties instead of one that startles and defies is and thus opens new avenues for thought and pleasure. It now reassures us of our place in what might otherwise seem a chaotic universe, or it provides a model of coherence to take a refuge in, a satisfying structure where the details all make sense… a spurious sense of rich complexity by reducing differences and distinctions3

Yet Attridge sees space for promise in the digital development of the text as well:

The very magnitude of the encyclopedic Joycean hypertext can itself be unsettling… and it may be possible to produce a hypermedia version of Ulysses that is anything but reassuring—one that revives, in new ways, the provocations and disturbances of the original publication… The best teachers (like the best critics) are those who find ways to sustain the disruptive force of Ulysses even while they do their necessary work of explaining and demystifying4

Despite there being no full digital edition of Ulysses against which to test these fears and assumptions, we already have some questions about what happens to a complex Modernist text when “everyone shows up”, and I’m hoping that by creating a site that experiments with allowing “infinite” contextual annotation of Ulysses (that’s where the “Infinite Ulysses” title of my project comes from), we can get a more realistic picture of what extreme annotation actually does to our experience of the text. When even Ulysses editor Hans Walter Gabler can still learn new things about Ulysses5 , I’m confident that Ulysses will persist as always partially unfixed, always giving back more.

Quantity and quality, signal and noise.

While the final effect of unlimited space to discuss and interpret the text remains to be seen, the “Infinite Ulysses” project will also tackle two more immediate problems: quality and quantity of annotation. If everyone’s submitting annotations to a digital edition, how can we automate the massive task of curation and moderation so that it occurs in a timely and unbiased manner? And once we’ve separated the wheat of critical yet diverse annotations from the chaff of repetitions, spam, and under-substantiated suggestions, how do we make the still-plentiful remaining material accessible to the users it would best serve? That is, how do we separate the signal from the noise when the “signal” of pertinent contextual annotations means different things for different reader needs?

So: what happens to a complex Modernist text when we allow “infinite” annotations on it, and how do we work with “infinite” annotations to filter, order, and display those annotations best suited to a given reader? Tomorrow’s post will explain the specific features I’m coding into my interface to approach these questions.

Update on 3/26/2015: is now a live site. Visit and read!

  1. Contextual annotation = notes that give a context for a word, phrase, or other chunk of text, ranging from definitions of advanced vocabulary to more interpretive annotations []
  2. Mark C. Marino, “Ulysses on Web 2.0: Towards a Hypermedia Parallax Engine” []
  3. Derek Attridge, Joyce Effects: On Language, Theory, and History, 185 []
  4. Derek Attridge, Joyce Effects: On Language, Theory, and History, 186-188 []
  5. Digital Ulysses Master Class with Hans Gabler, June 2013 []

DH User Testing: A Four-Part Series on Making Your DH Work User-Driven

I wrote my iSchool master’s thesis on evaluating and serving audiences for digital humanities archives (full text available here; my advisor was Professor Paul Conway), and my nascent Ph.D. dissertation will incorporate user testing of my own digital literary code as well as suggest some best practices for DH user testing on a limited budget. Given this background, I’ll be publishing for the next three four days on user testing, an area of evaluation that foregrounds the user–the use, usefulness, and usability of your digital work for a given audience. It’s my hope to make each of these posts applicable to people working on small budgets and/or on their own. The posts in this series are:

  1. Wednesday, October 31st: Quick and Dirty DH User Testing
  2. Thursday, November 1st: The Amateur in the Archive: Toward a Wider Audience for Your DH Project
  3. Friday, November 2nd: DH User Testing: What’s Been Done? (Adding to a Growing List)
  4. Saturday, November 3rd: Testing DH Sites: More on Use, Usability, and Usefulness

The posts cover “quick and dirty”, practical ways of making your DH project more user-aware; an argument for designing DH projects that serve not only scholars, but also a wider, public-DH audience of “amateurs”; a description of a use study run on amateur users of digital humanities archives; existing examples of DH user testing by researchers from different backgrounds, on different types of audiences; and choicemaking about evaluating use, usefulness, or usability for your DH work.

Follow this blog this way. You might also want to check my crossposts of these blogs over on HASTAC (Humanities, Arts, Science, and Technology Advanced Collaboratory), which have received some comments with additional resources for DH user testing.

Evaluating Non-Traditional Digital Humanities Dissertations

Most of the online advice I’ve encountered for evaluating non-traditional scholarly projects seems to focus on seeking jobs or applying for tenure with a portfolio that is significantly digital; the process of translating that body of work for a reviewing committee matches up pretty well with how evaluation works for the dissertation defense process for a non-traditional and/or digital project (and I’ve covered some of these suggestions at the end of this post). Charting progress and recording effort throughout the non-traditional dissertation, though, is a mostly undiscussed topic.

Having a unique format scrubs a lot of the traditional methods for evaluating these factors: maybe there’s no “chapter” to turn in, or you’ve undertaken a critical building project in a programming language your committee can’t read (and even if your committee could read through the entire body of code you author, you wouldn’t want them to need to—just as with a monograph project, you want to be respectful of mentors’ time by asking for feedback at specific milestones and with some kind of concise artifact to assess).

My dissertation has a unique format—there’s no monograph, and the bulk of the work is blogging, design, coding, user-testing, and analysis of testing data. I’ve come up with a comfortable set of practices that let me keep my dissertation committee updated and on-board with my expected outcomes, give me some handy accountability to produce a reasonable amount of weekly work, and clearly document my effort and progress in case administrators even request evidence of the equivalency of my research with that of students using a more time-tested format. This post will cover how I’m recording my progress and sharing recognizable milestones with my advisor and committee;  I use a combination of GitHub, Zapier (an IFTT-like service), Basecamp, scheduled emails, and GoogleDocs.

GitHub for evaluating any type of writing

I’m using GitHub to manage my code (more below), but it turns out GitHub has a lot of nifty ways for me to let my advisor and committee have a sense of my coding progress and effort—visualizations that would work just as well for someone using GitHub to track changes in their writing, for example. I’ve added my advisor as a collaborator on my dissertation code repository so that he can also see the progress charted there.

List of commit messages for GitHub code

Example code commits give a summary of what the code I’m adding to the repository achieves (e.g. did it fix a specific bug? add a new feature?).

Whether or not the text you’re working with is code, GitHub lets you track frequency and quantity of work, and offers some nifty visualizations to give you an overview of your work habits. With a project where you’re learning or advancing a technical skill, this can be a nice way of showing that even if something you spent two weeks on didn’t pan out (e.g. I ended up not using my Islandora digital edition work), you were doing active work during that time.

GitHub visualization options include (note all examples are from a repo opened at the end of August):

  • Viewing the number of code commits made by week:

GitHub bar chart of code commits by week.

  • Evaluating your work habits by showing a “punchcard” of what days of the week and times of day you commit the most code (mostly sticking to my commitment to not dissertate on weekends so far!):


GitHub "punchcard" chart of code additions and deletions by day and time of day.

  • Charting code additions and deletions by week:

GitHub chart of code additions and deletions by week.

 GitHub for code tracking



I’m managing my dissertation code via a completely private repository that regularly gets pulled into a private repository that only certain collaborators can view (sharing read-only code to only select people is slightly roundabout).

I’m using GitHub to backup and track my coding work for all the regular webdev reasons (lots of copies keeps stuff safe, it will be easy to release code to the public when it’s ready, etc.). Part of my deliverable work is good documentation of my coding, so I need to keep the repository README updated. (A repository, aka “repo”, is the place on GitHub where my code is stored, which you can visit and view as webpages; a README is a text file sharing some information about that repo, visible at the bottom of the front page of a GitHub repository).

Screenshot of the README file from my code repository.

README file explaining what the code in the repository does/will do.

I’m using the README in two ways:

  1. A place to explain what my code does, why I made certain technical choices, and (once the repository becomes publicly accessible) how anyone can use my code for their own research (e.g. dependencies, installation and configuration instructions, where to ask for help).
  2. A place to track and visibly credit the various pieces of FOSS (free and open-source) code that my project builds off.
Screenshot of README file describing provenance of code in my repository.

Tracking the provenance of various code collections I’m building off.

My project focuses on creating and testing new interface abilities by building off existing wheels such as Drupal (for the site structure itself) and Annotator.js (for the core textual annotation functionality). Many of the terms of the various FOSS licenses on these code collections dictate that I attribute this work properly, but even without such licenses visible and detailed attribution would be extremely important to me. Everyone should get credit and thanks for sharing their work with others! (See the awesome “Collaborators’ Bill of Rights” that came out of a MITH workshop for more on why correct credit should matter to everyone). Plus, making it clear that this project is founded on a variety of existing code sets might encourage others to think about platforms that could emerge from combining or augmenting existing code, giving us new tools and approaches instead of sinking time into reinventing wheels. Since I’ll also be producing a lot of my own reusable code (plugins, interface), this page will also help me make it clear to future employers what pieces of code are authored by me.

Basecamp for project progress

Basecamp is a project management tool that also works well for teaching courses (and it’s free for teachers!) or working on solo projects. Among other features, it lets you create various task lists containing tasks with due dates:

Screenshot of tasks on a Basecamp task list

Some of the tasks in my Basecamp project; this list is helping me track what needs to be done before I start alpha-testing.

Basecamp will also help me automatically turn feedback from user-testing into tasks on my dissertation to-do list, using Zapier, a service that connects various other internet services (e.g. get an email when one of your tweets is RT’d, and other IFTTT-like options).

Screenshot of using Zapier to turn GitHub issues into Basecamp tasks

Using Zapier to create Basecamp tasks from GitHub issues.

GitHub repositories allow people to create “issues”, which are like comments on the code or software in the repository; I’ll then place labels on these issues depending on what type of problem they represent: a bug, a feature request, or something out of scope for the dissertation project. Once issues get a label, they automatically appear on a Basecamp task-list as to-dos containing the text in the issue and assigning me a due date to address that issue in the next two weeks. You can get started similarly managing GitHub issues using this Zap by using the template I’ve shared.

Screenshot of issues in my GitHub repository

Some example issues on my GitHub repository; each issue automatically became a to-do with a due date on my Basecamp project after I labeled the issue, thanks to Zapier.

Emails and GoogleDocs

I’ve scheduled myself to create a weekly email to my advisor covering three things:

  1. what I accomplished on the dissertation over the past week,
  2. what my goals were for the following week, and
  3. anything I needed him to do or read.

These emails are both useful in keeping my advisor up to date with my work and as a record of that progress, should we ever need to demonstrate it to university administration (although I’ve gotten written consent for my unique dissertation format from the interested parties as described in this post, so I’m not expecting to actually need these records). Weekly emails are also effective for me as a carrot and/or stick: I know I need to make some progress during the week or the email will be empty, and writing the email reminds me of all the things I got done in a week.

My four dissertation committee members have been awesomely willing to meet with me as a team. I’ve met with them all in the same room twice so far at milestones during my dissertation process (once while designing the dissertation and preparing the prospectus, and once when I was starting to have a good idea of what the final project deliverables would look like). We’ll be meeting again in the next weeks as I finish site functionality and begin user-testing.

Meeting as a team has been a great way to make feedback into a conversation among several areas of expertise. It’s also been great to reaffirm that the committee understands what the deliverables at the time of the defense will look like, and that they still find that outcome satisfies the requirements of a dissertation in my department. Once I find a meeting time that works for everyone, I send them an agenda for what I’ll cover in the meeting; this helps me use their time wisely, as well as let them know the type of feedback I’m seeking or any questions I might need them to answer (e.g. I let them know I’d be checking if they still felt my deliverables worked as a full dissertation, so they can revisit my project beforehand and come to the meeting prepared with any questions or concerns). To make getting prepared for the meeting easier, I’ll also share a GoogleDoc highlighting what I’ve accomplished since our last meeting; how my project, timeline, or expected deliverables have changed (if at all); and a list of linked titles for the blogs I’ve published since the last meeting.

Evaluation at the end of the dissertation

That’s how I’m tracking my work throughout the dissertation, but how do you establish when a non-traditional dissertation fulfills the normal checks on dissertation completion—has the project demonstrated the ability to design and pursue a scholarly research project at the doctoral level? If you parallel the dissertation with the tenure-application process, you increase the amount of available reading about digital humanities research evaluation. There are guidelines like the MLA’s for digital projects, the Center for Digital Research in the Humanities’ resource page on “Promotion & Tenure Criteria for Assessing Digital Research in the Humanities“, and the Carolina Digital Humanities Initiative’s resource page on “Valuing and Evaluating DH Practice”. Digital humanities scholars who seek tenure are publicly blogging about their experiences more often, and tips about packaging for tenure review blog posts, tweets, and other examples of scholarly public engagement are frequent among digital humanists on Twitter. Just as tenure applications are starting to include ways scholars have engaged the public, keeping track of online reactions to your project (blog post reading stats, Twitter comments, and RTs for tweets about your dissertation) can serve as a testament to the meaningfulness of your research and dissertation format, as well as offer a kind of shallow mid-stage peer review (thinking along the lines of: retweets of links to posts about your project may be an extremely mild and imprecise form of peer review).

Would you like to volunteer as an early user of the Infinite Ulysses site? Sign up here!

Infinite Ulysses Technical Update: Summer 2014

The core functionality of my website is the ability to highlight pieces of the Ulysses text and leave annotations on them (questions/answers, interpretations, contextualizations, notes); to achieve this, I’m building off Annotator.js—Javascript code that lets you do what I just described, with a variety of plugin code that lets you add neat features like a heatmap of where the most annotations appear on a text (e.g. on what pages did readers have the most questions? what pages had the most contentious interpretations?). Since the site itself is built with Drupal, I’m using the Drupal modules already built to let Annotator.js work with, for example, Drupal’s database as the place to store readers’ annotations.

Not as easy as it sounded (is it ever?)

That the Drupal log under the skull and crossbones.

That’s the Drupal logo under the skull and crossbones. This was on my home dry-erase board for a while…

My original project plan was to spend a lot of time coding the look and behavior of the reading interface, with the site functionality not re-inventing various wheels, but instead making connections among existing sets of code that allow textual annotation and social commentary, voting, ranking, etc. In practice, the existing Drupal-module versions of Annotator.js are buggy. Two big examples (for me):

  • the Drupal Annotation modules (Annotation and Annotator) don’t store tags added to annotations, so any tags added by a reader disappear forever on refreshing or changing the page (and tags really need to be treated as entities in their own right, and tied into the core Drupal taxonomy); and
  • the code that sets up an individual textual annotation to be an “entity” (a Drupal type of content—such as a blog post, book page, or comment—that other existing Drupal modules automatically know how to act on) isn’t completely working either (e.g. you can’t add fields—such as a field to add tags to an annotation, fixing the tag storage problem—with the current code).

Having everything not go as smoothly as planned was actually great, though! I had a realistic goal in terms of what I needed to get done by the end of the summer to stay on track (have the site’s main functions almost all working, ready to be completely finished at the end of September), and I wasn’t worried about doing different work than expected to achieve those goals. Having the Drupal annotation modules need more work than expected meant I got to spend the summer trying various things to solve these issues, and learning a lot in consequence—like creating non-basic custom Drupal entities.

For example, I learned how to make a custom entity fieldable (able to have fields, like a place where you can add tags and have them autocomplete if they match existing tags) and add some fields to it, both by writing code (“programmatically”) as well as via the Drupal admin interface (after making the annotation visible to the core Field UI module). I also read through the Annotator code so many times that I became much more comfortable with it, so I’ll be able to augment it in the future far more easily.

In the end, it looks like I’ll be building off the code fix created by another DH developer who was kind enough to start an online conversation with me about our mutual work improving textual annotation in Drupal (more on that project once they’ve decided to make their code public). I went back and forth about whether I wanted to build off their solution or keep hacking at my own, but in the end I hadn’t been expecting to need to make those code fixes in the first place, so I decided I’d keep my remaining coding time this year free for expected work with the actual reading interface, like making a heatmap of the annotations and porting other existing Javascript/Coffeescript plugins to work with the Drupal version of Annotator.

Other summer work

Screengrab of many many tabs open across the top of my browser.

Screengrab of many many tabs open across the top of my browser. A fairly normal view when I’m coding!

Patching. I submitted an extremely simple patch to the Drupal Annotation module (fixing a misspelled variable) as a way of learning the process; my patch was accepted into the module. Now I know how to create patches when I want to share further work with others using Drupal Annotation.

Installation profiles. I’m currently working on two separate local versions of the Infinite Ulysses website: one to work on improving the Annotation function code, and one for work on everything else (e.g. user profiles, site content, theming). As I work on the site’s annotation abilities, I fairly often want to reinstall the site as a slightly quicker alternative to weeding all the references to annotation code out of the database (I should set up uninstall hooks to do this for me eventually…), or to see whether the code works correctly when installing the modules into a new site. Keeping the annotation work on a separate site lets me do this without losing my other work on the site.

But! I don’t want to start from a standard Drupal blank site every time I need to reinstall my annotation code; I want a sample page of Ulysses to test the code on, user permissions already set up, modules installed, and various other things configured that I’d rather not do manually every time I reinstall the site. Installation profiles are a way to avoid starting from square one with a Drupal site, instead automatically pre-configuring the site as you’d like it to be (think of it like a module that tells the site to configure and install things a certain way during the site installation process). When I visit install.php, I now have a “Infinite Ulysses Pop-Up” option in addition to standard or minimal Drupal installations. Eventually, this profile will turn into a distribution, such that anyone can use it to install a Drupal site pre-configured to act as a participatory digital edition.

Annotation sidebar. The Annotator.js code I’m working with displays annotations as pop-ups hovering over the text, but from experience with my UlyssesUlysses prototype and usability research, I prefer having annotations authored and displayed in a sidebar. This will also be handy for the tablet version of the website, as you’ll be able to slide the annotation sidebar off and on the page like a drawer. I’ve been playing around with a couple ways of achieving this: modifying this existing CoffeeScript or this extension of that code (demo’d here) to work with Drupal, creating the sidebar by resetting the current CSS on annotation pop-ups and styling them into a sidebar, or a tactic such as this repo which purports to display annotations as Drupal comments.

Book-page styling. I’ve been vacillating a bit as to the final display structure of the novel: I’m pretty sure I either want to have one page per chapter of the book and infinite scroll to put all the text of a chapter on one page, or have separate pages to match up with each page of the 1922 text of the novel (which I’m using). This is something I might end up doing A/B testing on, showing the text to some visitors one way and to other visitors the other way, and comparing their reactions and stats such as time spent on the site. Here are some questions I’ll need whatever solution I use to address:

  • How many annotations can load on a page at a time? How does this limit or non-limit affect site speed?
  • How do annotations load on a very long page that only loads partial sections of the text at a time (when they’re scrolled to, i.e. infinite scroll)?
  • How will bookmarking on the site work—will a reader’s last visited and furthest page in the book be marked as where they should start next, or will users have the option (or necessity) to bookmark one or more pages themselves?
  • If I do offer one long scrolling webpage per chapter of the novel, can I easily generate bookmarks that jump a reader down to the last line they were reading?
  • What other ways can users chart their progress through the book, such as badges per chapter, or a progress bar for pages read?
  • What if readers want to read it out of order, or only some parts online (and mark parts they read offline as completed)? What if they want to glance ahead without losing the bookmark at where they actually want to pick up reading next?

Long-term thinking. I met with the artist and IP lawyer behind the excellent web/iPad comic of Ulysses (Ulysses Seen). We’re all keenly interested in opening Ulysses to more public reading and conversation; they want a way for comic readers to be able to jump from the comic to the text, and I’m also interested in supporting the comic’s readers in enjoying the text—so we’ve been discussing combining our websites into one super-accessible, textual and visual Ulysses edition, after I’ve defended my dissertation. This is also a good way for me to think about how I’ll manage hosting costs and moderation effort in the future.

Other summer activities included lots of reading (as much as I read, I’m always adding more links to my reading list every time I go on Twitter), researching the social modules I want to use on annotations, and drafting site content. In particular, I’ve been looking into how existing communities (especially online ones where people submit text and/or converse) have succeeded in building friendly spaces, self-moderating, creating community guidelines, and fostering an inclusive and welcoming atmosphere.

Would you like to volunteer as an early user of the Infinite Ulysses site? Sign up here!

User-testing a digital edition: Getting the feedback you need

I’ve been thinking about specific questions I want to ask during the user testing for Infinite Ulysses (my dissertation project)—more specifically, Rachel’s tweet had me thinking about how to describe to user-testing volunteers what kind of feedback I’m seeking.  I came up with some statements that model the types of thoughts users might have that I’d like to know about. For the first phase of beta-testing on my project, I’ll ask testers some abbreviated form of:

Have one of the following thoughts, or similar? Please elaborate:

  1. “I wanted to see ___ but didn’t / couldn’t locate it / stopped looking for it even though it’s probably there”
  2. “This is broken”
  3. “This is cool”
  4. “This could be done better (by ___?)”
  5. “This doesn’t work how I expected (and that is good / bad /should be changed ___ way)”
  6. “Where is ___ again (that I used before”
  7. “This requires too many steps / is hard to remember how to use”
  8. “Don’t see how / why I’d use this”
  9. “I’d use this ___ way in my reading / teaching / work”
  10. “I gave up (where, when)”
  11. “____ would make me like this site / like it better / use it regularly”
  12. “I’m not interested in Ulysses, but I’d like to use this interface for a different novel / non-fiction text / (other thing)”
  13. “Starting to read the text on the site took too long (too much to learn / too much text or instruction to wade through) / took the right amount of time (intro text and instruction was appreciated or easily skippable, or site use was intuitive enough to get started)
  14. “I would recommend this site (to x person or y type of person)”
  15. “The problem with this site is ___”
  16. “Reading on this site would be easier if ___”
  17. “I wish you’d add ___”

Testing stages for Infinite Ulysses

As I start to get all my design and code ducks in a row for this project this month, I’ll be moving into a cycle of user-testing and improving the website in response to user feedback. I’ll be testing in four chunks:

  1. Self-testing: Working through the site on my own to locate any really obvious issues; for example, I’ll step through the entire process of signing up and reading a chapter on the website to look for problems. I’ll step through the site with different user personas in mind (imitating the different backgrounds and needs of some of my target audiences, such as first-time readers of Ulysses and teachers using the site with a class). I’ll also apply various website assessment tools such as validators for code and accessibility.
  2. Alpha testing: Next, I’ll run some low-stakes testing by inviting my dissertation committee, close friends, and family to break the site. This might get me to a point where the next stage of testers aren’t hitting any problems big enough to take the site down or make testers wait while I take days to fix a difficult issue.
  3. Beta testing: I’ll conduct beta-testing this fall and spring by opening the site to exploration and use by the people who have generously volunteered via this sign-up form. Phase I will take place this fall and take feedback from volunteers using the site individually; Phase II will take place in winter and early spring, continuing individual use of the site, and adding in people using the site in groups, such as teachers with their classes, or book clubs reading together.
  4. Post-release testing: I’ll continue to take feedback once the site goes live for use by anyone in June 2015, although I’ll need to scale down work on requested enhancements and focus on bug fixes and continued data gathering/analysis on how people use the site to read. Setting up site logging and Google Analytics on my site will help me monitor use as time allows.

User testing how?

I’ll be building on my user-testing experience from my master’s research and the BitCurator project, as well as trying some new tactics.

The thesis for my information master’s degree involved a use study exploring how members of the public (and others with a content interest in a website, but lack of experience with digital humanities and edition/archives commonplaces) experienced scholar-focused DH sites, using the Blake Archive and Whitman Archive as examples. I was particularly interested in identifying small design and development changes that could be made to such sites to better welcome a public humanities audience. For my master’s research, I built off existing user study metrics from a related field (learning technology) as well as creating and testing questions suggested by my research questions; feedback was gathered using a web survey, which produced both quantitative and qualitative data for coding and statistical analysis.

I’m hoping to further set up

  • web surveys for willing site visitors to fill out after using the site
  • shorter web pop-up questions—only for users who check a box agreeing to these—that ask quick questions about current site use (perhaps incentivized with special digital site badges, or with real stickers if I can get some funding for printing)
  • in-person meetings with volunteers where I observe them interacting with the site, sometimes having them talk aloud to me, or with a partner, as to their reactions and questions as they use the site
  • various automated ways of studying site use, such as Google Analytics and Drupal site logging

For bug reports and feature requests, site visitors will be able to send me feedback (either via email or a web form) or submit an issue to the project’s GitHub repository. All bugs/enhancement feedback will become GitHub issues, but I don’t want to make users create a GitHub account and/or figure out how to submit issues if they don’t want to. I’ll be able to add a label to each issue (bug, enhancement request, duplicate of another request, out of scope for finishing my dissertation but a good idea for some day, and won’t fix for things I won’t address and/or can’t replicate). I’m using Zapier (a If This Then That -like service) to automate sending any issues labeled as bugs or enhancements that I want to fix before my dissertation defense to Basecamp, in an appropriate task list and with a “due in x days” deadline tacked on.

To read more about user testing in the digital humanities, check out my posts about

User testing for the long haul

I’ve got one major technical concern about this project (which I’ll discuss in a later post), and one big research-design concern—both related to the “Infinite”-ness of this digital edition. My research design concern is the length of this user-testing; I’m pursuing this project as my doctoral dissertation, and as such I’m hoping to defend the project and receive my degree in a timely manner. Usability testing can be done over the course of a few months of users visiting the site and my iterating the design and code; testing use and usefulness, as in

  1. how people want to use the site (i.e. perhaps differently from how I imagined),
  2. how people read Ulysses (a long and complex book which, if you’re not attempting it in a class or book club, might take you months to read), and
  3. what happens to a text like Ulysses as it accrues readers, their annotations, and the assessment the social modules lets readers place on others’ annotations (the more readers and annotations, the more we can learn)

are things I can begin to gather data on, and begin to speculate on what trends that data suggests we’ll see, but I won’t be able to give them the full treatment of years of data gathering within the scope of the dissertation. To address this, I’ll both analyze the data I do gather over the course of months of user testing, and try to automate further data-gathering on the site so that I can supplement that analysis every few months or years without requiring too much effort or funding to sustain this work.

Would you like to volunteer as an early user of the Infinite Ulysses site? Sign up here!

Getting Digital Humanities Done: Schedule, Software, Etc. for a Digital Dissertation

Attending the Scholars’ Lab/NEH Speaking in Code symposium emphasized for me how much of a research developer’s work involves tacit knowledge, and this opacity extends to scholars in general when it comes to how stuff—books, articles, projects—get made. You get semi-polished glimpses of a work becoming itself, from its beginnings (grant applications, abstracts) to the draft stage (actual drafts, as well as blog posts and conference talks feeling out the research) to the end point (white papers, finished products, and sometimes honest discussions of a project’s struggles). But what does doing this work look like on a daily basis?  What tools do you use, and where, and when? Matthew Kirschenbaum writes:

Nobody teaches you how to write a book. Yes, in graduate school, you may get “feedback” on your dissertation to a greater or lesser extent from mentors and peers. But that typically has very little do with the process of executing on a marketable book project. (Kirschenbaum, blog post)

Kirschenbaum’s post attempts to improve on this lack of example for how scholarly work gets done, discussing practical details of his book-writing process such as citation management. I’ve tried to emulate this example-sharing by blogging about my dissertational process, from the technical choices I’ve made to the theoretical questions I’m considering. In this post, I’ve asked myself what I’d like most like to learn about others’ DH work set-up, and decided that the physical and screenspace environments, most-used software and websites, and work habits were the aspects I would most benefit from seeing other examples of. I’ll describe the workplace set-up, schedule, and software that help me make progress on my Infinite Ulysses project, with the hope of hearing more from others about the day-to-day environment and behavior that produces their digital humanities work.


MITH. I’m in residence at MITH as the Winnemore Digital Dissertation Fellow this academic year, and the plan is to do most of my remaining dissertation work at the office. MITH is generally either quiet or has a pleasant hum, but sometimes I’ll block out sounds while I’m blogging or on a particularly frustrating problem. I saved up for some noise-canceling headphones, which are pretty magical at producing quiet even when I’ve been trying to work in an airport. In the past, I’ve used cheaper but still useful alternatives such as earplugs or normal headphones playing free white noise. Coffitivity is great when you want want something a little more realistic in the background.

I use an external monitor, keyboard, and mouse, both for the extra screen space and to position everything at the optimum height for my wrists and neck.

A photo of my fellowship desk at MITH, Fall 2014.

Home. I worked entirely at home over the summer, but my set-up there is pretty similar (external monitor, desk/chair/externals adjusted to promote good posture/happy muscles).

In terms of screen space usage, I’ll have one Chrome browser window open with a number of tabs up (or two separate windows if I want to separate research/question-solving from website work), as well as a minimized Firefox window (for checking site design and script using the plugin Firebug). I’ll also have a text editor open with several different code files I’m working on or consulting, and the GitHub for Mac application open for committing (saving/uploading) new code to my private dissertation code repository (to be made public soon).

Work schedule

Time. I have the most energy in the morning; I was the kind of person who preferred taking classes in the 8am slot during college. Being up early lets my day start at a slow pace, and I have a bit of time by myself to get into a good working frame of mind and plan what I want to accomplish that day.

I get to MITH a little before 8 A.M. and stay until I hit a good stopping point for the day, either mid- or late afternoon. I do this for four consecutive days a week, with some extra email/organization/reading work on those evenings after I go home and try to jog. On the remaining three days, I try to do nothing related to the dissertation, focusing instead on fun projects (creating a new theme for this blog, learning to bake really good bread, trying to increase my running distance, learning Go). So far, this is a great schedule: I get way more done than if I try to say “I should be working on my dissertation 40+ hours a week and on every single day!”, and I’m always refreshed and excited to get back to my research after a few days resting my brain with other things. I think I’d allow myself to work on one of these days off if I was really excited, but I never want to feel like I need to; getting my project done a bit sooner isn’t worth burning out or not being able to think about anything but my project.

Scheduling tasks. I previously used TaskPaper, a task list app from the makers of the distraction-free writing app WriteRoom (which really helped with writing my candidacy exam presentation!), to keep track of what I needed to do on my dissertation. That got a bit unwieldy. I’ve recently switched to keeping a TaskPaper list for things I want to do on non-dissertating days (so I can write them down and not have them distracting me), and using Basecamp to manage teaching and research tasks. Trello is a good, free alternative, but I ended up finding the set-up for Basecamp matched the structure of my task list a bit better (I’m also more used to Basecamp from using it for MITH projects).

On Basecamp, I’ve attempted to break down everything I need to do in the remaining year (knock on wood) of my dissertation into small tasks, then added a date to the tasks I want to complete in the next month, so I can filter to a view of just what I need to accomplish today or this week in order to stay on track. Checking off tasks is pretty gratifying, and having a clear overview of what needs to be done by what point (after first, of course, creating a project timeline that I could reasonably follow, with a lot of empty buffer time at the end for all the things I can’t plan for) keeps me from getting stressed about what I haven’t done yet—if I complete the tasks I need to do this week this week, I know I’m on schedule.

Within a given week, I try to vary between coding-focused days and other activities such as organization of tasks and URL bookmarks, blogging, and reading/research. I generally start the week with coding so that if a problem proves frustrating, I can switch to a non-code day to clear my mind before trying to figure out the issue again.


Citations. I’ve managed citations in the past with the free and awesome Zotero (e.g. for my master’s thesis), and I expect to do so again once I start creating my final article about my project. If you do a lot of your research on the Web, Zotero is fantastic—you can use it to pull citation info from a page you’re reading, as well as take a snapshot of the page in case it disappears later.

Research for me right now means a mixture of theoretical articles (e.g. papers about the social reception of Ulysses, or studies on meaningful crowdsourcing for the humanities, or blog posts), example digital editions and other digital textual work (e.g. Alan Galey’s Visualizing Variation), and support for my research coding/design (e.g. code snippets, the Drupal StackExchange, tutorials, and documentation). I initially saved these bookmarks into various folders, but the folder names at the start of my project proved to be way too vague and all-encompassing two years down the road (“Scholarly Sources to Read”, anyone?). I recently re-organized these so it’s easier to find what I need, with more specific folder titles.

Screenshot of Chrome bookmarks menu with folders for various dissertation topics

I wanted to keep the folders in a certain order, but kept losing that when I’d sort their contents by title—thus the numbering per folder. In my research resources folder, you can see some more specific sub-folders, such as one where I’m keeping track of the various digital edition and textual annotation tools I considered and tested before ending up working with Annotator.js.

Coding. I read, edit, and write code in bbEdit (the pro version of the free TextWrangler), which has far more fancy features than I can regularly use. Features I’ve found useful include scripts that convert minified CSS and JS into readable, well-structured code; folding (collapsing) sections of a code file I’m not working on and don’t need to see; and searching across multiple folders.

Track changes. Keeping track of changes in my code is important for several reasons: I want to document effort for my dissertation committee (especially as my dissertation looks different than those they’re used to), give proper credit and correct licensing for any pieces of open-source code I build on, and be able to roll back my code to an earlier state if I need to. I use git to do this, mostly through the website GitHub and a GUI app (GitHub for Mac) that is quicker for me than using git on the command line.

I’ve started creating Drupal installation profiles after I make major changes to my work, so I can easily re-create my site without needing to reset all the options and reinstall various modules; eventually, I’ll package all my work as a Drupal distribution, so that anyone can spin up a participatory digital edition website like mine with no more trouble than setting up a vanilla Drupal website (and if setting up a vanilla Drupal website is difficult or unknown to you, I’ll provide user-friendly instructions on that as well—or you can check out Quinn Dombrowski’s awesome Drupal for Humanists).

Command line and local work. I don’t use the command line as much when I’m just working locally (i.e. on a version of the website that lives on my computer and isn’t accessible on the Web), and I’ve been mostly working locally lately. When I need it, I use Mac’s Terminal app to manage things that are easier to do from the command line than via the GUI, such as Drush (a set of commands that makes doing sysadminy and developer things with a Drupal site quick), or to move files around. When I’m working on a site on the Web, I use the command line to install things, edit files, and more. When I want to work locally on my Drupal site, I use MAMP Pro (the paid version of the free MAMP), which imitates the things I’d be using on a web server to make a Drupal site work (e.g. MySQL).

Technical support. An important piece of tacit understanding we discussed at Speaking in Code was the ability to successfully search for answers to technical problems; surprisingly, knowing how to Google well as a coder—especially when technical words have alternative popular meanings—is a difficult skill to impart. Since someone has probably encountered the same or a similar problem to yours before and written about it, and coding today is all about not re-inventing the wheel but building off of existing open-source code, knowing how to search for what you need is a vital technical skill.

I’ve been working on some of the core functionalities of my Drupal site lately (things related to highlighting and annotating text in a digital edition), so my main searches have to do with Drupal, PHP (which Drupal is largely written in), and Javascript (the core functionality of my site is provided by Annotator.js). Besides pure Googling when I hit an obstacle, I visit the module pages for Annotator and Annotation as well as their issue queues (to see if anyone else has run into the same issue), and the Stack Overflow,  Drupal Answers, and Server Fault Q&A sites. I’m also on the Annotator.js list-serv, which is fairly low traffic and offers a mix of news about the community of Annotator users/developers, and developer questions.

Blogging. I’ve covered how I figured out what I needed to blog about in this post on affinity mapping.

Other work set-ups

If you’ve similarly written about the day-to-day process of your scholarship—or are inspired to do so—let me know and I’ll link to it!