Crowdsourcing Interpretation / Praxis and Prism

Our goal in the Scholars’ Lab Praxis Program is to address methodological training in the humanities not just through workshops and courses, but by involving graduate students in digital projects from the ground up. This means learning by creating something – together – with all that entails: paying attention both to vision and detail; building facility with new techniques and languages not just as an academic exercise, but of necessity, and in the most pragmatic framework imaginable; acquiring the softer skills of collaboration (sadly, an undiscovered country in humanities graduate education) and of leadership (that is, of credible expertise, self-governance, and effective project management). All this also involves learning to iterate and to compromise – and when to stop and ship.

To do this, our Praxis team needed a project. We wanted it to be a fresh one, something they could own. It was important to us that the project only be in service to the program – that its intellectual agenda was one our students could shape, that they set the tone for the collaboration, and that – as much as possible – it be brand-spanking-new, free from practices and assumptions (technical or social) that might have grown organically in a pre-existing project and which we might no longer recommend.

In this inaugural year of the Praxis Program, the Scholars’ Lab, in consultation with some colleagues from UVa’s College of Arts and Sciences, is providing the central idea for the project. It’s just too much to ask that students new to digital humanities work invent a meaningful project from whole cloth on Day 1 of the program – especially one that, we hope, will make a meaningful intervention in the current scene of DH research and practice. That said, by the end of this year, our current Praxis team plans to have conceptualized a second project (or perhaps an extension of this one) to pass on to next year’s group.

Here endeth the preamble. What are we up to now?

This year, the Praxis Program is building a web-based framework, codenamed “Prism,” for collective marking of texts according to small and constrained (but flexible) interpretive vocabularies. Prism will enable visualization of those marks – made by many users on the same document – as zoomed-out, rainbow-like spectra. It will also (should we get so far!) allow for comparison and analysis of the results of users’ activity (that is, their collective attention paid to certain passages of text, and the categorizations they make of those passages) by treating them as input for the data-mining techniques we can apply against large corpora of digitized texts. In other words, Prism will be a blunt but very interesting and user-friendly tool for crowd-sourcing humanities interpretation.

The basic concept has several sources. It stems in part from conversations on categories of textual interpretation, led by Johanna Drucker and Jerome McGann, in which I participated as a graduate student at SpecLab (see especially chapter 2.5 of Drucker’s book), as well as from a fond memory of markup games I played in my UVa Media Studies classroom and with SpecLab colleagues, including (among several others) Drucker, McGann, Andrea Laue, Worthy Martin, and Nathan Piazza. These games and discussions fed into McGann’s “Marking Texts of Many Dimensions,” and Jerry and I spoke about our experiences of them last year, in response to a Scholars’ Lab talk on “N-dimensional Archives” by Julie Meloni. SpecLab participants called this (quite complicated) thing “the ‘Patacritical Demon.” It also stems from work on folksonomy which I undertook in designing the NINES/Collex software as a postdoc with McGann, and more recent Scholars’ Lab discussions about color-coded text visualization with Alison Booth, in the context of her project to define and mark narrative structures in biographies of women.

The concept I presented to the Praxis team last week as an inspiration for Prism is simpler in scope and beholden more to the material and pedagogical markup exercise than to text-theoretical debates. I’ll say no more here than that the original game involved shared, Xeroxed page images, transparent overlays, dry-erase markers, a common interpretive prompt, and a moment in which somebody yelled “Stop!” and the transparencies were stacked up for discussion.

Members of the Praxis team will be describing their vision for the user interface of Prism in more detail as the weeks and months progress. In our early conversations, the whole team has seemed energized by the potential of the tool for classroom use. But it’s important to say that we’re not just replicating an offline pedagogical exercise in the browser.

Prism updates the concept in some important – and we think timely – ways, some of which are meant as interventions in the current scene of DH project development and conceptualization:

• We recognize that there’s a huge vogue for “crowd-sourcing” in the digital humanities right now, but have been feeling like there’s potential for much more interesting work in this domain. We don’t want to treat the “crowd” only like robots or mechanical turks – asking for transcription labor, or refinement of OCR output, as valuable as those products may be. What would happen if we could systematize, capture, and build collective interpretive energy – on shared understandings and unexpected disagreements?

• We also feel ready to build on design lessons from citizen-science/citizen-scholar projects, like those created by the Zooniverse group, to create a DH tool that appeals to the general public and is easy and fun and effective for pedagogical use. We’d like to be able to use Prism as a laboratory exercise for thinking about design and development in the public humanities, and on the relation of audience and user communities to the questions we can ask in DH research.

• Finally, in an era of mass digitization, we’re keen to engage with big data in the humanities. Once the basic framework for Prism is established, we want to be able to experiment with the flow between user-friendly input and “easy” and attractive visualizations (like our spectra) and the deeper questions that can be asked and harder information design problems that are encountered when we move into computational linguistics & text mining techniques such as sentiment analysis.

This is a tall, tall order – but neither the Scholars’ Lab staff nor our Praxis students are the sort to be attracted to an unambitious project. We hope you’ll follow along this year as we see just how far we can get, and what we can learn along the way.