Auditing, Counting, and Tracking CanLit

One of my son’s favourite books when he was little was a picture book called Counting on Frank. The protagonist is a boy who loves to count. For example, he counts how many years it would take to fill a room with peas if he were to knock fifteen of them off the dinner table every night. Lately, I feel like I have been spending a lot of time counting peas. It is not so much that I love to count but that I am constantly being asked, or feeling compelled, to do so. Some days I spend more time in Excel and on Google Analytics than I do in Word. I have even begun to think in terms of tables, spreadsheets, and pie charts. This is quite a cognitive shift for someone who barely scraped through an undergrad statistics course.

In the last few months in particular I have been bombarded by the demand for numbers. Statistics are being used by, with, for, and against us. With Margery Fee and Donna Chin, I contributed to the impact section of Canadian Literature’s triennial SSHRC application (monitoring numbers of submissions, rejections, referees, subscriptions, and assessing web traffic, as well as the most accessed articles and reviews via EBSCO and ProQuest). I also used Google Analytics to track the number, behaviour, and location of visitors for our web-based teaching resource CanLit Guides. I produced an annual report for my department, a numeric snapshot of my research output and funding for an external departmental review, and filled out a SSHRC CV for two Insight Grant applications I was part of. I also ranked candidates on the job search committee I chaired and had to report to the university on the demographics of the applicant pool. I submitted grades for the classes I was teaching. Like many others, I read the annual Maclean’s Magazine University Rankings and noted my department’s fluctuating place on the World University Rankings. Further, I served on the board of directors for Canadian Women in the Literary Arts (CWILA) and helped manage the now-annual count of reviews. In March, as part of the CWILA Research Network at UBC, I participated in a panel entitled “CWILA and the Challenge of Counting for Race” with my colleague Mary Chapman and author Madeleine Thien (who shared her powerful and provocative “Seventeen Thoughts on the Question of Numbers”). Last week, alongside 59 other academics working in groups of three or four, I applied to job share the position of President/ Vice Chancellor of the University of Alberta as a communal gesture to protest the disparities between administrative salaries and those of faculty members and adjuncts in this time of “austerity measures.”¹ The other day, I read Rachel Rose’s brilliant poem “Thirteen Ways of Looking at CanLit.” For better and for worse, audit culture has hit Canadian literary studies.

Why is the turn to numbers noteworthy? The diverse sets of data I mention here illustrate the paradox at the intersection of audit and literary cultures. On the one hand, as part of the increasing corporatization of everything in these neoliberal times, people turn to numbers for proof of productivity and the value that can be monitored and measured annually. Anthropologist Cris Shore writes about how audit culture has transformed the university from “a place of higher learning into the modern idea of the university as a corporate enterprise whose primary concern is with market share, servicing the needs of commerce, maximizing economic return and investment, and gaining competitive advantage in the Global Knowledge Economy” (282). How does your research support the GKE or the GDP? The organizing work of institutional management and the quantification of labour in this budget model of education is often couched in the rhetoric of transparency, benchmarks, and global standards. With the system being driven along market lines, students and parents, as well as taxpayers and donors, become consumers seeking quality assurance.² However, corporate measurements of productivity often leave gaps around invisible labour.³ Some numbers are more valued than others. Still others are not noted at all. Shore draws on Michael Power’s argument that since the 1990s, Britain has been an “audit society” when he writes that an “audit society is one where people are interpolated as auditees” (281). The same can be said for Canada. I think that this suggests an initial answer to Thien’s question of “Why are people so afraid of numbers? Why are people so threatened by another way of looking?” (n. pag.). It is frightening to be hailed: “Hey you! I am tracking you!” because it is often followed by “you have to guess why.” In higher education, as in much contemporary society, we are being asked by various accountants to count so that we are accountable, but we always know that we might be asked to account for gaps if we account to those who discount our work or at least make us recount how we need to re-count our numbers for repeated proof of our accountability. A question persists: what counts for whom and why?

On the other hand, people have turned to statistics to strategically bolster support for issues of social justice and as ammunition for important cultural work. If information is power, there is a will to count. Recently, numbers have been used to provide evidence for arguments about disparities that have previously been dismissed as anecdotal.⁴ The more precise the measurement, the stronger the argument. People turn to numbers to rationalize the need for action. I dare say, they are instrumental to doing so. As Thien explains, “counting is one way of comparing one set of incidents against another. Counting can be seen as another way of telling a story” (n. pag.). When backed by statistics, adjectives like undeniable, irrefutable, and incontrovertible often prevail in narrating that story as they are tellingly placed before nouns like injustice and inequality. In my last editorial, I called for better metrics about graduate student placement for this very reason. The Dalhousie University professor who led the group applications for the U of A presidency, Kathy Cawsey, made a similar call for better cross-Canada comparative data on salaries in the fight against administrative bloating. The guiding principle behind such calls for statistics is that such numbers might potentially lead to the greater accountability of the university to its employees and constituents, and, even ideally, of the state to its citizens. Accountability should flow both ways. I don’t see this so much as subversion from within audit culture as harnessing the tools at hand to try to rattle its foundations and to put information to the good use of “another way” of telling a story.

Computer-generated numerical tracking also gives us new kinds of information and tells us stories to which we might not have previously had access. Statistics are being used to measure trends and behaviours, enabled by new technologies. Sometimes this is market driven and sometimes this is driven by a desire for further knowledge qua knowledge. I am thinking here of what Franco Moretti calls the “specific forms of knowledge” that come out of the “distant reading” of graphs, maps, and trees (1). But, as Moretti notes, there are limitations to seeking the internal shape of a cycle and the “hidden tempo” of a period, a text, or an event. He writes that “quantitative research provides a type of data which is ideally independent of interpretations . . . and that is of course also its limit: it provides data, not interpretation” (9). Here too, then, as with strategic counting, there is always the question of who is tracking what and to what end? We need to ask, how do we quantify responsibly? What are the risks of misinterpreting the numbers?

When might counting be both informative and potentially damaging? The SSHRC application I just helped to fill out is one place. As we were applying for the funding the journal requires to function, there was much at stake in this audit. There was also much to be learned about internal operations. In attempting to measure impact, an extraordinarily difficult task for a journal in the Humanities, we had to dig into the numbers that were available and try to read them productively. Some interesting stories emerged that tell us quite a bit about invisible labour, reading practices, and contemporary Canadian literary studies. A journal relies a great deal on work that remains anonymous and counts for little in academic reporting but which is essential to the production and dissemination of the publication.⁵ Here are some of the numbers we reported: our office consists of one editor and seven associate editors who work in two languages at three universities, as well as two full-time staff members and up to six undergraduate and graduate students. The editorial board consists of 36 specialists from six countries who read between one and six articles a year. Peer-evaluation reports by 226 referees were written on articles submitted to the journal.⁶ Between 1 May 2012 and 30 April 2014, we received 206 submissions and published 55 articles (by 11 graduate students, seven postdocs, seven assistant professors, 16 associates, 13 full professors, and 11 sessional instructors or freelance writers or librarians). In 2012 and 2013, we reviewed at total of 634 books and averaged parity on the number of reviews of books written by male authors and female authors, although female reviewers outnumbered male reviewers almost 2 to 1. Likely because most Canadian readers access via the web (library subscription services), most of our print subscribers come from outside Canada (64%) and almost all of them (95%) are institutional. Between 2012 and 2014, 70% of the 452, 278 visitors to canlit.ca accessed the website from within Canada. In 2013 alone, canlit.ca had visitors from 198 countries. The website had an average of 758 visitors a day or 18,112 a month in 2013-14. Our Facebook page has 323 likes and our Twitter account has 2,900 followers. According to EBSCO, one of our aggregators, in 2012, there were 193,506 downloads of our articles. The top article was downloaded 24,796 times, while the tenth article was downloaded 627 times.

We use information technology developed for business optimization for our non-commercial purposes as well. Google Analytics has allowed us to drill down into the behavior of our visitors. The information is irrelevant to SSHRC and says very little about our quantifiable impact but it tells us as researchers and editors a good deal about contemporary interests in the field. The reviews that were most often read online were about The Book of Negroes, Three Day Road, In Search of April Raintree (critical edition), No Great Mischief, and Traplines. Over three years, the most consistently accessed special issues were, #91 (on Timothy Findley), #124-25 (Native Writers and Canadian Writers) and #161-62 (on Thomas King). The special issues on Asian Canadian writing and diasporic writing were also accessed more often than almost any other issues, except the recent one on poetics (#210-211). The most often visited CanLit Poets pages are ones on David Solway, Daniel David Moses, Patrick Lane, David Zieroth, and Wangshu Dai. There is a link between these poets and the names plugged into the author searches where we find that Patrick Lane is most often sought, followed by Daniel David Moses, Cyril Dabydeen, Tom Wayman, Robert Kroetsch, Adam Dickinson, Uma Parameswaran, Rita Wong, Afra Kavanagh, and Marie Noëlle Ng. We can see in the title search drilldown that readers have looked most often for articles on Three Day Road; various works by Alice Munro (post-Nobel announcement), as well as Mavis Gallant and Farley Mowat (after their deaths); The Wars; The Apprenticeship of Duddy Kravitz; No Great Mischief; Jpod; and The Stone Angel. These results suggest to me that there is an incredibly strong interest in writing by Indigenous writers and Asian Canadian writers, that the work of male writers is being critically engaged more often than that of female writers, that our readers come with a diversity of interests, and that some older canonical texts are still being read (and likely taught).

We have also turned to various forms of counting for information about the CanLit Guides (CLG).⁷ In the fall of 2012, a few months after we first launched the open access online teaching guides, doctoral student Mike Borkent and I visited six undergraduate classrooms. We taught a unit from the guides and conducted surveys of the students after each class. The CLG team then held two follow-up focus groups with 20 keen students to figure out how exactly they were engaging with the resources. Such a survey, is of course, a form of counting. The information garnered from the focus groups changed how we looked at our own work. We enumerated responses and were surprised at how popular the practical/ skills/ writing pages were (especially the close readings section—where we walk the reader through a line-by-line reading and subsequent close reading of Eve Tihanyi’s poem “Blind Man”). Learning from the focus groups, we are adapting our content to include more hands-on activities that engage directly with the journal. Such changes rely on what Karen Correia Da Silva, as a member of the CLG team who worked on the surveys, views as the “feedback loop” possible in the digital humanities.

Besides consulting students for feedback on the guides, we also turned to Google Analytics tools for user statistics to try to gauge the project’s success. In the year after its launch, CanLit Guides had 18,701 visitors from 121 countries who looked at almost 75 thousand pages. Unsurprisingly, the most traffic came from Canada (14,705) and the US (1,335). The top ten traffic sources rounded out with many visits from India (340) and the UK (337), followed by Poland (251), Germany (260), then Australia, Spain, the Philippines, and Sri Lanka. We can tell that a reader in Bolivia spent 21 minutes on the site and a reader in Cuba visited 20 pages in 44 minutes. We can see how many people checked in from Wollongong, Westport, and Warsaw, and know how long they stayed (and I admit that such surveillance is creepy). Proving that the international visitors were not necessarily just nostalgic Anglo-Canadians abroad, the top 8 languages to which their browsers are set are American and British English, Mandarin from China, Polish, German, French, Spanish, and Taiwanese Chinese. We know that in 2012 people stayed on the site for an average of 4:30 minutes (or 2:14 for new users and 9:18 minutes for return users). We also know that 661 people read more than 20 pages of the guide in those early months. Judging from the dates when visit numbers spike, we suspect that some people used the guides in their classes. Finally, we know that the six most visited sections were, in this order, about The Jade Peony, the close reading section, Duncan Campbell Scott’s sonnet “The Onondaga Madonna,” E. Pauline Johnson’s bio-note, the history of nationalism in the 1960s and 70s, and the gender overview.

What do we learn from these numbers about the guides and the journal? First, we have a global readership and audience. We are not just writing for Canadians, for classrooms, or even for those who speak English as a first language. This has implications for Canadian studies and the international study of CanLit, particularly given Canadian government cuts to international Canadian Studies programs. We know that Canadian Literature is being read carefully in Canada, Poland, Germany, India, and many dozens of other locations. We have learned that our readers are not necessarily students and teachers in local classrooms but scholars and interested readers around the world.

Second, we were struck by the popularity of some pages over others. We love The Jade Peony, but were also surprised to see just how popular it is with our visitors. Thousands of people have visited that case study. When we started the project, we surveyed instructors on the works they taught most and we also considered which works had received coverage in the journal. Now, perhaps we’ll consider a special issue on the work of Wayson Choy and think about how to better integrate our two special issues on Asian Canadian writing (#163 and #199) into the guides. The counting we have done on the guides and the journal will likely have an impact on future directions of both. In turn, perhaps in the future the guides and the research published in the journal might have an impact on what instructors choose to teach in their CanLit classrooms.

When Cynthia Sugars and I created our anthology, Canadian Literature in English: Texts and Contexts, our publisher sought feedback from 49 experts (seven readers per chapter). We worked with their comments, advice, and suggestions. Since publication, however, although we know how many copies have been sold (we get royalty statements that give us those numbers), we don’t know which sections in each volume are most taught or most read. Because we can’t do a Google Analytics visit breakdown for a paper book, we don’t know which ones are most often accessed. And yet Google Analytics data itself is limited as it would be unable to tell us which selections are most appreciated by students and which ones work best for teachers. The data cannot tell us that a story has had a profound impact on a student, as I have been told after teaching both Thien’s “Simple Recipes” and Alistair MacLeod’s “The Boat.”

There is little doubt that numbers are being used against us in the humanities as our worth is measured out in the coffee spoons of audit culture. While we need to remain skeptical about the motivations behind the burgeoning number of requests for accountability, I want to guard against being too skeptical about the potential of counting. Numbers can be also used strategically and they can lead us to important stories. There is a potential problem, though, when numbers stand in for the story. We should also be aware of the kinds of stories that might be overlooked in larger trend spotting. I hold on to the value of studying something that might not be “statistically significant” but might still be absolutely socially or creatively relevant. An article can have great impact if it really changes the way a dozen people think about an issue. I am not convinced that an article with 3,000 downloads will have more lasting impact than one with 50 downloads. While I am curious as to what many people think, I am wary of having research driven by majority interest or by appealing to audit culture. Further, as literary researchers, we need to recognize that we might not all be fully equipped for all manner of interpretation. This is where what Danielle Fuller calls “collaborative interdisciplinary work” is necessary (75). We either need to collaborate with those properly trained in quantitative analysis or we need to learn new methodologies ourselves, indeed as some digital humanities scholars have done. If we decolonize quantitative methodologies and open numbers to a variety of interpretations, I suspect that we can productively engage in meaningful qualitative thinking. What makes numbers meaningful for a literary scholar is how we interpret them, how we use them to make plausible narratives and to support ideas. It is really how we read and speak them. Tracking CanLit, either by choice or by necessity, can help connect reading practices with literary production and elucidate how texts are valued and circulated. It does not, however, finish the story. We need to count our peas, but we also need to eat them if we want to grow resilient in action and imagination.

Acknowledgements

I want to thank to Mary Chapman for acting as a sounding board for this paper and pointing me to the idea of invisible labour, Brendan McCormack for pushing me to think about accountability and instrumentality, Margery Fee for her timely editorial advice and mentorship, Beth Veitch for ushering through all the submissions over the past year, Donna Chin for managing it all so ably, and Zoya Mirzaghitova for catching all the mistakes with such lovely humour.

Notes

1. None of the groups to apply got an interview but together the 60 academics sparked international discussion of increasing administrative bloating (including media articles in Canada, Australia, Britain, and even a forum in the New York Times).
2. In his comments on a draft of this editorial, Brendan McCormack usefully responded: “It’s not just a question of how your research supports GKE or GDP, but whether your curricula and pedagogy can maintain market share. In discussing our most recent teaching evaluations, a colleague and I noted how closely both the metrics of evaluation and the substantive feedback from students rehearse the rhetoric of an Amazon.com product/purchase review. The central question implied isn’t ’how much did you learn’ or ’have you become a better critical thinker,’ but ’would you buy this course again?’”
3. Communication management is an excellent example of invisible labour. Look into the emailbox of an associate professor at, say, UBC and a you might find 648 email messages sitting in her inbox or 2208 messages accrued in one year concerning the editing of a journal or 4266 messages about serving on the board of a national literary organization over a couple of years or 4488 messages about various research projects over the past decade. You might find 848 messages about writing letters of reference, 1819 messages about graduate students in her department, 486 messages about her first year class from last term, and 571 in a file somewhat terrifyingly marked Misc. Admin. Look into another academic mailbox and the numbers will be higher in some areas and lower in others.
4. This is the guiding principle behind CWILA but it extends to many areas of social justice, broadly defined. Recently, grade five students at Shorewood Hills Elementary in Madison, Wisconsin counted over 600 Lego sets looking at gender and cultural diversity. They looked at Lego figurines and found that 75% were “boys,” 12% were “girls,” 5% animals, 8% aliens. They noted too that in older “classic” sets, 58% were male and 42% were female. They also noted a lack of racial diversity when they found that 94.3% of the figures were those they classified as “European.” After they tracked the disparities, the children wrote to the Lego Company and demanded more balanced gender and racial representation. (see www.WhatItIsIsBeautiful.com).
5. In her excellent article on “Editing as Carework,” Sarah Blackwood notes that “[i]t will be difficult to articulate the editorial labor I put into Avidly within the structure of the standardized tenure file, which is interested mainly in my voice, heard singly and forcefully within the strictures of liberal individualist understandings of authorship. I think this conundrum of how to frame editorial work is also true for even those endeavors more ’inside’ the academy: peer-reviewed journals, edited collections of essays. Excellent editing erases itself: it’s mending the dress so well that the fit is perfect, and the holes are invisible.”
6. It is notable that in this culture of ’just say no,’ we tend to have to ask two or three people for every one acceptance of a request to serve as a referee and even more to write a review. For some articles the number goes as high as a dozen before we find a qualified reviewer willing to serve.
7. Karen Correia Da Silva and I gave a preliminary version of the CanLit Guides data I present here in Metacriticism, Digital Humanities, and Teaching Canadian Literature to Digital Natives (Part One) at the Association of Canadian Quebec Languages (ACQL) Conference, University of Victoria, June 2013.

Works Cited

• Blackwood, Sarah. Editing as Carework: The Gendered Labor of Public Intellectuals.Avidly: A Los Angeles Review of Books Channel. 6 June 2014. Web. 17 June 2014.
• Cawsey, Kathy. University of Alberta President / Vice-Chancellor Application Group. Facebook. Status Update. 10 June 2014. Web. 25 June 2014.
• Clement, Rod. Counting on Frank. Sydney: Collins and Anne Ingram, 1990. Print.
• CWILA and the Challenge of Counting for Race. Institute for Gender, Race, Sexuality, and Social Justice, University of British Columbia, Vancouver. 19 March 2014. Address.
• Fuller, Danielle. “‘Beyond CanLit(e)’: Reading. Interdisciplinarity. Transatlantically.” Shifting the Ground of Canadian Literary Studies. Ed. Smaro Kamboureli and Robert Zacharias. Waterloo: Wilfrid Laurier UP, 2012. Print.
• Moretti, Franco. Graphs, Maps, Trees: Abstract Models for a Literary History. London: Verso. 2007. Print.
• Shore, Cris. “Audit Culture and Illiberal Governance: Universities and the Politics of Accountability.” Anthropological Theory 8 (2008): 278-298. Print.
• Thien, Madeleine. “Seventeen Thoughts on the Question of Numbers.” CWILA and the Challenge of Counting for Race. Institute for Gender, Race, Sexuality, and Social Justice, University of British Columbia, Vancouver. 19 March 2014. Address.