From MOOCs to Learning Analytics: Scratching the surface of the 'visual'

The two most prominent trends in education technology, for the moment, appear to be MOOCs and data analytics. While MOOCs are frequently accompanied by references to "disruptive innovation" [1], so-called "big data" in education, or "learning analytics" as it is often termed [2], is also cited in lists of imminent educational trends [3]. Elsewhere, and rather optimistically, the two developments in tandem are claimed to be able to "reverse engineer the human brain" [4]. MOOCs and data analytics seem well suited to one another; more data about student behaviors and activities would appear to suggest greater accuracy in prediction and personalization, and the huge enrollment numbers in MOOCs (see [5]) might then hold such a promise [6].

However, the relationship between these apparent trends is much more profound, and resides at the heart of the way technology is predominantly understood in education. Just like the MOOC, learning analytics promises a technological fix to the long-standing problems of education. Instead of web technology that supposedly dissolves economic and geographical barriers to the elite educational institution, this time it is computational routines that vow to reveal unprecedented insights about the learning process. The invisibility of the technology itself is common in both these ideas, and an emphasis on the outcome rather than the means through which they are produced. This emphasis means when we consider MOOCs and data analytics, we tend to look at what is made visible by the technology, not necessarily all facets of the technology itself. In this article I want to focus on two very obvious things we "see" in relation to MOOCs and learning analytics—the MOOC video and the data visualization—and discuss what is it that we are not seeing?

The 'Materiality' of the Video

The prominent MOOC platform providers—Coursera, edX and Udacity—are centered on the video lecture, a technology that seems to provide a transparent window to prestigious universities. Yet, for all its visibility and it's supposed uncovering of "real human" teachers in online education [7], the video performs a remarkable concealment of the technologies that produce it.

The underlying technology required to make a Harvard or Yale lecturer appear at will across the globe is not really intended to be part of the MOOC story. We are not supposed to see, experience, or really engage with the vast amounts of Internet infrastructure that carries the video lecture, or the software codes and algorithms, that make its transportation possible. Indeed, the promotional narrative would seem to have us believe the Internet is uniform, accessible anywhere, by anyone, and without regional inconsistencies or local challenges. What we are supposed to focus on is the lecture itself, and indeed studies have described the development of intimate relationships between students and their video-professors [8]. While that may indeed be advantageous in some ways to the learning process, it diverts attention away from the contingent processes that have produced the video; which is itself "just the visible surface of a large realm of software, a complex amalgam of data structures, algorithms, packages, [and] protocols" [9]. Rather than simply watching the video lecture, I think important educational insights can be gained from looking beneath the surface.

A somewhat obvious example, might be the algorithms required to (re)produce the video image. There would be nothing new in suggesting a need to teach coding, so that technology "users" might understand the software that underpins their digitally-infused lives, or the videos that appear during their MOOC experiences. In the UK, learning to code is firmly on the political agenda, and decisively in the national curriculum, with Australia perhaps set to follow suit. However, as Williamson insightfully shows, it is not just learning to code that is important, but the ideological assumptions already engrained in how we go about teaching it [10]. Code For Australia seems to encapsulate this "functionalist and technicist" [10] way of thinking perfectly in their three-step process: 1. problem, 2. idea, and 3. solution. Learning about the algorithms that might compress and decompress a MOOC video is therefore bound up in a powerful discourse of "solutionism" that views the world largely as a complex set of conundrums, for which computation provides the step-by-step remedy [11]. So why when we teach coding, do we also have to teach the idea that age-old social problems, let alone educational ones, can be eradicated with a few algorithms and a user-friendly interface? Alongside exploring the significant benefits of learning to code, I think we also need to ask questions about why the tech industry is largely contained within a particular geographical location with increasing claims of discriminatory employment practices [12], ageism [13], and social inequality [14]. These are examples of the very real material conditions that lie beneath the slick facade of technologies like video streaming, yet they don't seem to be issues worthy of inclusion in the ways we educate about technology. A further striking example is the power consumption of large data centers and their significant role in local pollution [15]).

While not exclusively or directly related to MOOCs, this is precisely the kind of material, real-world condition that is required for the image of a renowned professor to be broadcast across the globe. Far from being a benign "cloud," the servers required to host global education projects like the MOOC require actual locations, physical storage disks, and huge amounts of power, yet none of this materiality is "made visible" by video technology. Alongside providing free, online educational content, I think we also need to consider the hardware infrastructures that underpin and enable such broadcasts; how they manifest locally to provide what kind of access.

My point here is of course not to suggest that a lecture about "Modern and Contemporary American Poetry" should somehow seek to tackle the environmental issues raised by data storage, or indeed the morals of Silicon Valley, rather it is to highlight our cultural perceptions of technology, and how they might limit the ways we can understand the complexities of an education increasingly pervaded by the digital. Digital "tools" may be integral to many educational tasks, but at what point should the pedagogy turn to focus on the tools themselves, as a way of fostering inquisitiveness, curiosity, and critical awareness? ould we really want an educational system where students used digital tools, but didn't really think about how those tools were designed and produced, or how they influenced or limited the kind of tasks that could be performed with them? Clearly the content of a video lecture is important to educationalists and students alike, but what might also be worthy of consideration are the complex factors that comprise digital video itself, as a material artifact, not just the pictures we see on the screen.

What Does That Have to do With Learning Analytics?

The analogy of the video lecture is important here because it signals the tendency to focus on the surface image, and disregard the processes that produce it. Indeed, it is visualization that I want to focus on here, a facet of the data analytics process that appears to be indispensable to the burgeoning field. Just as the MOOC video is implied to make learning more accessible, so "visualizations, diagrams, charts, tables, infographics and other forms of representation …make education intelligible to a wide variety of audiences" [16]. Indeed, continuing the visual metaphor, learning analytics is claimed to be "essential for penetrating the fog that has settled over much of higher education" [17]. However, I think we need to carefully consider this propensity to visualize educational data, and the supposed authenticity of "seeing" user behavior. This is because a graphic representation, not only diverts attention away from the algorithmic procedures that have collated and analyzed the data, it also seems to mask its own condition as a constructed artifact, produced through processes of selecting, categorizing and arranging the data. In other words, visualization doesn't get us closer to the authentic, "true" state of the data; more accurately it adds a layer of analysis and interpretation. As Williams suggests, "[t]he capacity to mobilise data graphically as visualizations and representations, or 'database aesthetics,' amplifies the rhetorical, argumentative and persuasive function of data" [18]. In short, the visualization is created, not to reveal reality, but to tell the story the analysts want to tell.

Emerging research associated with Coursera and edX conforms to the visualization trend in analytics, producing customary "heat maps" of global MOOC participation [19, 20, 21, 22]. Here various nations are colored according to enrollment numbers derived from IP address locations, ostensibly depicting "where" MOOC students are located. However, when we consider the complex methods of IP address location, we begin to get a sense, not only of the multi-layered procedures of data collection and analysis involved, but also of the inconsistencies in global Internet infrastructure. There is a reason why the African continent is largely left blank in these world visualizations, but a graphical representation of enrollment statistics isn't really going to provide us with much of an explanation, or indeed any indication about what one might do about it. Rather, the story to tell here appears to be one of global expansion, highlighting areas of corporate influence and, significantly, identifying those areas requiring it. This seems to be a salient example of the way "powerful visualizations are now being deployed to envision and diagrammatize the educational landscape 'out there,' and to make it amenable to having things done to it" [16].

As if a commitment to visualization were not enough, we are also encouraged to understand the field of learning analytics through the now de rigueur "infographic," which of course reveals much broader deployment opportunities than the humble MOOC. The prominence of competency maps, or the "traffic light" systems, exemplified by Course Signals, in mainstream, on campus programs are testament to the appeal of data analytics in education, and the slick visual layers through which we see them. Using such systems, we are supposed to focus on the shape of the graph or the color of the display, not worry about the workings underneath. But who decides which data points are the most valuable to measure, and who is choosing the shape and hue of the graphic, and to what rationale?

The point here is not to question the precision or accuracy of the algorithmic processes at work here, or indeed the valuable work of those undertaking such projects. Rather the purpose is to highlight how visualization deflects attention from questions of process and method, and posits the surface image as incontrovertible truth. Indeed, that is precisely the point of data visualization; that we don't need to look at the data itself, just the much more understandable graphic that explains it. Significantly, data scientists working in broader fields are beginning to question the blind acceptance of their visualizations by the mass media and the public [23]. Burn-Murdoch situates the problem firmly in the visualization layer, not necessarily the data science underneath. Furthermore, our faith in visualization is identified precisely in educational practices: "While text is frequently presented to students for critique, diagrams and data visualisations are overwhelmingly used simply as a medium of displaying final results" [24]. So, are graphs, charts, and diagrams engrained in us as the incontrovertible truth of data, and if so how can we overcome that and develop a more critical engagement?

Ferguson recommends, two ways forward for learning analytics research, involving the inclusion of learners' perspectives and clear ethical guidelines [25]. Both of these directions require openness and clarity; not in the form of more surface visuals, but rather as a practice of co-involvement and co-development. This means not only acknowledging the routines and procedures beneath the visualization, but also encouraging educational communities to take an active role in influencing them. This would require the technology of learning analytics to be, not invisible, but "transparent, enabling learners to respond with feedback that can be used to refine the analytics, and enabling them to see how their data are being used" [25]. From such a perspective, engaging in learning analytics becomes educational in itself, as students actively involve themselves in the production of data, and gain awareness of how it generates knowledge about them. This might be one way in which we might move beyond an obsession with the image as an unquestionable representation of reality, and toward an active practice in which we recognize the mutability of visualization, and our own involvement in it.

References

[1]Yuan, L. and Powell, S. MOOCs and disruptive innovation: Implications for higher education. eLearning Papers 33 (May 2013).

About the Author

Jeremy Knox is lecturer in digital education at the University of Edinburgh. He teaches on the M.Sc.in Digital Education program, and is a member of the team behind the "E-learning and Digital Cultures" MOOC in partnership with Coursera.