ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

5 (really) hard things about using the internet in higher education

By Grandon Gill / March 2006

Print Email
Comments Instapaper

The companion piece to this article, "13 (Educational) Things I'd Rather Do Over the Internet" responded to a recent article in eLearn's sister publication, Communications of the ACM (CACM), that had expressed considerable misgivings regarding the educational value of Internet-based education [6]. The focus of that piece, as suggested by its title, was on identifying ways in which the Internet could be applied to increase educational effectiveness. Unfortunately, this narrow objective meant it could easily be interpreted as an unqualified endorsement of Internet-based education. That was not its intent, however. To the contrary, using the Internet as an educational tool can be really hard—and it is not clear that things will get much easier in the near term. This article identifies five particular things that have particularly vexed my colleagues and me when using the Internet in our undergraduate and graduate classes.

FIVE ITEMS

The list of items being presented derives primarily from my experiences, and those of my colleagues, in teaching both online and blended courses in the MIS area over the past six years (see "Table 1" of the companion piece). It also includes observations made in the process of developing case studies of classes—across many disciplines, at both graduate and undergraduate levels—for an annual summer institute conducted by the University of South Florida's Center for 21st Century Teaching Excellence (C21TE). What the list attempts to do is to distill what has been hardest for us—and, I suspect, will be hardest for other faculty—in moving class content online.

1. Lack of models from our own experience
If you believe that we learn from experience, then many of us come to Internet-enabled education woefully unprepared. Unless you graduated in the past decade—five years is probably more representative for most schools—you probably have not had the chance to experience an Internet-based course as a student. Thus, when we decide to incorporate the Internet in our own courses, we do so without the broad base of exposure to what works and what doesn't that we have for other approaches—such as lectures.

I believe that the importance of such exposure cannot be overstated. One of the most interesting things about the CACM article mentioned in the introduction was its relentless focus on contrasting "Internet-education" to classroom lecture (perhaps augmented by a group project). Making the comparison in these terms is unsurprising. If you've followed a typical path to academia, by the time you receive a master's degree, you have likely taken 50 to100 courses following the lecture/project model—well over 1000 individual lectures—starting as early as middle school. Such a set of observations provides us with an ample basis for inducing a great many things about effective lecturing—e.g., how to incorporate slides into lectures, with what lecturing style are we most comfortable, how to interpret student verbal and non-verbal cues during lectures. It also means that "the lecture" tends to become default standard for delivering education—always safe, never controversial. Interestingly, we make lecture "the standard" in spite of a long history of studies showing that lecturing produces, at best, a short term benefit in factual recall over other teaching approaches (e.g., discussion) and generally falls short of other methods when it comes to comprehension [7].

Individual faculty members may also have exposure to a number of alternative pedagogies, depending both on their discipline (e.g., for medicine and law, the "default" approach to graduate education is not based upon lectures) and their life experiences. In my own case, I did my MBA at an institution that was extremely devoted to the case method—meaning that the program consisted of roughly 800 case discussions and, perhaps, a dozen lectures. I also spend five years in the submarine force, which employs a radically different peer-centered pedagogy. While these experiences definitely increased the range of possible online activities I was willing to contemplate [2, 4], they did little to prepare me for the qualitative differences I experienced in online teaching.

So why is teaching without experiential models so difficult? In essence, it makes every new thing you try an experiment (rather than being a replication of an experiment in which you've already been a participant). This experimental character of Internet-based learning has a number of important implications for us, as instructors. First, it means that we must constantly be acquiring data to assess our effectiveness. Regrettably, such data gathering is not necessarily easy, particularly for courses consisting mainly of online activities. For example, at my institution (the University of South Florida) we are not allowed to use online end-of-semester evaluation forms unless absolutely no other means is feasible owing to the abysmal response rate (~20 percent) we experience when online instruments are used. Naturally, it is possible to increase response rates—we've gotten up to 70 percent of active students by offering substantial extra credit for submitting the course survey. Unfortunately, doing so typically involves rewarding data-gathering activities that are not part of the course objectives or punishing students for failure to engage in activities that are supposed to be voluntary.

The second challenge of the experimental nature of online activities comes from interpreting the results. In a lecture environment, most faculty members seem content to assess success based on course performance (e.g., quality of exams and papers) and perceived student satisfaction (e.g., course evaluations). If we've taught for a while, however, we have also come to realize that a considerable amount of variation in the measures, from semester-to-semester and even from section-to-section, is inevitable. Where a course introduces substantial online content, on the other hand, we do not have the same reservoir of experience in class-to-class volatility as students (i.e., good lecturers vs. less good lecturers) and as teachers (e.g., good evaluations vs. less good evaluations for the same course). As a result, it is much easier for us to be swayed by individual outcomes.

A final implication of the experimental nature of online teaching is the difficulty of interpreting the perceptions our students. For example, in the CACM article it was reported that "74 percent of students in the Internet class believed they "missed out" educationally because they took an Internet class" [6] and that they judged the best Internet courses to be "those that are tightly structured around one textbook and having no group work" (p. 101). Both of these attitudes seem to suggest a preference for the status quo; a familiar course design that they have already learned to work with. The challenge this presents to instructors is determining what portion of observed resistance is a consequence of novelty, what portion is a result of the particular course structure employed, and what portion is inherent to Internet-enabled learning. Unfortunately, course evaluation instruments are rarely sophisticated enough to be useful in making such distinctions.

2. Constant disruptions precipitated by evolving technologies
The second hard thing involves continually adapting to technological change. The particular challenge here is that Internet-based education tools have yet to stabilize and it seems as if it may be a long time before they do so.

What do we mean by a technology "stabilizing?" If you look over the history of computing, major technology introductions have tended to be accompanied by periods of rapid innovation, followed by periods where subsequent innovations were mainly incremental, and not nearly so disruptive. We see this pattern in both hardware and software. Every new mainframe model rendered previous models obsolete until IBM released its 360 series—with its novel notion of "upward compatibility"—in the late 1960s, after which the market converged on IBM as its de facto standard. Microcomputer operating systems (OS) went through a painful series of disruptive events (e.g., incompatibilities) in their first 15 years as they transitioned from the 8-bit, standalone, single user, limited-scope, text-based OS (e.g., CP/M and DOS) to the 32-bit, graphical, networked, Internet-aware OS (e.g., Windows 95, NT). In the decade that followed, however, changes were much kinder and gentler—at least, from the user's perspective. End-user software has followed the same pattern—starting as a collection of discrete applications (e.g., word processors, presentation software, spreadsheets, desktop databases) with vastly different interfaces in the early 1980s and evolving into the integrated office suite of the mid-1990s, with subsequent enhancements occurring largely around the periphery of user needs.

In the field of Internet-based education, there is evidence that we are moving towards a period of stability but that such stability may yet be five to ten years away. On the plus side, some key capabilities—such as email, text chat, discussion groups, file exchange, and HTML—have started to mature and, equally important, have become integrated into course management systems (CMS), such as Blackboard and WebCT. These developments parallel the integration of individual tools into office suites (e.g., MS Office, Wordperfect Office) that occurred in the early 1990s. We have also seen the emergence of a common format for communications (XML) and introduction of standards directly relating to certain forms of teaching materials (e.g., SCORM) that make it easier to develop content that is less platform-specific. Despite these advances, however, we have yet to reach complete stability in this area. There are still many CMS vendors (including a number of open source entrants) and compatibility between them is limited. As a consequence of this, many institutions find themselves rolling the dice and creating a portal system tied to a particular vendor. What happens if the vendor should fail is, for the most part, an unspoken concern once the choice has been made.

On the disruptive side, it is also clear that a number of technologies that are highly relevant to education have yet to be integrated. At USF, for example, we recently customized our Blackboard system to incorporate Web log and wiki functionality that was requested by faculty members. We are also experimenting with implementing Web services (a tool that allows external programs to access designated functionalities within the system) in Blackboard to allow individuals—such as a group I am working with—to develop functionality not supported within the CMS itself. The obvious drawback to these activities is what will transpire when some other version of the functionality later becomes integrated into Blackboard by the vendor. At that point, we'll have two (presumably incompatible) ways of doing the same thing. This type of problem is inevitable when working with technologies that have not yet stabilized, sometimes characterized as the "bleeding edge."

Based on our experiences, some of the most crucial technologies with the potential to impact education are still in their infant stages. Particularly promising in this regard are technologies for the creation of multimedia content and for synchronous conferencing. Using tools such as Techsmith's Camtasia Studio, it has become a relatively trivial matter to create nicely packaged multimedia versions of MS-PowerPoint lectures that can be viewed at nearly any connection speed [5]. It is much more time consuming and technically demanding, on the other hand, to incorporate animation and interactivity into these presentations, entailing the mastery of complex tools such as Macromedia Studio. We anticipate that considerable simplification in the area of content development is likely to occur over the next five years or so (much as it has done over the past five years) and that capabilities of hardware particularly well suited for content creation (such as the Tablet PC) will be increasingly supported.

Internet-based synchronous classrooms may be the technological area where the greatest advances are likely over the next decade. Although online conferencing applications for business (e.g., NetMeeting) and recreational use (e.g., PalTalk) have been around for a long time, their use in education has been hampered by cost, reliability, participation limits and prohibitive bandwidth requirements (in a dialup world). In addition, it was not always clear how online classroom needs differed from business meeting needs, creating a design obstacle. In the past few years, however, a number of synchronous products specifically designed for classroom use—such as Elluminate—have been introduced. As it stands, these products tend to be very expensive, not entirely reliable, somewhat awkward to use (e.g., half duplex), non-trivial to learn and prone to many outside sources of interference, such as firewalls and security [8]. Nonetheless, they represent a very promising first step—as noted in the companion piece.

If you are willing to venture online but unwilling to risk becoming the victim of disruptive technological changes, there are two obvious ways to proceed. First, you can stay within the relatively stable umbrella offered by your institution's CMS—distributing documents to students, creating a few discussion groups and limiting other uses to activities such as entering grades. Second, you could—as the CACM article proposes—share course responsibilities with faculty or curriculum developers who are more attuned to online instruction, allowing them to take the risks while you take responsibility for classroom activities, such as lecturing. Unfortunately, both of these options have serious limitations. Taking the most conservative path to using the Internet, you tend to be limited to the few capabilities that are truly mature—distributing static content and providing discussion board support. Exploiting only these capabilities, you will likely find that either the Internet is either playing a very limited role in your classes or you are getting negative student reactions similar to those described in the CACM article. On the other hand, to proceed down the path of partnering with others can entail considerable loss of control over the contents of your course. For many faculty members—including me, I'm ashamed to admit—that can be a very difficult loss to accept.

3. Explaining our courses to others
The flexibility offered by technology means that Internet-enabled courses can (and often do) turn out looking very different from their traditional counterparts. That means that we, as instructors, not only have to worry about conveying course content, we also have to have to devote a considerable amount of time to explaining our designs and justifying our outcomes. Furthermore, it is not just the students that we have to worry about. Other faculty and administrators may also need to be educated in the nature of our courses if we are to avoid unfortunate consequences.

The need to educate students regarding the requirements and processes of Internet-based courses is, to a great extent, likely to depend upon their familiarity with other online courses. In the case of the programming course we teach, for example, most of our students have never taken an online course before. Over time, we have concluded that it is almost impossible to devote too much effort to ensuring they understand the nature of the course early in the semester. Among the steps we have taken in this regard are included:

  • a 75-minute lecture on the nature of the course given at their first (and only) classroom meeting—with a parallel version made available on the course Web site
  • assigning each student to a mentor teaching assistant, and requiring they meet in the first week
  • providing a "general questions" discussion group on Blackboard specifically devoted to course procedure issues
  • providing students with a multimedia roadmap to the course (that can be accessed through Blackboard) offering detailed practical instructions on getting started in the course and suggestions for completing each course assignment
  • requiring students sign up for a ten-minute, face-to-face interview with the instructor during the first three weeks of the semester, and
  • offering participation credit for "checking in" each week, so we can track each student's progress (the "progress monitoring" systems described in the companion article).

Since we instituted these requirements, course completion rates have increased from under 50 percent—about the same as lecture versions of this demanding course—to over 70 percent. Nonetheless, it would be nice if some of this "process education" time could be devoted to actual course content.

Another group that needs to be educated with respect to Internet-based techniques is other faculty members. In our experiences with the programming course, we've seen this challenge manifest itself in two ways. First, faculty can be confused by changes in student strengths and weaknesses brought about by online techniques. For example, a few semesters ago a faculty member teaching an intermediate programming course that followed our introductory course complained that the students seemed to do very well on projects, but scored very poorly on their first exam. We found his assessment somewhat disappointing but not altogether surprising. The source of the problem seemed to be that the professor employed a highly traditional "lecture and test" course design, while the students had been prepared using a self-paced design where group work was emphasized and traditional midterms and final exams had been replaced by assignment-specific validations (such as oral examinations, administered face-to-face or online). In other words, our students lacked any experience taking traditional programming exams. The question then becomes: what approach to assessment provides the more reliable indicator of student learning? Unfortunately, questions such as this do not lend themselves to easy answers.

A further challenge in educating faculty with respect to what we are doing stems from motivation. In MIS, my own field, rewards and recognition stem almost entirely from the publication of scholarly research [1]—a complaint voiced to me by faculty in many other disciplines as well. Given such a reward system, it is hard to see why faculty members should want to leap enthusiastically into developing creative ways to apply the Internet to their educational needs. Indeed, cultivating ignorance or indifference may be the best defense against being drawn into the time-consuming process of developing the skills needed to be an effective online instructor.

The final group that can be hard to educate with respect to online education is administrators. The challenge here is that there is a widespread perception within higher education that online courses are, in some way, inherently inferior—the province of "diploma mills" [10]—and that they should also be much less costly to deliver than traditional "brick-and-mortar" classes. These themes permeated the CACM article, worrying about student expectations that they "should be able to earn a top grade almost irrespective of effort and ability" [6] and observing that cost seemed to be a principal motivator for such courses.

In another eLearn article [3], I previously explored the question of distance learning quality and cost. To summarize, the reality is that online education comes in many forms, and that costs and quality can vary dramatically according to one's educational objectives. Unfortunately, this can mean that a faculty member may face the unenviable task of both justifying the resource requirements for his or her design to an administrator and explaining why such requirements are so much greater than the administrator's original expectations.

4. Adjusting to a new rhythm of life
Teaching in the traditional classroom setting allows the faculty member to establish a very pleasant routine. Every week, he or she shows up at specific places at designated times, with some additional time scheduled for office hours (often sparsely attended). Beyond that, the faculty member has considerable control over his or her activities. In other words, the synchronous world of classroom teaching offers a perfect balance of structure and freedom.

For the time being, at least, courses designed to make heavy use of the Internet nearly always rely far more on asynchronous elements than their classroom counterparts. The reason for this relates to the comparative strengths of classroom versus Internet techniques. Even today, it is clearly possible to deliver all content synchronously over the Internet—e.g., using live webcasts to deliver lectures, supplemented by synchronous chat (text and/or voice). What is less clear, however, is how—beyond pure convenience—a student could benefit from such an approach over one consisting of face-to-face lectures and classroom discussions. In fact, with a few exceptions (e.g., application sharing and facilitating external participation in class sessions, as noted in the companion piece) online versions of synchronous activities tend to be lesser substitutes for their face-to-face equivalents. For asynchronous activities, on the other hand, the situation is reversed. The Internet enables many activities that are very hard to do in the traditional lecture format (e.g., centralized question and answers, practice testing, content on demand, content accumulation, distribution/exchange of materials, and self-paced learning—just some of the examples in the companion piece). Thus, it makes sense that use of the Internet will generally be accompanied by increased reliance on these types of activities.

The transition to asynchronous learning forms can require considerable adjustment sto a faculty member's day-to-day schedule. Activities such as checking email and online discussion groups must be conducted with far greater frequency. When demands for assistance are made, they need to be acknowledged and addressed promptly. If response times become too long, students may resort to more intrusive technologies (e.g., cell phones, instant messaging). Even worse, they may simply abandon the class—another all-too-common phenomenon in online venues.

One particularly interesting aspect of this changing rhythm may be a change in what students' perceive to be "instructor availability." By way of illustration, Table 1 compares a variety of metrics for the programming class, as given in spring 2005 and summer 2005 (during which time course content and requirements were virtually identical). As shown in the table, for the last seven weeks of the ten-week summer 2005 semester, I was traveling to a variety of conferences and workshops. As a result, face-to-face contact was impossible and Blackboard/email could only be checked in my hotel room (or lobby), twice per day—I made sure every hotel I traveled to had broadband, another concession to online learning. Despite this fact, the score on the "instructor accessibility" metric rose dramatically for the summer—and was the highest score received of all seven evaluation categories that summer. Any plausible explanation for this anomaly has to include the fact that I posted my cellular phone number on the main screen of the Blackboard Web site once I started traveling (since my home and office numbers were no longer relevant). Interestingly enough, over the seven-week period when I was gone, I received only seven calls—four of which were from the same person. But, somehow, the very act of providing a cell number conveys availability to today's students. Now I find myself wondering if I should further disrupt my life's rhythm by carrying my cell phone with me even when I'm not traveling and, perhaps, even (gasp!) adding text messaging service.

5. Adjusting to our new role
The rather flip characterization of the faculty role in classroom versus online learning is "sage on the stage versus guide on the side." In reviewing the literature on the subject, researchers have found that students strongly believe in a reduced instructor role online, while faculty members were divided on the subject [10]. Questions have also been raised relating to whether or not such distinctions are more a matter of style than of delivery technology.

In our own classes, my colleagues and I have found that the nature of the instructor's role depends on the subtle interplay between delivery mode (synchronous vs. asynchronous) and pedagogy (e.g., lecture vs. discussion). At one extreme, where a lecture is broadcast to a class synchronously and students are allowed to ask questions, there is little reason to believe the instructor's "sage" role is jeopardized in any way. Even when lecture content is delivered asynchronously, via videotape or the Internet, that role may be preserved. As an example, the economics analyst for the Economics U$A telecourse [3] still receives fan mail from high school students watching the course on video tape—over 20 years after the segments were produced.

Where a discussion-focused pedagogy is employed, on the other hand, the guide role is generally mandated for the instructor—whether it be online or in the classroom. We have found this to be particularly true in the case of asynchronous case discussions, where an instructor's opinion on a subject revealed too soon can lead to an immediate convergence of subsequent discussion [2]. Similarly, the support role engendered by activities such as online Q&A and creating content on demand tend to increase the perception that the faculty member is a service provider, rather than a leader. This can be particularly true if such duties are shared between the instructor and teaching assistants.

Given, then, our previous prediction that Internet-based education will tend to emphasize asynchronous approaches, combined with the widespread belief that effective online education will emphasize student-to-student interaction [10]), it is reasonable to expect a significant change in role will be experienced by most faculty members transitioning to increased use of Internet-based techniques. Since most of today's faculty members were attracted to the field—at least in part—by their admiration of their own teachers, few of whom employed online techniques (unless the faculty member graduated quite recently), it is reasonable to suppose that such a change in role—from master to facilitator—will be viewed in a negative light. For my own part, I've become resigned to the fact that virtually every semester my teaching assistants will receive higher marks from students than I do and that students participating in asynchronous discussions will view my role as being less important than they do in a classroom discussion [2]. But it can be a shock to the uninitiated.

The change in status that can accompany a faculty member's move towards increased use of Internet-based teaching is not limited to student perceptions. As emphasized in items one through three, developing Internet-based techniques tends to be experimental in nature and it is therefore natural—perhaps even a duty—for instructors who are pioneering these efforts to communicate their findings. Unfortunately, from a career perspective, the publication outlets focusing on discipline education tend not to be very prestigious—for example, in a recent composite study of IS journals [9], the top education-focused journal ranked 33rd. Furthermore, those of us in the MIS field are actually lucky—in the sense that non-education journals (e.g., second-ranked CACM) will consider articles examining the use of information technology in educational situations. Unfortunately, these highly-rated journals present a different problem. With backlogs that can range from three to four years, findings that may have been current when submitted can appear dated, or even ludicrous, by the time they appear in print. This situation is likely to continue until the high pace of change in Internet-based education (see Item 2) subsides. Thus, from a career perspective, the move to researching Internet-based education can lead to a marked decline in professional status.

CONCLUSIONS: WHY ARE WE DOING THIS?

Given this litany of challenges associated with employing the Internet for higher education, is it a wonder that any sensible faculty member would choose to consider it? Fortunately, operating on the bleeding edge of technology in education also has its compensations. First and foremost, as both we and the technologies we employ become more adept, we will find more and more things that we can do better by using the Internet. I have no doubt that the baker's dozen items presented in the companion piece will expand to a couple dozen before many years have passed. Although our publications may not garner the respect of the academic researchers in our disciplines, they do have a delightful characteristic that academic research in most disciplines lost long ago—our articles are avidly read by practitioners (who just happen to be other faculty). Thus, we can realistically hope to see our efforts influence current practice, and not just future research. Furthermore, when Internet-based learning technologies finally stabilize—I'd guess about a decade, but it could just as easily be half (or double) that time—few faculty members are likely to have the luxury of deciding whether or not to use the Internet in their classes. Even if their administration is tolerant, students will demand what they have seen used effectively elsewhere. At that time, it will be the experiences of those of us who, today, continually risk being bloodied that will determine how these tools are used. Being a pioneer is important work—even if it can be (really) hard.

References

1. Gill, T.G. (2001) "What's an MIS Paper Worth? An Exploratory Analysis," Data Base. 32(2). 14-33.

2. Gill, T.G. (2004a) "A Protocol for Online Case Discussions," Decision Sciences Journal of Innovative Education, 3(1). 141-148.

3. Gill, T.G. (2004b) "Distance Learning Strategies that Make Sense: A Micro Analysis," eLearn, 8.

4. Gill, T.G. (2005) "Teaching C++ Submarine Style," IEEE Transactions on Education, 48(1). 150-156.

5. Gill, T.G. (2007) "Quick and Dirty Multimedia." Decision Sciences Journal of Innovative Education. Publication forthcoming in 5(1).

6. Hirshheim, R. (2005) "The Internet-Based Education Bandwagon: Look Before You Leap." Communications of the ACM. 48(7). 97-101.

7. McKeachie, W.J. (1990). "Research on college teaching: the historical background," Journal of Educational Psychology, 82(2), 189-200.

8. Neal, L. (2005) "Opinion: In Search of Simplicity." eLearn. Accessed on 9/18/05 at: http://elearnmag.acm.org/archive.cfm?aid=1104968

9. Rainer, K. and Miller, M. (2005) "Examining differences across journal rankings," Communications of the ACM, 48:2, 91-94.

10. Sarker, S. and Nicholson, J. (2005) "Exploring the Myths about Online Education in Information Systems." Informing Science. 8. 55-73.

Tables

T1Table 1. Programming Class Availability



Comments

  • There are no comments at this time.