ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

A Roadmap for Evaluating Online Teaching
A review of 'Evaluating Online Teaching: Implementing Best Practices'

By Adam Wayne Jenkins / August 2015

REVIEW: LITERATURE, TYPE: MANAGEMENT
Print Email
Comments Instapaper

A professor of mine once remarked "it depends" is usually a correct, if incomplete, response to any question on the subject of education. The idea behind that quip is the practically infinite uniqueness of contexts in which people teach and learn makes one-size-fits-all answers all but impossible to produce. With this in mind, I am skeptical about any piece of education literature that has the "best practices" label attached to it. After all, the "best practices" described by the author might work in their distinct educational situation, but would it necessarily work in mine?

The recently published Evaluating Online Teaching is subtitled Implementing Best Practices, with "best practices" printed in all capital letters on the cover. When I considered the potential range of online teaching contexts in higher education (academic discipline, institution type, student profile, faculty expectations and resources, etc.) it seemed unlikely that a single book could be very useful in developing effective methods of evaluating online teaching. However, the authors Thomas J. Tobin, B. Jean Mandernach, and Ann H. Taylor have written a book that places heavy emphasis on contextual variables. Evaluating Online Teaching is not so much a how-to manual as it is a roadmap to developing your own context-specific online teaching evaluation methods, and consists largely of posed questions that must be addressed and the identifying of considerations that must be made when doing so.

Evaluating Online Teaching is praiseworthy for the way in which the authors emphasize the importance of context in evaluation. Each chapter starts with a real-world story or case study about a challenge faced by a person or institution related to the evaluation of online teaching, which helps establish the relevance of the content to follow. Each chapter ends with a thought exercise the reader is suggested to undertake; the reader is asked to relate the content from the chapter to their own institution in some way. Readers can also download rubrics and other forms that can be adapted to fit specific needs via the publisher's website.

Evaluating Online Teaching is organized into four sections—Planning, Formative Evaluation, Summative Evaluation, and Sustaining a Culture of Evaluation—with each section containing between one and four chapters. The first chapter of the planning section begins with an explanation of the theoretical foundation of the book: Chickering and Gamson's "seven core principles" for good teaching provide the fundamental backbone of the book [1]. The chapter outlines the common challenges faced by those who are tasked with evaluating online instructors. Chapter 2 guides the reader through a series of questions toward building awareness about "institutional," "implementation," and "impact" factors that must be considered when creating online teacher evaluation protocols. Chapter 3 builds on the previous chapter and suggests a "…decision-making process for considering the focus, nature, and purpose" of developing online teaching evaluation procedures.

In the second section, the authors identify formative evaluation as "…gaining feedback during a course to improve the ongoing teaching and learning process." In Chapter 4, the authors discuss the difference between formative and summative evaluation and the unique challenges of collecting information for formative evaluation in online courses. A sequence of steps for conducting formative evaluation is also explained. The authors note formative evaluation is often more informal ("organic" is the term the authors use) than summative evaluation, but there are still a number of considerations that must be addressed when planning formative evaluations. A "who, what, when, where, why" checklist, of sorts, is provided to help with the planning of formative evaluations. For conducting effective, and ongoing, formative evaluations, the "SCARF" loop is introduced [2]. The word "SCARF" serves as mnemonic device for the steps in the loop: "(S)olicit the desired information, (C)ompile and analyze findings, (A)djust teaching based on the feedback, and (R)eport (F)eedback to stakeholders." Suggestions are given for each step of the SCARF loop.

In section three, summative evaluation is described as being "…conducted at the end of a course or program" to ascertain the overall quality of teaching. Chapter 5 addresses issues related to student evaluations of teaching and gives an overview of a handful of existing student measures of teaching performance that can be adapted for online education, including: Student's Evaluation of Educational Quality (SEEQ), Student Assessment of their Learning Goals (SALG), and Student Rating of Teaching Effectiveness (SRTE). Chapter 6 looks at summative evaluation from the perspective of an administrator, and suggests ways in which traditional face-to-face models of administrative evaluation can be adapted to the online format. Specifically, this chapter provides a table that identifies observable online teaching behaviors that are indicative of Chickering and Gamson's seven principles of good teaching. In Chapter 7 the authors share some existing tools and frameworks through which summative evaluations can be conducted. Among the tools and frameworks discussed are: the Checklist for Online Interactive Learning, the Quality Online Course Initiative, the Quality Matters Rubric, the Online Instructor Evaluation System, and the Peer Review of Online Teaching. The section concludes with a discussion on how data analytics can be used as part of the evaluation process in Chapter 9.

The final section of Evaluating Online Teaching focuses on sustaining a culture of online teaching evaluation. In Chapter 9 the authors explain the need to create a "holistic evaluation plan" that includes both formative and summative evaluations. Chapter 10 suggests a path for overcoming obstacles that might prevent the implementation of change to the way online teaching is evaluated at an institution. The ways in which practices such as faculty hiring, training, and development should align with evaluation are discussed in Chapter 11. Finally, the book finishes with Chapter 12, which pulls all of the material together.

Although the authors state Evaluating Online Teaching includes teaching faculty as a "key audience," I believe much of the text would fall outside of the areas of concern for most teaching faculty. Having said that, the chapters that discuss good practices and formative evaluation are worth a read for faculty who are new to online teaching. Evaluating Online Teaching is certainly most useful for those who work in distance education administration. The book would be invaluable to anyone tasked with implementing, refining, or overhauling the way online teaching is evaluated at an institution.

References

[1] Chickering, A. and Gamson, Z. Seven principles for good practice in undergraduate education. The Wingspread Journal 9, 2. (1987).

[2] Tobin, T. J. Best practices for administrative evaluation of online faculty. Online Journal of Distance Learning Administration 7, 2 (Summer 2004).

About the Author

Adam Jenkins is a doctoral student in adult education at Penn State University. He is also an instructor of online and face-to-face communication classes at several colleges and universities.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

2015 Copyright held by the Owner/Author. 1535-394X/15/08-2815650 $15.00



Comments

  • There are no comments at this time.