ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

Findings on Modeling as Impact to Practice from the HumanMOOC

Special Issue: Paradigm Shifts in Global Higher Education and eLearning

By Heather A. Robinson, Maha Al-Freih, Whitney Kilgore, Patrice Torcivia Prusko / May 2019

Print Email
Comments Instapaper

The growing pool of MOOC reports and research continues to highlight the heterogeneity of learners’ goals and purposes for joining MOOCs, with goals related to professional development and career advancement emerging as a consistent theme in these studies [1]. Unfortunately, most of these studies are limited to the identification of these goals and purposes for joining a MOOC without investigating whether participants were able to achieve these personal and professional goals as a result of their experience in the MOOC. There is an emerging consensus among researchers in the field regarding the need for new approaches to capture and measure useful and contrasting dimensions of the MOOC experience that can be used as a proxy for MOOC effectiveness and efficacy [2, 3]. This study extends this discussion by examining MOOC effectiveness in terms of participants’ ability to utilize and transfer the knowledge and skills gained in a MOOC to their professional practices. Studies suggest that some learners fail to use the knowledge they gain in MOOCs in their jobs and that improvements in MOOC design could help remedy this problem [4]. The urgency of investigating how learning in a MOOC for professional development purposes impacts actual practices is highlighted given the fast pace at which knowledge and job roles and responsibilities are changing, rendering traditional on-the-job professional development and training insufficient in addressing professionals’ continuing need for learning and development [5]. Understanding what factors in MOOC design, pedagogy, and learning activities relate to the transfer of knowledge, which can only be understood by examining learners’ experience after they leave a course, is an important and interesting area of study.

In an attempt to address this gap in MOOC research and literature, a qualitative phenomenological study was conducted to examine participants’ perception of their experience translating what was learned in a MOOC entitled “Humanizing Online Teaching and Learning” or HumanMOOC into actual practice. The HumanMOOC was designed to be a professional development experience for those who wanted to improve their online teaching practices by introducing them to the community of inquiry (CoI) framework [6]. The first week of the MOOC oriented participants to the course, provided a schedule of activities, and explain how to effectively participate in the MOOC using social media. The following three weeks were divided into three weekly modules with each focusing on one of the three presences of the CoI framework (i.e. instructor, social, and cognitive presence). Within each weekly module, learning objectives, to-do-lists, articles and annotations, and assignments were provided. Further, a live Google Hangouts session with an expert in the field was streamed and recorded each week [7]. This article describes some of the findings that emerged based on semi-structured interviews that were conducted with 10 participants who have joined the HumanMOOC in one or more of its iterations.


The reasons to examine the effectiveness of a professional development course or program can vary from uncovering successes and failures, looking at the need for revision, or to learn about the impact of the program on the participants’ practice. Studies on impact to practice are fairly limited in the area of online learning. O’Donnell conducted an examination on the research in the areas of fidelity of implementation, efficacy, and effectiveness studies to clarify definitions and to guide researchers interested in these forms of evaluation [8]. She discovered a deficiency of literature in the education area but discovered additional research in the public and mental health fields to further ground her study. Overall, overlaps and disparities in the literature on the variety of terms and measurements used to research fidelity of implementation, efficacy, and effectiveness studies were discovered and there is simply not enough guidance for researchers [8]. It is recommended that researchers “establish the program theory a priori and determine what it means to implement the program with fidelity” and measure user’s fidelity to program components [8].

If participants’ instructional practices are aligned with that of the course, this contributes to a higher level of implementation [9]. Further, alignment between the scope of the program or course with the participants' scope and or district policies was a consideration for implementation into practice. When learners have an interest in what they are learning it makes a difference. It “can engender a sense of the excitement of learning that is then transferred to the classroom, conferring a sense of ownership of new ideas as they apply to theory and practice” [10].

Using design framework principles appropriately in learning environments support the impact to practice for change [10]. The design framework principles applied to adult learners include professional development programs or courses that are learner centered, knowledge centered, assessment centered, and community centered. Participants of these programs are encouraged to discuss areas where additional learning is needed and integrate their new knowledge into a real-life scenario, such as a new lesson into their curriculum during their time in the course. Further, feedback on this integration aids in the transfer of the technique into practice and the continued access to this community further supports learners [10].

Instructor rapport building and modeling in online learning are additional components in this discussion. Instructor modeling was identified as one of the most important elements in online learning [11]. Modeling expected behaviors in online discussions is commonly used by online instructors to establish instructor presence [12]. Creating instructional videos for online courses is becoming more common. This approach has been effective for students with little prior experience with a task [13, 14]. A new study on the use of video lecture to optimize learning looked specifically at the effect on learning using an instructor’s face in the video and an instructional video without. The modeling with the instructor in the video did attract and keep a learner’s attention as opposed to just voice instruction [15]. Instructor based traits and practices like social presence, enjoyable interaction, and a personal connection were preferred by learners in establishing rapport in online courses [16].


While it was not a consideration to conduct an impact to practice study when the HumanMOOC was active, the feedback we received from evaluations necessitated further exploration on this phenomenon. After interviewing participants who completed the course it was discovered instructor modeling was a pivotal factor on the impact to practice. Seven of the 10 participants interviewed spoke about some form of modeling without being directly asked about it. Modeling both in terms of course design, as it was based on the CoI framework, as well as facilitation and interactions within the MOOC were helpful.

Speaking from experience as an online instructor and someone who already utilizes quite a bit of technology, Tim stated: “I think it was modeled for me really well in the HumanMOOC to see that you all, all of the Wayfinders didn’t necessarily comment on everything that everybody said. And I think because there’s 3,000 or 4,000 posts that are really going on. But it definitely felt like you all had a presence there. And it was really neat to see in a large course like that, you also had four or five of you to help with that. Of course, we all talked about that as Wayfinders do.”

Eric described how modeling of effective practices led to an impact on his program stating “so that was something, I think that I really picked up from the time at Human MOOC as I was in the middle of revamping that program. And I just thought okay, this is really significant to keep in there. And that isn’t to say that we don’t try other methods through the ... which we now use Canvas, through the Canvas classroom to engage with students and show a video and just discussion forums and stuff. But to continue that sort of live synchronous session was I think a significant part of what we continued as a result of the class.”

Nell explained how the experiences from the course helped him better understand the importance of scaffolding and modeling stating: “My biggest transformation, my thinking was that I’ve got to go a little slower than I thought with people, because I’ve got to give them things that they can have a really positive experience with to build up their confidence so that they will kind of take off with things.” Additionally, Nell stated: “I really understood that it’s about who you’re learning with, or, you know, and how people are engaging with each other that actually challenges you and makes your thinking bigger and makes you think about things in a different way than you have before.”

On new technologies utilized, Tim explained that the course “opened my eyes to a few others, and even had me look at some other ones that we already used in a new light...there’s that one complimenting technology, VoiceThread... that was something I always thought was cool.” Heidi liked the VoiceThread chat used in the course, “it was also done in a way that an effective elearning course is done...interactive e-learning like MOOC style or higher ed style as opposed to e-learning corporate style. Which tends to be you and the computer. Which is a new paradigm for some people. It was a many-to-many as opposed to one-to-many or one-to-one kind of setting, and so the course modeled very well what was being taught.” Seeing the integration of different technologies directly was helpful to Eric. He recalled the use of the instructors giving video feedback which was new to him. That modeling was helpful to his implementation in his current course and something he continues to use. Another way to participate via open technology was to join live Google Hangouts using live Twitter chats running at the same time. Jessica was inspired by this approach and is now using Twitter in her teaching.

Specific to the modeling of course design, Eric mentioned “one thing was seeing the inquiry model, seeing that laid was just helpful to see the community of inquiry model. Tim also commented on the design elements, “in the HumanMOOC, I think I learned things like it really needs to be really bite-sized. It needs to be short, it needs to be something that people can easily consume and the thought would be that they’ll consume more, be exposed to more. It’s much more approachable. Then even trying to think about that engagement part, like, ‘Hey, at the end of your discussion post, or at the end of your video ask the class a question.’”


Our goal was to discover if actions from the facilitators of the course affected or contributed to the actual, real-world implementation to the participant’s classroom or work. One key discovery was the use of instructor modeling in the design of the HumanMOOC and instructor and peer-technology modeling was effective and influenced the future use and integration of what was learned into the real-world environment. We did not intentionally integrate a direct form of modeling per se into the HumanMOOC, however we did develop pre-recorded videos and live sessions (Google Hangouts) that could be described as modeling. The passive instructor modeling that course participants identified as helpful is something we see as a best practice in online learning and online course design.

Participants reported this passive-instructor modeling was a positive example for technology integration, but was also reported as helpful in understanding the CoI framework and how that could be implemented in the Canvas LMS. Learning about the CoI through self-study and Google Hangouts lectures combined with the immersion of the framework in the LMS appears to be an effective combination. The instructor is important; the value of a strong instructor presence is frequently studied and is being considered in online course design more frequently. People need other people and social presence is an additional design component for the online space. Those we interviewed seemed to be looking for a community or tribe, looking to learn or extend their professional learning network (PLN). That intention of learning from and with their colleagues came first. Modeling and example sharing of ideas molded this learning intention deeper for them to actually implementing into their practice. If this is possible and an effective strategy in a MOOC with hundreds of students, it must be considered for traditional online courses.


[1] Littlejohn, A., Hood, N., Milligan, C., and Mustain, P. Learning in MOOCs: Motivations and self-regulated learning in MOOCs. The Internet and Higher Education. 29 (2016), 40–48. doi:10.1016/j.iheduc.2015.12.003

[2] DeBoer, J., Ho, A. D., Stump, G. S., and Breslow, L. Changing “course”: Reconceptualizing educational variables for massive open online courses. Educational Researcher 43, 2 (2014), 74–84. doi:10.3102/0013189X14523038

[3] Ho, A. D. et. al. HarvardX and MITx: Two years of open online courses. HarvardX Working Paper No. 10. 2015. doi:10.2139/ssrn.2586847

[4] Mathews, D. MOOC students “passive”, study suggests. Times Higher Education. (2014).

[5] Littlejohn, A. and Milligan, C. Designing MOOCs for professional learners: Tools and patterns to encourage self-regulated learning. eLearning Papers 42, article No. 4 (2015).

[6] Garrison, D. R., Anderson, T., and Archer, W. Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education 2, 2-3 (2000), 87-105.

[7] Kilgore, W., and Al-Freih, M. MOOCs as an innovative pedagogical design laboratory. International Journal on Innovations in Online Education, 1, 1 (2016).

[8] O’Donnell, C. L. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K–12 curriculum intervention research. Review of Educational Research 78, 1 (2008),33–84.

[9] Barlow, A. T., Frick, T. M., Barker, H. L., and Phelps, A. J. Modeling instruction: The impact of professional development on instructional practices. Science Educator 23, 1 (2014).

[10] National Research Council. How People Learn: Brain, mind, experience, and school, Expanded edition. National Academies Press, Washington D.C., 2000.

[11] Vesely, P., Bloom, L., and Sherlock, J. Key elements of building online community: Comparing faculty and student perceptions. MERLOT Journal of Online Learning and Teaching 3, 3 (2007), 234–246.

[12] Richardson, J. C., Besser, E., Koehler, A., Lim, J., and Strait, M. Instructors’ perceptions of instructor presence in online learning environments. The International Review of Research in Open and Distributed Learning 17, 4 (2016).

[13] Renkl, A. Toward an instructionally oriented theory of example based learning. Cognitive Science 38,1 (2014) 1–37.

[14] van Gog, T., and Rummel, N. Example-based learning: integrating cognitive and social-cognitive research perspectives. Educational Psychology Review 22 (2010), 155–174.

[15] van Wermeskerken, M., Ravensbergen, S., and van Gog, T. Effects of instructor presence in video modeling examples on attention and learning. Computers in Human Behavior 89 (2018), 430–438.

[16] Wright, R., and Jones, G., and D'Alba, A. Online students' attitudes towards rapport-building traits and practices. International Journal of Innovation and Learning 17 (2015), 36–58.

About the Authors

Whitney Kilgore, Ph.D. is Co-founder and Chief Academic Officer of iDesign, a partner to universities who wish to build, grow, and support online and blended course and program offerings. iDesign provides concierge, white-glove instructional design support to faculty partners, bringing expertise, service, and project structure to bear and ensuring that faculty feel comfortable, informed, and in control throughout the process. Her research interests are focused on learner experience design. Dr. Kilgore has led the development of programs across the U.S., Spain, the Philippines, China, Australia, Latin America, and the U.K. She is also an adjunct faculty at the University of North Texas (UNT) in the Learning Technologies graduate program. As an academic, Dr. Kilgore has received numerous recognitions for her work, including a research award from UNT and inclusion in the Top 10 Research Articles (co-authored with Aras Bozkurt of Turkey and Matt Crosslin from UTA) for 2017 list produced by Dublin City University for their work on Bot-Teachers in Hybrid Massive Open Online Courses (MOOCs): A Post-Humanist Experience. Whitney has been a part of the recent EdTech efficacy research project conducted in conjunction with Columbia University and the University of Virginia on EdTech Decision Making in Higher Education. She is currently working on research related to care theory in online learning and the impact to practice of humanizing online teaching and learning, and is deeply engaged in the Empirical Educator Project. She is an editor of the International Journal of Innovations in Online Education and recently edited and published the book Humanizing Online Teaching and Learning as an open educational resource. Her next edited book, entitled Connecting the Dots: Book: Improving Student Outcomes with Exceptional Instructional Design, is scheduled to be available as an OER at the end of 2019.

Patrice Prusko is the Assistant Director of Learning Design, Teaching and Learning Lab, Harvard Graduate School of Education.  Dr. Prusko oversees the design, development and project management of online and technology-enhanced courses in the TLL portfolio. Her team thoughtfully incorporates LX design, UDL, learning theory, and instructional design practices to create courses that are inclusive and designed for a diverse student population.  Prior to joining the TLL team I worked as an Instructional Designer at Cornell University in the Center for Teaching Innovation, and as a faculty member at the State University of New York, Empire State College, Center for Distance Learning and International Programs. Her research looks to develop course and systems level structures that support inclusive, transformational learning experiences for all students, especially first-generation and underrepresented student populations in Science, Technology, Engineering and Mathematics.  Dr. Prusko holds a degree in Mechanical Engineering (B.S.), and Business Management (MBA) from Union College, and Curriculum and Instruction (Ph.D.) from University at Albany.

Heather Robinson has worked in the computer technology field for more than 20 years and is presently the Instructional Technology Information Specialist at Jackson Hole High School and adjunct faculty at the University of North Texas. She holds a master’s in information science and a Ph.D. in learning technologies from the University of North Texas. Heather has presented and is published on her research on social-constructivist online learning, instructional design, and faculty experiences with online technology adoption. She is currently working on research related to care theory in online learning and the impact to practice of humanizing online teaching and learning.

Maha Al-Freih received her Ph.D. in learning technologies design research from George Mason University, USA in 2017. She is currently an Assistant Professor of Instructional Design & Technology at the College of Education at Princess Nourah Bint Abdulrahman University (PNU) in Riyadh, Saudi Arabia, where she has been a faculty member since 2011. She is also serving as the Vice Dean of Learning and Teaching at the College of Business Administration at PNU. For the past 10 years, she has provided numerous training events for both K-12 and higher education faculty in areas such as technology integration and innovative learning and assessment strategies and has participated in numerous national and international conferences. She is also a member of the #HumanMOOC team since 2015 and has participated in the design and delivery of the MOOC during its offerings on the Canvas Open Network. Her primary research interests include learners’ engagement and persistence Massive Open Online Courses (MOOCs), Personal Learning Environments (PLEs), Self-Regulated Learning (SRL), and Design-Based Research (DBR).

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

2019 Copyright is held by the owner/author(s). Publication rights licensed to ACM. 535-394X/19/06-3313875


  • There are no comments at this time.