ACM Logo  An ACM Publication  |  CONTRIBUTE  |  FOLLOW    

The Case for “e-Supplements” for Improving Instructional Health: Do they make a difference?

By Martin Sivula / November 2011

Print Email
Comments Instapaper

Fifty MBA graduate students (two sections) in a very traditional research methods course had the option to use hyperlinks, PDF files, Doc files, YouTube videos and other topical resources to supplement and augment their classroom experience. The purpose of the study was to determine if the use of electronic media and digital media, otherwise referred to as e-supplements, with traditional classroom sessions affected final exam grades and course grades.

The Wikipedia entry for electronic media states:

Electronic media are media that use electronics or electromechanical energy for the end-user (audience) to access the content. This is in contrast to static media (mainly print media), which today are most often created electronically, but don't require electronics to be accessed by the end-user in the printed form. The primary electronic media sources familiar to the general public are better known as video recordings, audio recordings, multimedia presentations, slide presentations, CD-ROM and online content. Most new media are in the form of digital media. However, electronic media may be in either analog or digital format.

Why Use Theory

McCombs and Vakili encourage using a theoretical framework with practical applications in an eLearning environment [1]. By such use of theoretical frameworks, hypotheses can be developed that can be tested with future research and common principles might emerge that can serve as guidelines for learners and instructors. Mayer calls for multimedia methods that are based upon empirical evidence [2, 3]. So there is a bridge to be crossed; one side is the theoretical (imaginary, conceptual) and the other the empirical (real-data-observations). The presence of the theoretical world is inferred from measurements and observations from this empirical or real world. The real world depends upon first hand direct experience-it is material, factual, perceivable [4].

There are two types of knowledge: declarative knowledge of facts ("what to") and the procedural knowledge of "how to" do various cognitive tasks. One of ACT-R theory's most important outcomes is that knowledge can be decomposed (into "chunks"). You can take a very complex domain of knowledge and analyze it into chunks using rule like units called productions to achieve tasks, goals, and sub goals [5, 6].

The Technology Acceptance Model (TAM) is more than 25 years old and simply states the easier the technology is to use, the more useful it is perceived to be, the more positive one's attitude and intention to use the technology [7]. The TAM is previously supported by Fishbein's and Ajzen's reasoned action model, which states intent to produce a behavior depends upon two basic factors: attitude toward the behavior and subjective norms [8]. Lederer, Maupin, Sena, and Zhuang reviewed more than 15 published studies using the TAM on perceived ease of use, perceived usefulness, attitude toward use, and usage of information technology [9]. Their results support the use of the TAM as an explanatory or predictive model with different technologies. Furthermore, King's and He's meta-analysis of 88 published studies on the TAM confirm the model can be used in a wide variety of contexts and perceived usefulness affects ease of use and intent to use the technology, and i.e., the model is fairly "robust" [10].

Engagement Theory states learning activities should: (1) occur in a group context (e.g. collaborative teams); (2) are project based; and (3) have an outside focus (authentic) [11]. The theory goes on to state students are intrinsically motivated due to the meaningful nature of the learning environment and activities through interaction with others on worthwhile tasks.

When working with instructional technology, especially software that has multimedia capabilities, we need to structure learning experiences and activities that maximize learning. Cognitive Load theory states working memory limits the processing and selection of sensory data [12, 13]. By having flashy and unimportant graphics, icons, etc., "bells and whistles," the learner may get overloaded in working memory before they get to the important concepts, skills, information, and/or knowledge. Mayer stated words and pictures should be used in the presentation of material [3]. Mayer and Moreno state in their "Theory of Multimedia Learning" that both narration and graphical images produce visual and verbal mental representations, which link and integrate prior knowledge and then construct new knowledge [14]. Mayer and Moreno basically say that deep learning (using multimedia) must work both visual and auditory working memory [14, 15, 16]. Visual graphics and some type of informal voice narration might be the best general methods while not overloading either. In many ways this is the old fashioned method of "show and tell," used for decades by elementary school teachers, but here you are using the technology as a tool to do the same. McCombs states "learner centeredness—it is a complex interaction of qualities of the teacher in combination with the instructional practices—as perceived by the learners" [17]. Given the aforementioned theoretical positions, the following hypotheses were developed.

H1: There is no significant difference between high and low student users of electronic media to supplement and augment face to face course with respect to final exam grades.
H2: There is no significant difference between high and low student users of electronic media supplement and augment face to face course with respect to course grade.


Subjects/Participants. Participants for this study were MBA graduate students (N = 50) in a large, southern New England university. These students were enrolled in a graduate research course over a term of 11 weeks. The majority of the students were international (where English is a second language) and between the ages of 22 and 35 years of age. One class met on a Wednesday evening (n =25) and the other met on a Thursday evening (n= 25).

Apparatus. A computer connected to the Internet is necessary. Any course management software would be appropriate, e.g., Blackboard. In our case McGraw-Hill Publishing Company's PageOut was used. Also, many Smartphones, Blackberries, iPhones, iPods, iPads could be used as well as long as there is access to the Internet and the course management software. Electronic media and digital media refer to any document, text, and/or video which were selected to relate to a major course topic as indicated in the course syllabus.

Design. The overall research design is a quasi-experimental method as each class was not randomly assigned to a group. The dependent variables were the final exam grade and the course grade. The independent variable was the frequency of use of the electronic media. Note here that a valid student session must have been five minutes or more. The instructor could control the posting of the electronic and digital media, the content, and the appearance (as much as the course management software would allow). PageOut has a function entitled "Web Links" where you can post and/or provide links for your digital and electronic media. The course in question was broken into 10 "chunks" that became modules for the entire course.

Procedure. To supplement and augment the educational objectives of the graduate research methods course, all students were required to register as a student into PageOut. They were instructed they could opt to use the technology as much or as little as they wanted as it was not a requirement. During the previous summer the instructor selected, reviewed, and tested relevant topical electronic media keeping in mind the theoretical positions previously mentioned. Then course objectives and course assignments, using Bloom's revised taxonomy [18], were cross-referenced with individual electronic and digital media. In the first class session the instructor showed the students how to register and how to access the electronic and digital media. Students were given the option to print a hard copy of the course lecture notes found on two document (.doc) files under "electronic media." This was the primer assignment to get them to the electronic media area.

During this process it was determined that no more than six electronic media would be used for any one topic, so the total number of electronic media was 60. All video clips on YouTube could not be longer than 10 minutes. The other materials were PowerPoint presentations, Portable Document Files (.pdf), Word documents (.doc), and actual websites specific to the task (topic) under study. As the course progressed the instructor would remind students of relevant electronic media in class (shown in real time), which were relevant to the topic at hand. The instructor also had the ability to move the electronic media (reorder) to the top of the list as the current topic under study dictated, moving already used links lower on the list (still available). The electronic and digital media were also selected to supplement and augment necessary task accomplishments to produce course assignments. For example one assignment was to read, review, and critique a research journal article. The Revised Bloom's Taxonomy target categories were: analyzing-breaking information into parts to explore the understandings and relationships; evaluating-justifying a decision or course of action; and creating-generating new ideas, products, and/or ways or methods of viewing things. So the subjective judgment was made as to what electronic and digital media (video, text, Power Points, etc.) would supplement and augment the desired topics/assignments. This procedure was done for each course objective and course assignment so that all were supplemented and augmented in some fashion.


For the 50 students the average frequency (of five minutes or more) of use of the e-supplements was M = 9.56, SD=6.74. Minimum e-supplement use was two sessions with a high of 33 sessions. The median of the group was eight sessions. The median value of eight was used to break the 50 into two subgroups, high users >= 8 sessions (n = 22) and low users < 8 sessions (n = 28). When examining each subgroup, the high users' average online sessions were 14.95 whereas the low users were only 5.32 (high users almost triple the low users).

Hypotheses I is rejected. The high user group (M = 83.2, SD = 10.2) out performed the lower user group (M = 75.1, SD = 11.0), p= .011 and the effect size d = .76 (moderately large). There is a significant difference between high user group and low user group with respect to their final exam grade. Hypothesis two is also rejected. The high user group (M= 90.3, SD =3.9) out performed the low user group (M = 87.3, SD = 3.4), p = .005, d = .82 (large). Therefore there is a significant difference between the high user group and low user group with respect to their course grade.

The students were also asked if the e-supplements: support what they needed to accomplish (M=3.47, SD=1.19); are easy to use (M=3.36, SD=1.13); and are useful (M=3.69, SD=.99). These were on the five point Likert scale.


The results of this study seem to support the use of electronic and digital media to support instruction (e-supplements). Anderson's model of learning with technology seems to be supported after placing the course requirements into chunks or modules [5]. Please note that individual students had the option to control their use of the e-supplements. So "choice" seems to be a factor whether or not students want to use the technology supplementing or augmenting their personal learning style. The technology acceptance model or TAM is also supported in this research where the students found ease-of-use and usefulness of the technology high or rated it high on a five-point scale. By applying theoretical models course design seems to make a difference in valued course outcomes, e.g., final exam and course grade. The limitation of this research is the lack of true controls. It is quite possible moderating variables, such as attention and distraction, might also influence the results. So at best the results are tentative, but promising. Further research should be conducted on "choice and need" (regarding hardware and software decisions) on various instructional tasks on which students are asks to perform.

About the Author

Dr. Martin Sivula is the director of research at Johnson & Wales University, Providence Campus. He is a former Director of Academic Computing and is a Certified Data Educator (CDE). In the early 1990s he served as a quantitative researcher and data analyst for the Public Education Fund study of the Providence (Rhode Island) Public Schools, which produced the Providence Report on Blueprint for Education (PROBE) Study (1991-1995). From 1994 through 2000 he served as a researcher and grant administrator for the Health Education Leadership for Providence (HELP), an organization to implement technology applications into the Providence Public Schools. Since 1999 he has served as a PT3 grant evaluator for Wheelock College's (Boston, MA) technology implementation and capacity building efforts. Recent research includes: Sivula, M. W., Hybrid graduate education: Assessing student comfort with technology interventions, Ubiquitous Learning An International Journal 3, 1 (2011), 35-42.


1. McCombs, B. and Vakili, D. A learner-centered framework for e-learning. Teacher College Record 107, 8 (2005), 1582-1600.

2. Mayer, R.E. Research-based principles for the design of instructional messages: The case for multimedia explanations. Document Design 1,1 (1999), 7-20.

3. Mayer, R.E. Multi-media Learning. New York: Cambridge University Press, 2001.

4. Aneshensel, C. S. Theory-based Data Analysis for the Social Sciences. CA: Sage Publications, 2002.

5. Anderson, J.R. Rules of the Mind. Hillsdale, NJ: Lawrence Erlbaum Associates, 1993.

6. Anderson J.R. and Schunn, C.D. Implications of ACT-R learning theory: No magic bullets. In Advances in instructional technology: Educational design and cognitive science (Vol. 5), ed. R. Glaser. Mahwah, NJ: Lawrence Erlbaum Associates, 2000, 1-34.

7. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13,3 (1989), 319-340.

8. Fishbein, M. and Ajzen, I. Belief, Attitude, Intention and Behavior: An introduction to theory and research. Reading, MA: Addison-Wesley, 1975.

9. Lederer, A. L., Maupin, D.J., Sena, M.P. and Zhuang, Y. The technology acceptance model and the World Wide Web. Decision Support Systems 29,3 (2000), 269-282.

10. King, W.R. and He, J. A meta-analysis of the technology acceptance model. Information & Management 43,6 (2006), 740-755.

11. Kearsley, G. and Schneiderman, B. Engagement theory: A framework for technology-based learning and teaching. 1999. Originally at

12. Sweller, J. Cognitive load during problem solving: Effects on learning. Cognitive Science 12,2 (1988), 257-285.

13. Sweller, J. Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction 4,4 (1994), 295-312.

14. Mayer, R. E. and Moreno, R. Aids to computer-based multimedia learning. Learning and Instruction 12,1 (2002), 107-119.

15. Mayer, R. E. and Moreno, R. A split-attention effect in multimedia learning: Evidence for dual processing systems in working memory. Journal of Educational Psychology 90,2 (1998), 312-320.

16. Mayer, R. E., and Moreno, R. Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist 38,1 (2003), 43-52.

17. McCombs, B. Assessing the role of educational technology in the teaching and learning process: A learner-centered perspective. The Secretary's Conference on Educational Technology, United States Department of Education, 2000.

18. Anderson, L. W. and Krathwohl, D. R., et al. eds. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Boston, MA: Allyn & Bacon, 2001.


  • There are no comments at this time.