Adaptive knowledge assessment using advanced concept maps with logic branching multiple-choice Google Forms

Concept maps (CM) are a learning tool that have emerged into an efficient e-learning and e-assessment knowledge tool. The aim of this research is to propose and share the most important aspects, practices, and achievements of using, with science teachers and a master student, a combination of a metacognitive tool-advanced concept mapping (ACM) to assess mental models with an immediate real-time feedback assessment tool. The use of the logic branching feature of multiple-choice Google Forms (MCGF) may enable teachers to customize surveys and to assess within many students' high order thinking skills, with the convenience and efficiency of an automatic grading system. Additionally, the ACM-MCGF enables the test taker to have an adaptive learning practice while undergoing assessment.

In today's world in the face of an enormous instability, there is a need for a rapid shift in educational paradigm. So, the new goal for education must be brain plasticity and, in this effort, every single tool available must be used to fulfill this aim. Cognitive performance and stress control (emotional skills) must be the targets, to increase neurogenesis and synaptic plasticity [1, 2]. Recent literature describes a deep connection and synergy between the concept mapping theory [3] and the actual function within the thinking brain-neocortex (neurosciences) [4, 5]. This "mutualist approach" may enable the achievement of higher order thinking skills (HOTS). In accordance with the complexity paradigm of Edgar Morin, to reach simplicity in learning there is a need to travel through complexity. Concept mapping is a metacognitive tool that enables this hardworking but rewarding journey to expert knowledge [6]. First you add information (novice), then there is the edition process (competent), and finally the integration of the information (excellence-expertise).

Despite all the contributions by the educational sciences, there is still an input-output black box metaphor at the root of the problems in modern age. The quality of understanding is only indirectly assessed, and it is engaged in root learning (remembering and eventually understanding) [7].

There is an urgent need to apply findings from cognitive science to improving cognitive skills, specifically meaningful learning, deep insight, and creative thinking. The algorithm must be from novice to elite expert but through "expert" learner. Thus, a skilled expert understanding must be promoted. There is a need to overtly focus on the key determinant of performance, which is "knowledge structure," thus there is a central goal, which is to foster development of adaptive expertise [8].

Development of a skilled cartographer as opposite to the black box metaphor will foster adaptive expertise. This skilled learning can be obtained by knowledge engineering, namely by fostering abstraction, analysis, synthesis, and inference abilities. This new cartographic metaphor of education can model adaptive expertise, promoting problem solving skills and meaningful learning [9]. There is a new opportunity to rationally design effective training interventions, considering learners' prior knowledge. To achieve this goal advanced/applied concept mapping can be used for knowledge cartography, enabling the identification of concepts that learners are struggling with, further equipping the teacher to target those specific areas [10, 11]. This can yield the most growth, whether teaching to a whole class or in smaller groups. Concept maps assess HOTS. Concept mapping is a graphical presentation of the relations between concepts using linking words. The concept mapping technique that was developed by Novak and Gowin defends the organization of concepts in a hierarchical manner [12] and is supported by Ausubel's meaningful learning theory [3]. This theory claims that for meaningful learning to take place, the learners need to assimilate the new concepts with prior knowledge by integrating them into a systematic structure [13]. Concept maps are powerful tools that can help to identify the misconceptions of the students as well as to realize the conceptual change and restructuring of the students' knowledge [14]. It is always very important and elucidating when learners can build their own concept maps in a "blank sheet approach," but for teachers reviewing lots of maps it can be very challenging and time-consuming. Marking them is also hard and rubrics can be confusing [15]. By "broadening the lens of the meaning" of concept mapping [16, 17] it is possible to use other approaches, namely the "Swiss cheese," by providing concept maps with a predetermined structure but missing content (concepts and/or linking phrases) that may be filled in. This is typically achieved from a bank of items [18] or by filling in content [19] or by providing options for selecting the most appropriate content [20, 21, 22]. The referred approaches enable the learner to impart some of their own representation as they may consider what is available to them in the map [23]. One key advantage of these approaches is that they enable the exercise of the HOTS as proposed by Bartlett in 1955 [6, 24]:

• Interpolation: filling in information that is missing from a logical sequence;
• Extrapolation: extending an incomplete argument or statement; and
• Reinterpretation: rearrangement of information to affect a new interpretation.

Emphasis can be placed on assessing concepts and how they relate. Answers become dependent on other answers, so learners must think through an entire segment of the domain rather than recognizing or recalling parts. It is therefore possible to express the depth of knowledge regardless of their writing capabilities.

Using advanced/applied concept maps merged with the branching feature of Google Forms makes it possible to set up conditional logic conditions, creating flexibility to personalize, differentiate or allow for choice in a very straightforward way [25]. This feature of Forms, which is often overlooked, can help teachers to construct dynamic questions that enable to show or hide certain fields of forms, thus sending respondents to specific pages based on their answers to multiple-choice questions. This performance-based adaptive approach keeps students motivated and challenged, which is critical and of utmost importance. This logic branch feature should be designed according to the algorithm complexity desired considering the target (middle or high school students, undergraduate university students, graduate students or even teachers). So, it is possible to customize the feedback survey thus promoting high order flexibility allowing also to personalize feedback [26]. This "tutorial" approach enables a more efficient teaching by fostering the rapid access to the results.

Methodology

Development and implementation of the ACM-LBMCGF. The main idea of this research was to use concept mapping-based assessment to give a deeper insight into a learner's knowledge structure about a specific topic. To start the design of this assessment tool we need to select a clear and very integrated concept map (see Figure 2). This concept map can be created from several software (for example, CmapTools, Lucidchart, or MiMind). In the selected concept map, the assessor needs to identify which concepts are to be assessed.

To obtain the assessment tool we need to merge the concept map with multiple-choice Google Forms. For each concept we need to create a multiple-choice question in a separate section. Each section will have the same figure (the concept map). The next step is to set up the logic branching feature of Google Forms, according to the target group that is going to be assessed (in our case K–12 or master students). Branching logic works only for questions that use multiple-choice or drop-down answers in them. Next, we need to click on a question to add branching logic (three dots) in the bottom-right corner and then click on "Go to section based on answer." From the list of answers, we must click the drop-down menu and then select the section we want it to connect to when the learner chooses it (Figures 3 and 4 illustrate the logic branching algorithm and feature using ACM-LBMCGF and the Biofilm Formation as the assessment concept maps). When we get to the end of a logic chain, the drop-down menu, at the bottom of the section, must be clicked to choose, from the list of options, "submit form." Analyzing the algorithm in Figure 5, it is possible to understand how the logic branching feature of the Google Forms was designed, namely which assessment outputs can be obtained in the end.

Use of ACM-LBMCGF as a formative assessment tool. The event focused on training eight science teachers and one master student in a "Microbiome and Biofilm" online course. One of the main objectives was to gather knowledge of how well the biofilm formation process is understood, since it is the major conceptual framework for the biofilm paradigm. For that aim ACM-LBMCGF was used as a formative assessment tool to obtain an effective diagnostic of the thematic conceptual framework change of the participants. To simulate an adaptive learning environment, participants were enabled to respond several times, asynchronously, during the course to the adaptive Google Forms. After the assessment task, and in the next session of the course, there was detailed group and individual feedback of the results and a discussion of the efficacy of the assessment tool using the Zoom breakout rooms feature and the assessment Google Forms as a guide.

Results

Scores to the ACM-LBMCGF were analyzed across days. Participants engaged the training material over the course of several days, showing increasing scores across time. There was marked improvement from the first assessments to the last, demonstrating that the assessment tool effectively gauged learning progress. All the participants showed performance improvement across time until obtaining a final average score of 89.5 percent, median 91 percent, and confidence interval 76–97 percent. This score was obtained after participants resubmitted the forms from two to six times.

Overall, participants gave very positive feedback during the individual and group feedback to the assessment tool, reinforcing that it enabled a thematic "deep dive." Nevertheless, some of the participants stated that the assessment tool is time-consuming and demanding and one statement disagrees with such an adaptive approach.

Discussion and Conclusion

The use of advanced concept maps coupled with the logic branching feature of multiple-choice Google Forms (MCGF) enabled active learning skills in participants by turning unfamiliar concepts and processes into habits of mind. The assessment tool enabled metacognition since the test taker had an adaptive learning practice while undergoing the assessment process. After the assessment and using the interviews, it was possible to follow up whether participants had identified their conceptual misunderstandings, thus enabling a change in their cognitive structures [12]. It is possible with this tool to customize surveys and to assess within participants higher order thinking skills, with the convenience and efficiency of an automatic grading system. Further studies must include a much wider sample to reinforce this preliminary work.

Acknowledgments

We are grateful to Extremina, C., M.S. and Fonseca, A., M.D., Ph.D. for critical reading of the manuscript and helpful suggestions.

References

[1] Sampedro-Piquero, P. and Begega, A. Environmental enrichment as a positive behavioral intervention across the lifespan. Current Neuropharmacology 15, 4 (2017), 459–470.

[2] Boran, L. et al. Brain training and integrated executive function. In 10th Annual Psychology, Health and Medicine Conference, Dublin City University, Dublin, Ireland, 2013.

[3] Ausubel, D. P. et al. Educational Psychology: A Cognitive View (2nd. ed.), Holt, Rinehart and Winston, New York, 1978.

[4] Baley, C. A. Concept mapping: A neuro-scientific approach. In Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education. University of Wisconsin-Milwaukee, Milwaukee, WI, 2005.

[5] Santos, F. S. et al. Interlocução entre neurociência e aprendizagem significativa: uma proposta teórica para o ensino de genética. Revista Brasileira de Ensino de Ciência e Tecnologia 9, 2 (2016), 149–182.

[6] Cañas, A. J. and Reiska, P. What are my students learning when they concept map? Concept mapping: Renewing learning and thinking. In Proceedings of the 8th International Conference on Concept Mapping. Medellín, Colombia, 2018.

[7] Delany, D. Advanced concept mapping: Developing adaptive expertise. In Proceedings of the Third International Conference on Concept Mapping. Tallinn, Estonia and Helsinki, Finland, 3, 2008, 32–35.

[8] Delany, D. Knowledge engineering: Building adaptive expertise. In Proceedings of the 5th Annual Excellence in Teaching Conference. King's College, London, U.K., 2011.

[9] Delany, D. Analytic constructivism: A novel cognitive science-based pedagogic paradigm for enhancing the speed, depth and quality of learning, problem solving, and innovation. In Inspiring Creativity, Innovation and Change in Higher Education. Dublin, Ireland, 2012.

[10] Fonseca, A. P. et al. Concept mapping: A strategy for meaningful learning in medical microbiology. In Proceedings of the First International Conference on Concept Mapping. Universidad Pública de Navarra, Pamplona, Spain, 2004, 167–169.

[11] Fonseca, A. P. The software CmapTools-IHMC and microbiology: A bridge to meaningful learning? Journal of Microbiology and Biology Education 13, 1 (2012), 98–99.

[12] Novak, J. D. and Gowin, D. B. Learning How to Learn. Cambridge University Press, New York, 1984.

[13] Kinchin, I. M. Visualizing Powerful Knowledge to Develop the Expert Student: A Knowledge Structures Perspective on Teaching and Learning at University (1st ed). Sense Publishers, Rotterdam, The Netherlands, 2016, 15–73.

[14] Ruiz-Primo, M. A. On the use of concept maps as an assessment tool in science: What we have learned so far. Revista Electronica de Investigacion Educativa 2, 2000, 29–52.

[15] Cañas, A. J. et al. CmapAnalysis: An extensible concept map analysis tool. In Proceedings of the Fourth International Conference on Concept Mapping. Universidad de Chile, 2010.

[16] Yelich Biniecki, S. M. and Conceição, S. C. O. Using concept maps to engage adult learners in critical analysis. Adult Learning 27, 2 (2015).

[17] Conceição, S. C. O. et al. Using concept mapping as a tool for conducting research: An analysis of three approaches. Cogent Social Sciences 3, 1 (2017).

[18] Torre, D., Durning, S. J., et al. Twelve tips for teaching with concept maps in medical education. Medical Teacher 35, 3 (2013), 201–208.

[19] Schau, C. et al. Select-and-fill-in concept map scores as a measure of students' connected understanding of science. Educational and Psychological Measurement 61, 1 (2001), 136–158.

[20] Correia, P. et al. Cmaps with errors: Why not? Comparing two Cmap-based assessment tasks to evaluate conceptual understanding. In Proceedings of the Sixth International Conference on Concept Mapping. Springer, 2016, 1–15.

[21] Moon, B. et al. Concept map-based assessment for adult learners. In Proceedings of the Fourth International Conference on Concept Mapping. Universidad de Chile, Viña del Mar, 2010.

[22] Moon, B. et al. Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge. CRC Press, New York, 2011.

[23] Kinchin, I. M. et al. Uncovering types of knowledge in concept maps. Educational Science 9 (2019), 131.

[24] Moon, B. A case for the superiority of concept mapping-based assessments for assessing mental models. In Proceedings of the Eighth International Conference on Concept Mapping. Institute for Human and Machine Cognition and Tallinn University, 2018.

[25] Haddad, R. J. and Kalaani, Y. Google Forms: A real-time formative feedback process for adaptive learning. In Proceedings of the 2014 ASEE Annual Conference & Exposition. ASEE, 2014.

[26] Ho, V. W. et al. Knowledge maps: A tool for online assessment with automated feedback. Medical Education Online 23, 1 (2018).

Authors

António Fonseca has a bachelor's degree in biology (educational branch and scientific-technological branch) from the Faculty of Sciences of the University of Porto, a master's degree in biomedical engineering from the Faculty of Engineering of the University of Porto, a doctorate in human biology from the Faculty of Medicine of the University of Porto, doctoral course in Educational Sciences of the Faculty of Psychology and Educational Sciences of the University of Porto. He is a specialist in biotechnology and a member of the College of Specialists - Order of Biologists (after curriculum evaluation since 2008). He is also a member of the College of Education of the same professional association (after curriculum evaluation). He is an accredited trainer by IEFP and by CCPFC - Universidade do Minho in the areas of biology and health sciences and in the field of health education. He is a doctorate collaborator at the interdisciplinary Center for Marine Environmental Research at the University of Porto - CIIMAR (2020-current). Researcher at INEB - University of Porto-U.P. (1996-2000), IPATIMUP-U.P. (2002-2008), REQUIMTE - U.P. (2009-2011), CIIE-U. P. (2009-2019). He has been teaching for 32 years, having initially been a science teacher then professor at the Faculty of Medicine of the University of Porto and currently a science teacher, professor of higher education (as a guest), researcher and trainer. He is the author of several publications and has supervised postgraduate work (master and doctorate) in the area of microbial biofilms and infections associated with medical devices (since 1999) and member of the editorial board (as reviewer) of Journal of Microbiology and Biology Education-JMBE (American Society of Microbiology-ASM), Science Journal (European Molecular Biology Laboratory - EMBL). He has also developed research in the area of cognitive cartography and critical thinking/knowledge engineering/concept maps in K-12 and higher education as well as in scientific research. He is also currently developing research work in the context of integrating concept maps with the concept of "adaptative assessment" with immediate and automatic feedback.

Hugo Faria has a bachelor's degree in biology (educational and scientific branch) from the University of Porto (UP), a master's degree in genetic resources and improvement of agricultural and forest species (somatic embryogenesis in narcissus triandrus, narcissus bulbocodium and narcissus rupicola) from University de Trás-os-Montes e Alto Douro, doctoral course in science of the UP (production of cardosins in in vitro systems-studies of intracellular expression and transport). He has been a secondary science teacher for 31 years and is CK-12 certified. His main research interest is in the use of model organisms (daphnia, planaria) in K-12 teaching, in concept mapping and computer adaptative assessment.

Figures

Figure 1. Concept map of the focus question, "What is advanced concept mapping coupled with logic branching multiple-choice Google Forms ACM-LBMCGF?"

Figure 2. Concept map of the focus question, "What is a Microbial Biofilm?"

Figure 3. The logic branching feature of Google Forms using ACM-LBMCGF as the assessment concept map.

Figure 4. logic branching feature of Google Forms using the Biofilm Formation as the assessment concept map.

Figure 5. Design of the algorithm using the logic branching feature of the Google Forms (ACM-LBMCGF as the assessment concept map.)

©2021 ACM  $15.00

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.