Automating board-game based learning. A comprehensive study to assess reliability and accuracy of AI in game evaluation

Issue title: Selected papers from the AIxIA 2023 Workshops

Guest editors: Andrea Brunello and Danilo Croce

Article type: Research Article

Authors: Tinterri, Andrea^{a; *} | Pelizzari, Federica^b | di Padova, Marilena^c | Palladino, Francesco^d | Vignoli, Giordano^e | Dipace, Anna^{f; 1}

Affiliations: [a] Department of Human Sciences, IUL Telematic University, Florence, Italy | [b] Department of Pedagogy, Catholic University of the Sacred Heart, Milan, Italy | [c] Department of Humanities, Letters, Cultural Heritage and Educational Studies, Foggia, Italy | [d] University of Modena and Reggio Emilia, Modena, Italy | [e] IESS, European Institute for Superior Studies, Reggio Emilia, Italy | [f] Faculty of Human Sciences, Education, and Sport, Pegaso Telematic University, Naples, Italy

Correspondence: [*] Corresponding author: Andrea Tinterri, E-mail: atinterri@iuline.it.

Note: [1] Conceptualization, A.T. and F.Pe.; introduction and theoretical framework, A.T..; methodology and context of application, F.Pe.; data acquisition, M. di P.; data analysis, A.T., game evaluation G.V., F.Pa. and F.Pe; writing—original draft preparation, A.T. and F.Pe.; writing—review and editing, A.T., M. di P., F.Pa., F.Pe., G.V., and A.D.; supervision, A.D. All authors have read and agreed to the published version of the manuscript.

Abstract: Game-Based Learning (GBL) and its subset, Board Game-Based Learning (bGBL), are dynamic pedagogical approaches leveraging the immersive power of games to enrich the learning experience. bGBL is distinguished by its tactile and social dimensions, fostering interactive exploration, collaboration, and strategic thinking; however, its adoption is limited due to lack of preparation by teachers and educators and of pedagogical and instructional frameworks in scientific literature. Artificial intelligence (AI) tools have the potential to automate or assist instructional design, but carry significant open questions, including bias, lack of context sensitivity, privacy issues, and limited evidence. This study investigates ChatGPT as a tool for selecting board games for educational purposes, testing its reliability, accuracy, and context-sensitivity through comparison with human experts evaluation. Results show high internal consistency, whereas correlation analyses reveal moderate to high agreement with expert ratings. Contextual factors are shown to influence rankings, emphasizing the need to better understand both bGBL expert decision-making processes and AI limitations. This research provides a novel approach to bGBL, provides empirical evidence of the benefits of integrating AI into instructional design, and highlights current challenges and limitations in both AI and bGBL theory, paving the way for more effective and personalized educational experiences.

Keywords: Board game-based learning, artificial intelligence in education, pedagogical frameworks, educational game design

DOI: 10.3233/IA-240030

Journal: Intelligenza Artificiale, vol. 18, no. 1, pp. 103-119, 2024