Publication: Towards Interoperability and Novel Methodological Approaches for Scalable Game-Based Assessment
Loading...
Date
2025-10-07
Authors
Gómez Moratilla, Manuel Jesús
item.page.secondaryauthor
Escuelas::Escuela Internacional de Doctorado
item.page.director
García Clemente, Félix Jesús ; Ruipérez Valiente, José Antonio
Publisher
Universidad de Murcia
publication.page.editor
publication.page.department
DOI
item.page.type
info:eu-repo/semantics/doctoralThesis
Description
Abstract
A medida que el siglo XXI progresa, surgen nuevas metodologías de evaluación que desafían los enfoques tradicionales para evaluar conocimientos, habilidades y comportamientos. La Evaluación Basada en Juegos (Game-Based Assessment, GBA) ha ido ganando atención durante los últimos años debido a su potencial para mejorar los enfoques de evaluación actuales, demostrando ser muy buenas alternativas a las formas convencionales de evaluación como las pruebas escritas.
Dado este gran potencial para transformar las prácticas de evaluación tradicionales, el objetivo principal de esta tesis es explorar y desarrollar enfoques metodológicos y técnicos que permitan utilizar las GBAs como una solución para ofrecer evaluaciones válidas y robustas. Específicamente, esta tesis plantea cinco objetivos: el primer objetivo propone un análisis y evaluación exhaustivos del campo de GBA, especialmente teniendo en cuenta el reciente aumento en la popularidad de los juegos y el aprendizaje digital. El segundo objetivo de esta tesis se centra en el diseño y desarrollo de un modelo semántico interoperable para datos de interacción de estudiantes con juegos. El tercer objetivo se orienta al diseño y creación de una arquitectura eficiente capaz de ejecutar GBAs a gran escala. En cuarto lugar se explora el uso de Inteligencia Artificial Explicable (XAI) para mejorar la predicción del rendimiento estudiantil. Por último, el quinto objetivo se centra en la optimización del proceso de etiquetado de datos para técnicas de IA en el ámbito de las GBAs mediante la creación de una herramienta práctica.
Para cumplir dichos objetivos, se ha seguido una metodología estructurada con pasos claramente definidos. En primer lugar, una revisión sistemática de 65 estudios recientes permitió identificar tendencias actuales, metodologías comúnmente utilizadas y brechas potenciales en la literatura. En segundo lugar, para abordar el problema de interoperabilidad identificado previamente, se diseñó e implementó un modelo ontológico para estandarizar distintos formatos de datos provenientes de diferentes juegos. Tras este desarrollo se realizó una evaluación formal y pruebas con datos reales, cumpliendo así con el segundo objetivo. Después, para abordar el tercer objetivo y proporcionar una arquitectura escalable, se desarrolló un sistema interoperable utilizando un motor de procesamiento de datos estructurados de código abierto como base, lo que permitió el procesamiento de grandes volúmenes de datos de forma eficiente. Posteriormente, se llevó a cabo una validación mediante un caso de estudio para demostrar la aplicación en escenarios del mundo real. El siguiente paso desarrolló un modelo de predicción del rendimiento del estudiante en un juego serio utilizando técnicas de Machine Learning (ML), incorporando interpretabilidad ya sea de forma inherente o a través de técnicas de XAI. Finalmente, el quinto paso de la metodología consistió en desarrollar una herramienta práctica diseñada específicamente para el etiquetado de distintos tipos de datos en entornos de GBA, con soporte para entradas de audio, video y datos de interacción.
Como principales conclusiones, esta tesis doctoral subraya el valor y potencial de las GBAs para la educación y la formación. Aunque hemos visto que es posible extraer e inferir conocimiento valioso de los datos de interacción de los usuarios, sigue siendo necesario avanzar hacia marcos de evaluación más estandarizados, ya que la actual especificidad limita la replicación de experimentos y la transferibilidad de los resultados a la práctica. Además, la revisión realizada también puso de manifiesto la importancia de diseñar juegos con diseños de calidad en lugar de las típicas implementaciones básicas, como la inclusión de cuestionarios durante distintas etapas del juego. Por último, cabe destacar la necesidad de métodos de evaluación más sofisticados que logren captar la complejidad y naturaleza contextual de ciertas habilidades y comportamientos.
As the 21st century progresses, new assessment methodologies are emerging and challenging traditional approaches for evaluating knowledge, skills, and behaviors. Game-Based Assessment (GBA) has been gaining increasing attention in recent years due to its potential to enhance current assessment practices. GBAs have been proved to be robust alternatives to conventional forms of assessment such as paper-and-pencil tests. Given this potential to transform traditional assessment practices, the main purpose of this Ph.D. thesis is to explore and develop methodological and technical approaches for delivering valid, meaningful, and adaptive assessment experiences. Specifically, this dissertation poses five objectives: the first objective proposes a comprehensive examination and evaluation of GBA field, particularly with the recent rise in the popularity of games and digital learning. The second objective of this Ph.D. thesis focuses on designing and developing an interoperable semantic model for log data. Then, the third objective is geared towards the creation of an efficient architecture capable of performing GBAs at scale. The fourth proposed objective aims to enhance student performance prediction by using Explainable AI (XAI). Last but not least, the fifth objective of this thesis seeks to optimize the data labeling process for AI techniques in the GBA domain through the creation of a practical tool. To achieve the proposed objectives, a clearly structured methodology was followed. Firstly, a systematic review of 65 recent studies in the GBA field addresses the first objective, facilitating the identification of current trends, commonly used methodologies, and potential gaps in the literature. Secondly, to address the interoperability issue identified in the review, we designed and implemented an ontology-driven model for standardizing various log data formats from different games. Then, a formal evaluation and real-world testing were conducted, therefore fulfilling the second objective. Thirdly, to provide a scalable architecture in which the ontology model could be integrated, an interoperable framework was developed using an open-source structured data processing engine as the basis to enable computation over large-scale datasets. The next step of the methodology fulfills the fourth objective and involved the development of a learner performance prediction model for a SG using Machine Learning (ML) techniques, incorporating XAI techniques to ensure model explainability. Finally, the fifth step of the methodology consisted of developing a practical tool specifically designed for labeling various types of GBA data, with support for audio, video, and game-event inputs. As main conclusions, this Ph.D. thesis underlines the value and potential of GBA for education and training. Although we have seen that valuable knowledge can be extracted from user interaction data, there remains a need for more standardized assessment frameworks, as the current specificity limits the replication of experiments and the transferability of results into practice. Moreover, the conducted review also revealed the importance of making good game designs instead of relying on vague implementations, such as embedding hidden questionnaires within the gameplay. Finally, it is also worth mentioning the need for more sophisticated assessment methods to appropriately capture the complexity and context-dependence nature of certain skills and behaviors.
As the 21st century progresses, new assessment methodologies are emerging and challenging traditional approaches for evaluating knowledge, skills, and behaviors. Game-Based Assessment (GBA) has been gaining increasing attention in recent years due to its potential to enhance current assessment practices. GBAs have been proved to be robust alternatives to conventional forms of assessment such as paper-and-pencil tests. Given this potential to transform traditional assessment practices, the main purpose of this Ph.D. thesis is to explore and develop methodological and technical approaches for delivering valid, meaningful, and adaptive assessment experiences. Specifically, this dissertation poses five objectives: the first objective proposes a comprehensive examination and evaluation of GBA field, particularly with the recent rise in the popularity of games and digital learning. The second objective of this Ph.D. thesis focuses on designing and developing an interoperable semantic model for log data. Then, the third objective is geared towards the creation of an efficient architecture capable of performing GBAs at scale. The fourth proposed objective aims to enhance student performance prediction by using Explainable AI (XAI). Last but not least, the fifth objective of this thesis seeks to optimize the data labeling process for AI techniques in the GBA domain through the creation of a practical tool. To achieve the proposed objectives, a clearly structured methodology was followed. Firstly, a systematic review of 65 recent studies in the GBA field addresses the first objective, facilitating the identification of current trends, commonly used methodologies, and potential gaps in the literature. Secondly, to address the interoperability issue identified in the review, we designed and implemented an ontology-driven model for standardizing various log data formats from different games. Then, a formal evaluation and real-world testing were conducted, therefore fulfilling the second objective. Thirdly, to provide a scalable architecture in which the ontology model could be integrated, an interoperable framework was developed using an open-source structured data processing engine as the basis to enable computation over large-scale datasets. The next step of the methodology fulfills the fourth objective and involved the development of a learner performance prediction model for a SG using Machine Learning (ML) techniques, incorporating XAI techniques to ensure model explainability. Finally, the fifth step of the methodology consisted of developing a practical tool specifically designed for labeling various types of GBA data, with support for audio, video, and game-event inputs. As main conclusions, this Ph.D. thesis underlines the value and potential of GBA for education and training. Although we have seen that valuable knowledge can be extracted from user interaction data, there remains a need for more standardized assessment frameworks, as the current specificity limits the replication of experiments and the transferability of results into practice. Moreover, the conducted review also revealed the importance of making good game designs instead of relying on vague implementations, such as embedding hidden questionnaires within the gameplay. Finally, it is also worth mentioning the need for more sophisticated assessment methods to appropriately capture the complexity and context-dependence nature of certain skills and behaviors.
publication.page.subject
Citation
item.page.embargo
1-ene-2999
Collections
Ir a Estadísticas
Este ítem está sujeto a una licencia Creative Commons. CC BY 4.0