Enhanced automatic question creator - EAQC: concept, development and evaluation of an automatic test item creation tool to foster modern e-education
MetadataShow full item record
Research in automated creation of test items for assessment purposes became increasingly important during the recent years. Due to automatic question creation it is possible to support personalized and self-directed learning activities by preparing appropriate and individualized test items quite easily with relatively little effort or even fully automatically. In this paper, which is an extended version of the conference paper of Gütl, Lankmayr and Weinhofer (2010), we present our most recent work on the automated creation of different types of test items. More precisely, we describe the design and the development of the Enhanced Automatic Question Creator (EAQC) which extracts most important concepts out of textual learning content and creates single choice, multiple-choice, completion exercises and open ended questions on the basis of these concepts. Our approach combines statistical, structural and semantic methods of natural language processing as well as a rule-based AI solution for concept extraction and test item creation. The prototype is designed in a flexible way to support easy changes or improvements of the above mentioned methods. EAQC is designed to deal with multilingual learning material and in its recent version English and German content is supported. Furthermore, we discuss the usage of the EAGC from the users’ viewpoint and also present first results of an evaluation study in which students were asked to evaluate the relevance of the extracted concepts and the quality of the created test items.Results of this study showed that the concepts extracted and questions created by the EAQC were indeed relevant with respect to the learning content. Also the level of the questions and the provided answers were appropriate. Regarding the terminology of the questions and the selection of the distractors, which had been criticized most during the evalu-ation study, we discuss some aspects that could be considered in the future in order to enhance the automatic generation of questions. Nevertheless the results are promising and suggest that the quality of the automatically extracted concepts and created test items is comparable to human generated ones.
Showing items related by title, author, creator and subject.
Guetl, Christian; Lankmayr, K.; Weinhofer, J. (2010)Assessment has to be seen as an integrated and important activity in the learning process. In particular modern educational approaches - such as self-directed or exemplary learning - and per-sonalized learning activities ...
Automatic Test Item Creation in Self-Regulated Learning: Evaluating Quality of Questions in a Latin American ExperienceWesiak, G.; Rizzardini, R.; Amado-Salvatierra, H.; Guetl, Christian; Smadi, M. (2013)The research area of self-regulated learning (SRL) has shown the importance of the learner’s role in their cognitive and meta-cognitive strategies to self-regulate their learning. One fundamental step is to self-assess ...
Evaluation of anthropometry activities for high school science: student outcomes and classroom environmentLightburn, Millard E. (2002)The study involved the evaluation of anthropometric activities for high school science. The activities actively engaged students in the process of gathering, processing and analyzing data derived from human body measurements, ...