Enhanced Approach of Automatic Creation of Test Items to Foster Modern Learning Setting

Abstract

Assessment has to be seen as an integrated and important activity in the learning process. In particular modern educational approaches - such as self-directed or exemplary learning - and per-sonalized learning activities cause a tremendous effort or make it even impossible to prepare appro-priate and individualized test items, assess them and provide feedback. This situation has motivated the Advanced Educational Media Technologies (AEMT) Group at Graz University of Technology to initiate a research program on e-assessment to cover the entire life cycle of the assessment process by semi-automated and automated approaches. In this paper we focus on the automated process to create different types of test items out of textual learning content, more precisely to create single choice, multiple-choice, completion exercises and open ended questions. Started in previous re-search activities by applying statistic approaches to summarize learning content and to identify most important words, our most recent approach applies a combination of statistic and natural language processing as well as semantic methods to identify most important concepts on an abstracted level.In our enhanced approach, identified concepts and differently related concepts represent the underpinning input for the test item creation. The implemented prototype can process learning content stored in various file formats, extracts most important content and related concepts, creates different types of test items and reference answers, and supports online assessments as well as exports the test items in QTI format. In this paper we cover in particular the following aspects: the motivation for automated test item creation, related work, requirement and design of the enhanced tool, implementation and usage viewpoints. Furthermore, we outline a study on the evaluation of the prototype, which suggests that the quality of extracted concepts and created test items is comparable with such ones provided by humans.