Toward Automated Computer-Based Visualization and Assessment of Team-Based Performance

Abstract

A considerable amount of research has been undertaken to provide insights into the valid assessment of team performance. However, in many settings, manual and therefore labor-intensive assessment instruments for team performance have limitations. Therefore, automated assessment instruments enable more flexible and detailed insights into the complex processes influencing team performance. The central objective of this study was to advance knowledge in automated assessment of team-based performance using a language-oriented approach. Fifty-six teams of learners (N = 224) in 3 experimental conditions solved 2 tasks in an online learning environment. They were analyzed with the Automated Knowledge Visualization and Assessment (AKOVIA) methodology. AKOVIA integrates a natural language-oriented algorithm and enables a structural and semantic compression of individual- and team-based knowledge representations. Findings indicate initial evidence of the feasibility and validity of the fully automated methodology. A framework for integrating research and methodology development is suggested for improving educational technology innovations such as computer-based assessment environments in international large-scale assessments.