Online Assessment System with Integrated Study (OASIS) to enhance the learning of Electrical Engineering students: an action research study

Abstract

World-wide, there has been a large increase in tertiary student numbers, not entirely matched by funding increases. Consequently, instructors are faced with large, diverse classes, and find themselves struggling to provide adequate assessment and prompt feedback, two quantities critical in an effective learning environment. Personal computers and the Internet can help solve this problem. The aim of this study was to develop, implement and validate a Web-based software package that, through providing practice and assessment opportunities, improved student learning and reduced marking and related mundane aspects of instructor workload. At the start of the study, such a package already existed in prototype form: OASIS (Online Assessment System with Integrated Study). As the study progressed, this software package was first fully rewritten and then repeatedly modified. OASIS delivers individualised tasks, marks student responses, supplies prompt feedback, and logs student activity. Staff can deliver sets of practice questions and assessments to students: assessments may involve different questions for different students, not just numerically different versions of the same questions. Given my role as teacher, the traditional research ideal of observing without affecting the research environment was both impossible and unconscionable. In particular, since preliminary evidence suggested that OASIS did enhance student learning, I could not adopt a ‘two groups’ approach to the research, with one group using OASIS while the other did not. Instead, an action research methodology was seen as most appropriate for my double role of teacher and researcher.This methodology enabled me, in the light of my findings, to continuously modify the learning environment and enhance student learning. The action research proceeded through a spiral of one-semester cycles of planning, acting, observing and reflecting. To maximize rigour, the research ran through eight cycles over four years and involved considerable triangulation. OASIS itself collected much quantitative data. Further data were collected via interview, survey, email and informal discussion from three groups: current students, postgraduates and academics. My colleagues provided alternative perceptions and interpretations, as did Physics Department academics who were using OASIS, and an external academic who interviewed academics and investigated the implementation of OASIS. Perhaps surprisingly, academics had generally adopted OASIS to promote student learning rather than to decrease their own workloads. In some cases workloads were reduced; however, where OASIS assessments augmented rather than replaced existing traditional assessments, workloads actually went up slightly. All instructors who used OASIS reported enhanced student learning and wished to continue using it. Student surveys, interviews, focus-group discussions and informal feedback showed that students found the software easy to use and considered that it helped them improve their skills and understanding. OASIS questions were preferred over textbook questions. Students commonly requested OASIS to be available in more of their areas of study. In general students wanted hints or model answers though some argued against their provision.The majority of students were enthusiastic about the use of OASIS for practice, and activity logs revealed that they did use OASIS extensively. These logs also revealed the motivating power of assessments: typically half the online practice activity took place in the last 36 hours prior to assessments. Interviews provided further interesting insights into the ways different students approached their studies and assessments. However, students did voice concerns about the validity of OASIS assignments, noting their peers could rely on the efforts of others to score highly in these. A number of steps were carried out in an attempt to defuse these concerns, including: disabling OASIS practice during assignments, basing assignments on previously unseen questions, and providing different assignment questions to different students. While this study has achieved the goal of developing, implementing and validating OASIS, many future opportunities exist. OASIS may be used in schools as well as universities. Non-numerical questions, where answers may be somewhere between right and wrong, are possible. OASIS can also be used to deliver concept inventories to students to support research into concept acquisition and retention.