Computer managed learning assessment in higher education: the effect of a practice test.

Abstract

This thesis reports the results of studies set up to investigate formative assessment in the context of a computer managed learning (CML) practice test. The studies sought to determine whether taking the practice test affects performance on later CML assessed tests for first year university students and to determine the characteristics of the most effective CML practice test. The study was carried out in the context of CML testing at Curtin University of Technology. Because data were collected in a real testing environment, the research questions were addressed using a series of small studies, each focusing on a one-semester unit for first year students. Those students who sat a practice test improved their performance from the practice to the assessed test. Further, they outperformed the non practice test group on the assessed test. The effect was statistically significant in eleven of the twelve studies where CML test results were investigated. Student ability, anxiety level, and sex did not affect test performance or choice to sit the practice test. Students preferred to be given the correct answer for an incorrect response and to have a practice test that was the same length as the assessed test but students continued to show improved performance regardless of these conditions. They reported that they used the feedback in a variety of ways including identifying important areas of content, identifying their own error areas and as a motivator for further study. The findings suggest that using the CML system as a formative assessment tool improves student performance on summative assessment. The practice test is contributing to improved performance, however this improvement cannot be attributed to a single factor. In those cases where the practice test only partially covers the content of the assessed test, the improvement is seen on that common part, however when there was no overlap of content the group who did the practice test still performed better on the assessed test than the group who did not. This suggests that a contributing factor may be familiarity either with the CML system, items or test type. It is also possible that the beneficial effect was due to prior exposure to the CML system and that only one test is required for this purpose.This research has implications for current teaching practices because the acceptance of a practice test provides feedback to both students and lecturers prior to the assessed test. The optimal practice test c covers the same content as the assessed test with the same number of items and provides the correct answer for a item answered incorrectly. The key recommendation for use of the CML system is the provision of a practice test for formative purposes, for the use of both lecturers and students. Lecturers need to encourage student participation not just on an initial practice test but on all practice tests provided. Students need to be encouraged to review their error summary, as is the current practice in the CML Laboratory. Lecturers need to make more use of the feedback provided by the tests, in terms of content coverage, revision and consolidation of work, and quality of test items.