Multiple-choice questions compared to short-answer response: which assesses understanding of chemistry more effectively?

Abstract

The research inquires into the effectiveness of the two predominant forms of questions that are used on the Victorian Certificate of Education (VCE) Victoria Chemistry examination. These are multiple-choice questions and short-answer questions. This research examines not only the style of chemistry question but also the content type examined (recall and application questions) along with gender differences in students’ responses to such questions. The research involved three phases, i) analysis of five years results from the VCE Chemistry examinations, ii) class trial testing students of both genders with structured questions that examined the same material content with each type of question (multiple-choice or short answer) and also examined the different type of chemistry content (recall or application) and iii) interviews with students and teachers.The first phase of the research analysed the available published VCE Chemistry results for the five years, 2003 to 2007. The findings of these data yielded statistically significant differences between the performances of students based on the type of question (multiple-choice or short-answer) and the content of the question. The second phase of analysis yielded comparative data to the VCE analysis but also provided detailed Rasch analysis of the question type and content as well as gender differences in performance.Important findings were: i) student performance on multiple-choice chemistry questions was significantly higher than performance on short-answer questions regardless of the content and ii), the performance of males was significantly higher than that of females in upper levels of achievement but not at the lower levels of achievement. Possible factors accounting for the observed difference were noted. Implications of these findings are discussed as well as suggestions for further research.