Evaluation of classroom performance system (CPS) technology integration in terms of classroom environment and attitudes

Abstract

The push for technology in education might not be as successful as some 21st century policy makers might expect. Numerous researchers have failed to find any positive outcomes following the integration of technology into education. This study compared classrooms in which Classroom Performance System (CPS) technology had been integrated with non-CPS classrooms in terms of the classroom learning environment and students’ attitudes toward science. Student perceptions of the learning environment were assessed with a modified version of the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI) and student attitudes were assessed with the Attitude and Efficacy Questionnaire. These instruments were administered to a sample of 971 students as a pretest and 389 students as a posttest with 17 different teachers, of students in grades 9 through 12, from a large school district in New York State.The gender breakdown for this sample consisted of 179 males and 210 females, and there were 178 CPS students and 211 non-CPS students. The differential effectiveness of CPS technology for different genders was also investigated.Simple correlation and multiple regression analyses were used to examine the relationships between student perceptions of the classroom environment and the student attitudes. The analysis involved a two-way MANOVA with the TROFLEI scales and attitude scales as the dependent variables. The two independent variables were the instructional method (CPS and non-CPS) and student gender. Differential effectiveness was considered to exist if there was a significant instruction x gender interaction for a particular dependent variable.Because the MANOVA using the Wilks’ lambda criterion showed significant differences for the set of dependent variables as a whole, the univariate ANOVA was interpreted for each individual environment and attitude scale. The average item mean, average standard deviation, F values and effect sizes from MANOVA were calculated for each of the scales of the TROFLEI and Attitude and Efficacy Questionnaire. Overall, students in the CPS class appear to have benefited somewhat from the CPS technology integration. To examine the magnitudes of the differences between instructional group, as well as their statistical significance, effect sizes were calculated in terms of the differences in means divided by the pooled standard deviation. The effect size for Equity was relatively small with a value of 0.21 standard deviations. This result suggests differences in students’ perceptions of Equity in the classroom learning environment between students using CPS clickers and those students that were not using the clickers.Although gender differences in learning environment perceptions and attitudes were not the focus of the research questions, some statistically significant results were revealed. Results show that both Task Orientation and Attitude to Subject yielded significant gender differences. These effect sizes indicate gender differences that are small in magnitude. Relative to males, females liked science more and perceived that their classes were more task oriented.The results from this study revealed a significant instruction-by-gender interaction for Computer Usage. For the experimental group, males perceived greater Computer Usage than females. However, for the control group, females perceived greater Computer Usage.Data analyses supported the TROFLEI’s and Attitude and Efficacy Questionnaire’s factorial validity, internal consistency reliability, and ability to differentiate between the perceptions of students in different classrooms. All TROFLEI scales correlated significantly and positively with student attitudes.