Effectiveness of student response systems in terms of learning environment, attitudes and achievement

Abstract

Most past research on the effectiveness of Student Response Systems (SRS) has focused on higher levels of education and neglected consideration of the learning environment. Therefore, this study is unique in its focus on Grade 7 and Grade 8 students and on the effect of using SRS on students’ perceptions of the learning environment, as well as on the student outcomes of attitudes and achievement. This study also validated a new questionnaire, the How Do You Feel About This Class? (HDYFATC), which incorporates a new learning environment scale (Comfort) developed by the researcher. As schools incorporate technology such as SRS into the classroom, it is important to evaluate its effectiveness in terms of students’ perceptions of the learning environment, attitudes, and achievement.Student perceptions of the learning environment and their attitudes were assessed with the HDYFATC, which combines four learning environment scales (Involvement, Task Orientation, Equity, and Cooperation) from the What Is Happening In this Class? (WIHIC) questionnaire with one created by the researcher (Comfort) to assess how comfortable students are in their science class, and an attitude scale (Enjoyment) from the Test of Science Related Attitudes (TOSRA). Students’ achievement was assessed using the average of their examination scores for the duration of the study.The HDYFATC was administered to a sample of 1097 Grade 7 and Grade 8 students from 47 classes in three schools in New York State. Data analyses supported the HDYFATC’s factorial validity, internal consistency reliability, and ability to differentiate between the perceptions of students in different classrooms. All items had a factor loading of at least 0.40 on their a priori scale and less than 0.40 on all other scales. The total variance was 76.13%, with the largest contribution from the Enjoyment scale. Eigenvalues ranged from 1.29 to 25.62. When the individual was used as the unit of analysis, the internal consistency reliability for different scales of the HDYFATC ranged from 0.94 to 0.95. ANOVA revealed significant differences between students’ perceptions in different classes for each learning environment scale, with eta² values ranging from 0.50 to 0.60 for different scales.To determine the effectiveness of SRS in terms of learning environment, attitudes, and achievement, data obtained from the HDYFATC and achievement scores were subjected to a MANOVA. The dependent variables were the five learning environment scales and two student outcome scales, while use or non-use of SRS was the independent variable. Because the multivariate test using Wilks’ lambda criterion yielded a statistically significant result overall for the whole set of seven dependent variables, the univariate ANOVA results were interpreted separately for each individual dependent variable. The F value for between-group differences was statistically significant for every scale. Very large effect sizes ranged from 1.96 to 2.46 standard deviations for the learning environment scales and were 2.19 and 1.17 standard deviations for attitudes and achievement. For every scale, the SRS group had higher scores than the comparison group.A two-way MANOVA was used to determine if the use of SRS was differentially effective for males and females. The independent variables were the use/non-use of SRS and gender, and the dependent variables were the seven learning environment and student outcome scales. Although both males and females benefited from the use of SRS, Task Orientation was the only scale for which a statistically significant interaction emerged. However, the degree of differential effectiveness found for males and females when using SRS was small and of very little educational importance. Females appeared to benefit slightly more than males from the use of SRS.Simple correlation and multiple regression analyses were used to investigate the relationships between students’ perceptions of the learning environment and the student outcomes of attitudes and achievement. All five learning environment scales correlated positively and significantly with both student attitudes and achievement. The multiple correlation of the five learning environment scales with student attitudes and achievement was, respectively, 0.79 and 0.45. Involvement, Task Orientation, and Comfort were statistically significant independent predictors of student attitudes, while Involvement, Equity, and Comfort were statistically significant independent predictors of achievement.