Nonrobustness of the carryover effects of small classes in project STAR

Abstract

© by Teachers College, Columbia University.Background: Class size reduction (CSR) is an enduring school reform undertaken in an effort to improve academic achievement and has been widely encouraged in the United States. Supporters of CSR often cite the positive contemporaneous and carryover effects of Project STAR. Much has been discussed regarding the robustness of the contemporaneous effects but not regarding that of the carryover effects. Purpose: This article checks the robustness of the carryover effects of STAR's small classes. Setting: STAR was undertaken in 75-79 schools in Tennessee. Participants: Each year in the experimental period, 6,000-7,000 students in grades K-3 participated in the experiment, for a total of 12,000 students during the entire period. Intervention: As students initially entered STAR schools, they were (arguably) randomly assigned to small classes with 13-17 students, regular classes with 22-25 students without teacher aides, and regular classes with 22-25 students with teacher aides. The experiment was performed from 1985 through 1989, but information on STAR students continued to be collected thereafter. Research Design: STAR is a randomized controlled field experiment. Data Analysis: In this article, STAR schools are divided into "effective" schools and "ineffective" schools. Effective schools are defined as schools where the test scores of students in small classes were statistically significantly higher than those of students in regular classes at the 5% level in both math and reading. By contrast, ineffective schools are defined as schools where the test scores of students in small classes were not statistically significantly higher than those of students in regular classes at the 5% level in either math or reading. Separately for effective schools, schools other than effective schools, and ineffective schools, the academic achievement of students is regressed on variables indicating small class assignment, along with student characteristics and school-by-entry wave fixed effects. Findings: The carryover effects of CSR are not robust; they are driven mostly by effective schools, which account for at most a quarter of STAR schools. During this investigation, it is revealed that, in contrast to the protocol of randomization, observable student characteristics in these schools are not randomly distributed between small and regular classes. They are instead distributed in such a way as to increase the academic achievement of students in small classes and decrease that of students in regular classes. Recommendations: Caution is recommended when citing the positive carryover effects of STAR.