Investigating the effectiveness of a professional development initiative for lower secondary teachers in Indonesia

Abstract

The overarching aim of this study was to evaluate the effectiveness of a teacher professional development initiative, designed to help lower secondary teachers to improve their teaching practice. To evaluate the effectiveness of the programme, a model of evaluation, influenced by three existing models, was developed. This holistic model was designed to suit the Indonesian context and included five distinctive phases. The first phase involved the collection of baseline data related to the teachers’ classroom practices and students’ perceptions of their learning experiences and attitudes. The second phase involved evaluating the teachers’ views of the relevance and utility of the professional development programme. The third phase involved examining the extent to which the teachers’ translated the knowledge and skills, imparted during the programme, into practice. The fourth phase involved examining whether there were changes in students’ perceptions of their learning experiences and attitudes over the duration of the one year programme. Finally, the fifth phase sought to examine whether there were contextual factors that might promote or impede the translation of the professional development programme into practice. These five phases were used to guide the collection and analysis of data.The sample for the evaluation involved different grain sizes. Quantitative data were collected from 2,417 students (drawn from 66 classrooms in 32 Indonesian lower secondary schools) who were present for both the pre‐test and the post‐test. Qualitative data, in the form of reflective journals, was collected from the 138 teachers who were involved in the professional development programme. In‐depth interviews were collected from 33 teachers (from whose classes the quantitative sample was drawn). Finally, six of these 33 teachers formed the basis of case studies that involved the collection of observation and in‐depth interview data.Data collected from teacher reflective journals and interviews with teachers were used to evaluate the teachers’ views of the relevance and utility of the teacher professional development programme (Phase 2 of the evaluation model). The findings indicated that the teachers generally perceived the professional development experience positively, however, they were critical about two aspects of the delivery, these being the calibre of two of the four trainers and the tight time schedule required to complete the programme.The findings suggested that there were differences in how the teachers translated the professional development ideas into their classroom practices (Phase 1 and 3 of the evaluation model). Two of the case study teachers, both of whom were teaching in urban schools, attempted to change their teaching practices to be more studentcentred. Meanwhile, the teaching practices of the remaining four case study teachers (one from an urban school and three from rural schools) continued to be predominantly teacher‐centred.To examine the pre–post differences in students’ perceptions of their learning experience and attitudes, MANOVA with repeated‐measures were performed using the sample of 2,417 students. The results indicated that there were statistically significant (p<0.01) differences for the six of the seven WIHIC scales, but not the attitude scale, with all statistically significant differences showing an improvement for the post‐test results. Despite the statistical significance for these WIHIC scales, the effect sizes were small (according to Cohen’s criteria), suggesting that the success of MGMP professional development programme was limited.The observations made at Phase 3 of the evaluation indicated that the MGMP Empowerment programme may have been more effective for teachers in urban schools than their counterparts in rural schools. A two‐way MANOVA, with the WIHIC and attitude scales as the set of dependent variables and the two independent variables involving a two‐level variable based on location (urban and rural) and two‐level repeated‐measure testing occasion pre‐test and post‐test indicated that, for six of the seven learning environment scales (the exception being the Involvement scale) and the attitude scale, a statistically significant interaction between testing occasion and location existed. For the WIHIC scales with a statistically significant interaction, in all cases, a locality gap existed at the pre‐test, which increased at the post‐test. For these scales, students in urban classrooms scored higher for the pre‐test and the post‐test than their rural counterparts. A comparison of the effect size indicated that, the magnitude of the changes were greater for the students in urban schools than their rural counterparts.The results indicated that there were three contextual factors that influenced the teachers’ efforts to translate the ideas of the professional development programme into practice, these being, the instructional context, the lack of support provided by the school administrators and the nation‐wide leaving examination. Whereas the case study teachers in urban schools felt that the status of the schools and the socio‐economic status of the families of the students supported their willingness to try out the new ideas, the teachers from rural schools claimed that issues related to student discipline, large classroom sizes and inadequate resources and facilities hindered their attempts to implement the new ideas in their classrooms. All of the case study teachers indicated that there was inadequate practical support from either the principals or superintendents. Finally, the nation‐wide leaving examination was perceived as the main hindrance for all six case study teachers. Although student‐centred approaches have been mandated in the national curriculum, it appears that the high‐stakes national examination forced the teachers to continue to use teacher‐centred approaches, as these approaches were viewed as more effective.The findings of this study have resulted in the development of a professional development model that brings together the field of learning environments with the evaluation of professional development suitable for use in Indonesia and it is anticipated that the findings will contribute to a better understanding of how teacher professional development in Indonesia can be improved. As such, the model can be used in the evaluation of future professional development programmes in Indonesia.