Process evaluation of a child pedestrian injury prevention intervention.

Abstract

The Child Pedestrian Injury Prevention Project (CPIPP) is a rigorous school- and community-based intervention trial delivered to 2,500 children in their second, third and fourth year of schooling in three communities in Perth, Western Australia, from 1995 to 1997. The CPIPP was designed to improve children's pedestrian safety knowledge, their road related behaviors - crossing and playing, and to reduce their risk in, and exposure to, traffic. This thesis addresses the process evaluation of the CPIPP school-based intervention. The Curtin University Human Research Ethics Committee provided ethics approval for this project.Evaluation of previous school-based pedestrian safety programs has focused mainly on assessing outcomes with little or no process evaluation. An absence of process evaluation increases the likelihood of Type III error, that is, incorrectly attributing null or weak outcomes to a program that has not been adequately implemented.In each of the three study years, following a teacher training, teachers were asked to implement the school-based intervention. Each year this comprised nine 40-minute pedestrian safety lessons and home activities. Lessons included road crossing practise on real and simulated roads.Data were collected from the student cohort (n=1049) and their Grade 2, 3 and 4 teachers. Four process evaluation instruments were developed and administered in each of the three study years. These included one student instrument (work samples) and three teacher instruments (lesson log, teacher post-implementation questionnaire and classroom observation). Student outcome data including their pedestrian-related knowledge and road crossing and playing behaviours were assessed using a pre- and post-test self report questionnaire.The majority of teachers (70-97%) and students (72-84%) responded positively to questions about their satisfaction with the CPIPP Grades 2, 3 and 4 curricular. Evidence in student work samples demonstrated that teachers taught 76% (seven of nine lessons) of the Grades 2 and 3 curricular, and 68% (six of the nine lessons) of the Grade 4 curricular. Teacher self-reported implementation rates using a 'lesson log' were 88%, 81% and 60% respectively for the three curricular. Teachers reported practising road crossing on a real road in 21% (one lesson) of six designated crossing practise lessons in 1996 and 36% (two lessons) in 1997.Multivariate analyses revealed students pedestrian safety knowledge was significantly associated with teacher implementation of the classroom curriculum. This relationship was one of dose-response. It demonstrated students who, each year, received at least 7 lessons (81% or more) of the three CPIPP curricular showed a greater improvement in pedestrian safety knowledge than those students who received a lower dose of the curriculum. Significant effects on pedestrian safety knowledge were also observed for students who, each year, practised crossing a real road in at least one lesson (17%) of the curriculum. The relationship between implementation and student road crossing and road playing behaviours was not one of dose-response.Student work samples, teacher lesson logs and to a lesser extent teacher self-report questionnaires, were found to be valid measures of curriculum implementation. This study also found that implementation of the CPIPP curriculum achieved a modest improvement in student pedestrian safety knowledge and possibly arrested the decline of safe road crossing behaviour. It also demonstrated that classroom pedestrian safety education alone, while necessary, is not sufficient to positively modify children's road crossing behaviours.The findings of this study demonstrate the importance of measuring teacher implementation. A process evaluation is essential to determine if an intervention has been implemented and to help explain the impact this level of implementation had on program outcomes. However, more research needs to explore the link between other factors in the process of curriculum delivery and program effects. Further research also needs to determine how to develop and measure an intervention that includes the key procedures and content that theoretically promote the desired behaviour, but also allows teachers to make adjustments to the program to suit their teaching style and the needs of their students.Child pedestrian injury is a complex problem that requires a multifaceted intervention, of which a classroom curriculum can form part.