A rasch modelling approach to measuring school need for psychological services

Abstract

The role and deployment of school psychologists in Western Australia has been reviewed a number of times since the establishment of services to schools. The current practice for the allocation of school psychologists to schools continues to rely on the student population of a school, its socioeconomic index and an appraisal of the school’s difficulty level. Psychological services are then allocated accordingly, with the decision-making mechanism based on an ad hoc conception of school need.The research reported in this study examined the issue of establishing the aspects or characteristics of schools which constitute a greater or lesser need for services and then attempts to measure this need in an objective, evidence-based manner. The various elements of school need for psychological services in Western Australia are posited to cluster around constructs extrapolated from the domains of service identified by Ysseldyke, Dawson, Lehr, Reschly, Reynolds, and Telzrow (1997), the delivery systems described by Oakland, Faulkner, and Annan (2005) and services delivered in Western Australia (Area Manager Student Services personal communication, 2007; Swan Education District Student Services Plan, 2009-2011; West Coast Education District Student Services Plan 2009-10). The constructs are characteristics of students, schools and teachers. These constituted the hypothesised theoretical framework for the study upon which the empirical investigation was based.The aims of the study are, first to make explicit the characteristics of schools, teachers and students that constitute concern and hence indicate the need for school psychological services. Second, to identify the characteristics of schools, teachers and students which differentiate the level of need for psychological services between schools and third, to demonstrate the validity of the instrument development process. The research questions were: 1. Can a rating scale instrument be developed to measure school personnel perceptions of their school’s need for psychological services? Specifically, in terms of measurement theory (Wright & Masters, 1982): (a) Was there uni-dimensionality? (b) Was there qualification? (c) Was there quantification? and (d) Was there linearity? 2. Is data from a measure of need for school psychological services associated with school demographic variables (e.g. socio-economic index)? 3. What facets of validity evidence described in the Wolfe and Smith (2007a and 2007b) framework are identifiable in the construction of a measure of school need for psychological services? Specifically: (a) Evidence of the content aspect; (b) Evidence of the substantive aspect; (c) Evidence of the structural aspect; (d) Evidence of the generalisability aspect; (e) Evidence of the interpretability aspect; (f) Evidence of the external aspect; and (g) Evidence of the consequential aspect?The methodology chosen for this research is quantitative and applies the principles of Modern Measurement Theory, using the Rasch Rating Scale model for measurement. The research was conducted in three developmental phases, with each phase building upon the preceding phase. First, theoretical framework refinement and item writing; second, developing and trialling a 120-item scale, leading to the third phase, the construction of a parsimonious 35-item scale which was used to measure the psychological service needs of a sample of schools. Careful documentation of the developmental process forms an important and necessary step in formulating a validity argument as evidence to support applications of the new measure. Furthermore, this has crucial implications for the credibility of any inferences that may be drawn from applying the instrument as an intended measure. In addition, publicly available school-level data were collected, such as socio-economic index, suspension and exclusion data, truancy, and students with individual behaviour management plans.The study identified examples of six aspects of validity evidence in the empirical investigation of school need for psychological services. This confirmed the usefulness of the multi-level theory of validity evidence postulated by Wolfe and Smith (2007a and 2007b). In addition, the analyses and graphical displays generated by the RUMM2020 computer program (Andrich, Sheridan, Lyne & Luo, 2005) proved invaluable in illustrating validity evidence. The measure of school need for psychological services was found to be significantly related to student suspensions data.The empirical findings of the study are discussed in the context of their application to informing decisions about the level of psychological services that should be provided to schools, congruent with measures of need for such services.