Evidence-based evaluation of programme interventions to achieve positive community integration outcomes for adults with acquired brain injury

Abstract

Background. The growing population of people with acquired brain injury (ABI) requires a strong focus on clients to be integrated into the community in order to use their productive skills in society, to help them live with greater independence, and to reduce community expenditure. To date, there is limited theoretical and conceptual development of frameworks documenting the factors influencing community integration as it applies to adults with acquired brain injury. Furthermore, although there is considerable attention paid to community integration as a desired outcome, there exists no evidence-based framework explaining characteristic of successful community interventions for people with acquired brain injury. In addition, fidelity of ABI interventions is not considered in the literature against an evidence-based framework.Objectives. This study was completed in three phases aiming (1) to develop a framework for community integration for adults with post acute ABI; (2) to investigate characteristics of successful community integration programmes in order to develop an instrument to evaluate fidelity of community integration program for adults with ABI; and (3) to field test the instrument to (a) identify implementation issues; (b) explore the relationship between the themes and attributes across the three services; and (c) establish some aspects of the psychometric properties of the instrument.Methodology and Results. The first phase of the study aimed to obtain a framework to define community integration for adults with ABI. The Participant Groups included six researchers, seven health professionals, six policy makers, eight people with ABI and ten family members. Following a pilot study which examined understandability of the documentation and terms used in the study, the Policy Delphi survey method (Hasson, Keeney, & McKenna, 2000; Turoff, 2002; 1995) was applied which included three steps. Each step of the study began with a survey or a semi-structured interview. The data were analysed and the results were used in the next step. At the end of the first phase of the study, a framework to identify community integration for adults with ABI was developed. A broad literature review confirmed its themes. The community integration framework (CIF) included seven community integration themes and descriptors. The themes consisted of Relationships, Acceptance, Community access, Occupation, Being at home, Picking up life again, and Heightened risks and vulnerability.In the second phase of the study, a multi attribute utility (MAU) technique was applied (Camasso & Dick, 1993; Huber, 1974; Lewis, Johnson, & Scholl, 2003). The CIF which resulted from the first phase was used in this phase as a basis to identify the characteristics of programmes that contributed to the achievement of community integration for adults with ABI. This part of the study was completed using four stages. In each stage, surveys or semi-structured interviews were used to gather the participants‘ opinions on the characteristics of a successful community integration programme. Analysis of the results of each stage informed the next stage.In the first stage of the second phase the data (descriptors of community integration programmes) were categorised into seven clusters and 26 sub-clusters. In the second stage, the Participant Groups reviewed and examined the importance of the programme characteristics using a Likert Scale. This stage resulted in determining the priority weights of the items. In the third stage, an Expert Panel reviewed the results during a day-long working group. The Expert Panel including 10 people as a sub-group of the participant groups modified the themes, their descriptors and attributes, and then identified indicators for the attributes as the programme characteristics for community integration for people with ABI. The themes, their descriptors and attributes were compared with ABI outcome measures currently available in the literature.During the fourth stage, the themes and attributes were sent to the Participant Groups and the Expert panel to finally confirm and determine the relative importance of each theme and attribute. This was the final stage in the development of the programme assessment of community integration attributes (PACIA) which included seven themes and 21 attributes. The themes consisted of Person centred approaches and planning, Relationships, Working together, Development of skills, Community based practices, Support for service users, and Service setting and atmosphere. Each attribute was further defined with indicators. Sources of evidence were identified to enable attributes to be rated.The third phase of the study included a field study of PACIA, an examination of possible implementation issues with the tool, and an evaluation of some specific psychometric properties of PACIA. The field study aimed to examine the effectiveness of the evaluation process, and investigate the relationship between the themes and attributes across the three services.A trained evaluation team first individually rated each service by PACIA through observation, reading the documents, and interviews with the staff, service users, and families. A conciliation meeting followed to achieve consensus on the service ratings. The three services achieved a range of scores on the PACIA themes and attributes based on their different qualities. This phase also explored the ease of use and issues in the implementation of the instrument based on the raters‘ feedback, and examined inter-rater reliability, face validity, and content validity of PACIA. Within the limitations of the field test method, the study indicated that PACIA is a valid and reliable instrument to evaluate community integration programmes.Conclusion. Integration or re-integration into the community is a vital social objective for people with ABI for whom intensive medical rehabilitation may be followed by an uncertain pathway of longer-term rehabilitation. This study developed an evidencebased community integration framework (CIF) to describe community integration for adults with ABI. The research method surveyed a wide group of key stakeholders whose views formed the basis of the framework. The framework may be useful as a basis for making policy decisions to enhance community inclusion. The CIF provides an agreed description of a theory or model upon which the characteristics of programmes aimed to facilitate community integration were identified and the fidelity of those programmes were assessed. The other result of this study was the development of a fidelity instrument named as Programme Assessment of Community Integration Attributes (PACIA). While there are obviously additional characteristics of PACIA that remain to be researched, this evidence-based instrument appears to be valid and reliable to test the fidelity of community integration programmes. Community integration programmes can be evaluated more comprehensively by PACIA than has been the position in the past. Such an evaluation instrument may help existing programmes re-focus to provide more efficient services for people with ABI.