A Mixed-Methods Evaluation of the Implementation of a Performance Measurement System for South Africa's Substance Use Treatment Services

Abstract

OBJECTIVE: Minimal knowledge exists on the factors that affect implementation of performance measurement systems, particularly in low- and middle-income countries (LMICs). To address this, we describe the implementation of a performance measurement system for South Africa's substance abuse treatment services known as the Service Quality Measures (SQM) initiative.

METHOD: We conducted a mixed-methods evaluation of system implementation. We surveyed 81 providers about the extent of system implementation within their agencies and the factors that facilitated implementation. We conducted 26 in-depth interviews of providers' perceived barriers and facilitators to implementation.

RESULTS: The overall penetration of this system was high. Almost all providers viewed the system as feasible to implement, acceptable, appropriate for use in their context, and useful for guiding service improvements. However, the extent of implementation varied significantly across sites (p < .05). Leadership support (p < .05) was associated with increased implementation in multivariable analyses. Providers reflected that high rates of patient attrition, variability in willingness to implement the system, and limited capacity for interpreting performance feedback affected the extent of system implementation.

CONCLUSIONS: It is feasible to implement a performance measurement system in LMICs if the system is acceptable, appropriate, and useful to providers. To ensure the utility of this system for treatment service strengthening, system implementation must be optimized. Efforts to enhance target population coverage, strengthen leadership support for performance measurement, and build capacity for performance feedback utilization may enhance the implementation of this performance measurement system.