Testing a program's theory of change using QCA

Objective: Save the Children Thailand implemented a school-based education and advocacy project to promote road safety among children in Bangkok, Thailand. With a complex theory of change, the project tested and improved upon its package of interventions throughout the project. For its final evaluation, Save the Children wanted to understand which package of interventions was most closely linked to desired outcomes and which of its investments made the biggest difference.

Approach: We designed a mixed methods evaluation that used qualitative comparative analysis (QCA) to identify pathways to successful outcomes (i.e., increased motorcycle helmet use among students). We selected key measurable components of the project's theory of change and collected data from 44 school liaisons through a phone survey. QCA findings were supplemented with school site visits and statistical analysis of existing project data.

Result: QCA highlighted for our client which combinations of interventions were linked to desired outcomes and which interventions made the biggest difference for success. Based on these results, we recommended three tiers of action as a model for future programs.