Review of Methods for Evaluating Components of Social Interventions

Design Options for an Evaluation of Head Start Coaching


This report is part of a larger effort to design a study that answers the question: What is the effect of individual coaching components on teachers and children participating in Head Start? Evaluating multi-component social interventions as a whole makes it difficult to determine which individual components are most effective and economical – leaving evaluators, policymakers, and program developers with a “black box”. This report reviews potential experimental design options that get inside the “black box” of social interventions by estimating the effects of individual components. Five experimental designs are discussed: factorial designs, comparative treatment designs, the individual experiments design, crossover designs, and adaptive clinical trials. Individual components of coaching interventions in Head Start contexts are used as an example throughout the report to describe and compare these designs. The review concludes that factorial designs are usually the most appropriate design for evaluating the effects of individual components.