Implementation evaluations typically seek to document the design and implementation of an intervention, assess its strengths and weaknesses, and examine how participants experienced services – among other research objectives. Rigorous implementation evaluations can provide context for understanding and interpreting the results of impact and outcomes analyses, and can help inform efforts to replicate or scale successful interventions. TAACCCT third-party evaluators must address certain research questions in their implementation analyses, but may also seek to answer other research questions of interest. (Refer to the appropriate Solicitation for Grant Announcement (SGA) for more information.)
While by no means exhaustive, the following list provides links to examples of implementation evaluation resources TAACCCT third-party evaluators may find helpful including resources on logic models and theories of change. Neither the U.S. Department of Labor nor Abt Associates endorses these materials or their authors. Some materials are publicly available and hyperlinks have been provided. Others must be purchased either directly from the publisher or from a bookseller, or can be downloaded via journal access databases.
Disclaimer: Neither Abt Associates, nor its TAACCCT evaluation partners, is responsible for the contents of any “off-site” web page referenced from this server or from private, third-party, pop-up, or browser-integrated software or applications. You are subject to that site’s privacy policy when you leave the TAACCCT Evaluation website. We are not responsible for Section 508 compliance (accessibility) on other websites.
Implementation Evaluation
| Document Title/Link | Description | Author/Institute |
|---|---|---|
| A Guide to Implementation Research | This book offers comprehensive, practical advice on conducting implementation studies in the context of program evaluation. | Alan Werner |
| A Procedure for Assessing Intervention Fidelity in Experiments Testing Educational and Behavioral Interventions. | This paper elaborates a five-step procedure for assessing intervention fidelity in the context of randomized controlled trials. | Michael Nelson, David Cordray, Chris Hulleman, Catherine Darrow, and Evan Sommer |
| CLEAR: Guidelines for Implementation Studies | This document reviews the guidelines that CLEAR uses to review selected reports that describe the implementation process of a program or policy, or that measure implementation inputs and outputs to assess the quality and fidelity of implementation of a planned program. | U.S. Department of Labor's Clearinghouse for Labor Evaluation and Research |
| Conducting Mixed Methods | This technical note by USAID provides guidance on using mixed-methods for evaluations. | USAID |
| Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge | This article discusses a conceptual framework for fidelity of implementation and a suite of data collection tools designed to be used across multiple programs. | Jeanne Century, Mollie Rudnick, and Cassie Freeman |
| Measuring the Degree of Program Implementation: A Methodological Review | This paper discusses conceptual and methodological issues surrounding the measurement of the construct and degree of implementation, drawn from a review of 74 previous studies which included such measures. | Mary Ann Scheirer and Eva Rezmovic |
| Policy into Action: Implementation Research and Welfare Reform | This book offers thinking on implementation research from leading policy researchers and evaluation practitioners. | Mary Clare Lennon and Thomas Corbett (The Urban Institute) |
| Rigorous Measures of Implementation: A Methodological Framework for Evaluating Innovative STEM Programs | This abstract details the practical need for multidimensional measures of fidelity of implementation of reform-based STEM instructional materials. | Amy Cassata-Widera, Jeanne Century, and Dae Kim |
Theory of Change/Logic Model
| Document Title/Link | Description | Author/Institute |
|---|---|---|
| Fully Articulating Your Vision: Using Logic Models to Support Innovation | A tutorial from the U.S. Department of Labor how to create a logic model. | DOL |
| Logic Models: A Tool for Designing and Monitoring Program Evaluations | A product of the Regional Educational Laboratory Pacific, this quick reference guide provides a brief introduction to logic models. | Brian Lawton, Paul R. Brandon, Louis Cicchinelli, and Wendy Kekahio |
| Logic Models: A Tool for Telling your Program’s Performance Story | This paper describes a logic model process and how managers can use it to develop and tell the performance story for their program. | John A. McLaughlin and Gretchen B. Jordan |
| Logic Model Development Guide | W.K. Kellogg Foundation created this guide to provide practical assistance to nonprofits engaged in demonstrating the effectiveness of their program. | W.K. Kellogg Foundation |
| The Logic Model Guidebook: Better Strategies for Great Results | This book offers step-by-step support for creating logic models and the modeling process in a range of contexts. | Lisa Knowlton and Cynthia Phillips |
| Theory of Change: A Practical Tool For Action / Results / Learning | This 2004 report prepared for the Anne C. Casey Foundation discusses the basics of the theory of change mapping. | Jane Reisman and Anne Gienapp (Organizational Research Services) |
| Theory of Change Basics – A Primer on Theory of Change | This guide describes the key components of a theory of change. | Dana Taplin and Helena Clark |
