What is Research on Evaluation (RoE)?
The RoE TIG convened a working group to help promote the field of RoE, and one of our first tasks was to come to a consensus on the definition of RoE.
There are many definitions of research on evaluation (RoE), which can make it difficult to understand. Through searching the literature and discussing with colleagues, we found four definitions (presented chronologically):
[Any] ... systematic inquiry into the methods, practices, and profession of program evaluation, with potential implications of its findings for evaluation theory (Brandon & Fukunaga, 2013).
Systematic empirical inquiry resulting in original findings or reexaminations of existing data about the practice, methods, or profession of program evaluation (Brandon, 2015).
Any purposeful, systematic, empirical inquiry intended to test knowledge, contribute to existing knowledge, or generate new knowledge related to some aspect of evaluation processes or products, ore valuation theories, methods, or practices (Coryn et al., 2015).
A research investigation that generates findings with the intended purpose of creating a stronger evidence base and infrastructure for the applied practice of evaluation (Fierro, working definition).
The first three definitions have a lot in common: they are focused on systematic inquiry using empirical methods aimed at examining a variety of aspects of evaluation, including the methods, practices and processes, products, theories, and profession.
The Brandon and Fukunaga (2013) definition also discusses implications for evaluation theory and the Coryn et al. (2015) definition discusses implications of testing knowledge, contributing to existing knowledge, or generating new knowledge. Essentially, these definitions point to RoE as research for research sake.
But the Fierro working definition suggests that RoE is for more than just creating a stronger evidence base, one that seems built by and for researchers. Rather, RoE is also for building an “infrastructure for the applied practice of evaluation” (emphasis added).
It is for that reason we adopted the Coryn et al. (2015) definition in conjunction with the Fierro working definition. The definition might look something like this:
“Research on Evaluation (RoE) is any purposeful, systematic, empirical inquiry intended to create a stronger evidence base and infrastructure for the applied practice of evaluation. ”
We believe this broader definition both (a) encompasses everything the Brandon and Fukunaga (2013), Brandon (2015), and Coryn et al.(2015) definitions propose and (b) emphasize the importance of RoE for researchers and evaluators alike. RoE is not conducted for the sake of conducting it, nor is an evidence base of research important unless it is useful and used by the intended audience—in this case, practicing evaluators.
We hope this definition encourages researchers of evaluation to pursue topics that not only create a stronger evidence base but also informs evaluation practice.
Special thanks to everyone in the working group who has been involved in this so far: Kathleen Doll, Michael Harnar, Nina Potter, Eric Barela, Gregory Greenman, Esther Nolton, Ramjee, Miriam Jacobson, George Harrison, Seema Majato, Leslie Fierro, and more!
