From Learning to Action: Reflections from Eval17

I am home from Eval17 and wanted to reflect on my experiences at the conference. There were many interesting sessions I attended, but three things have really stuck with me.

Evaluators as Knowledge Brokers

I intentionally attended multiple sessions exploring knowledge brokering—and especially evaluators as knowledge brokering. This is something my advisor has considered for some time and those of you who follow me closely on Twitter know that I am taking up her call for research on the topic by choosing to study it for my dissertation.Our field has focused on evaluation use for decades, but it doesn’t seem until recently that researchers began focusing on research use after realizing much of their research remains in the ivory tower of academia. They have just “discovered” Weiss’s taxonomy of types of use and are exploring methods to bridge the research-practice gap. However, as I will argue in my dissertation, evaluators may be better suited to act as knowledge brokers between researchers and practitioners than researchers themselves. Many evaluators have one foot in the world of research and another in practice (with their mind, body, and spirit situated firmly in evaluation!) and have the interpersonal competencies, know participatory styles of evaluation, and have the ability to understand and adapt to the local context that can best enable them to live in both communities of practitioners and researchers. Researchers could learn a lot from evaluators! This is the argument that I earned me the AEA Student Travel Award.

Evaluator Competencies, Guiding Principles, and Professionalization

In the past couple years, the AEA board created two task forces, one that focused on revising the AEA Guiding Principles (which are supposed to be revisited every 5 years) and another to develop a set of evaluator competencies. These guidelines—in conjunction with the evaluation standards—serve as guides to inform our practice.There were many sessions from the two task forces to update AEA membership. The competencies task force is currently finalizing the set of core competencies, which include the domains of professional practice, methodology, context, planning and management, and interpersonal. Results from the membership survey indicate strongest endorsement of competencies that deal with ethical practice within each domain. Not surprisingly, “conducts informal and forma meta-evaluations of studies, identifying their strengths and limitations” and “obtains internal or external evaluation work” were lowest rated items. Our field has not demonstrated much interest in or use of meta-evaluation.The guiding principles task force is in the early stages of revising the guiding principles. Interestingly, they are proposing to add a fifth guiding principle of equity. As I mentioned in my feedback during the session, I am not sure whether or how it differs from the “respect for people” domain or “respect for public and general welfare” domain (which they recommend renaming to public interest and social benefit). Others mentioned not being sure whether equity should always be a guiding principle (i.e., warranting a separate domain) or if it is sometimes a guiding principle (i.e., it is a sub-component of another domain). Furthermore, equity or equality? There is much work to be done on this and I am not sure how the field will receive these changes.Some evaluators are wary of these changes, particularly citing access and equity as necessary things to consider as we develop our field. However, most seem wary about using the competencies as a move towards professionalization. The competencies task force is not charged with doing anything more than developing a set of competencies, gaining feedback, and providing feedback to the AEA board (and AEA membership) for approval. That has not stopped some evaluators from being very vocal against the competencies. I appreciate their critical feedback, but at the same time I think the competencies will be beneficial in and of themselves even if there is no further work in professionalizing the field.I very much enjoyed the invigorating conversations on these topics at Eval17 and can’t wait to hear about further work from the task forces between now and Eval18.

Networking

Through Twitter, I have made many friends in the evaluation community. Since I have finished coursework, Twitter has been a source of professional development and helping me stay abreast of the latest development and news in the field worldwide. I had the pleasure of meeting many Twitter evaluators at Eval17—including Tom Archibald, Nicole Clark, Kylie Hutchinson, Sheila Robinson, Deven Wisner, and many more that I can recall—whom I am not sure I would have met without our connections on Twitter.I also continued the tradition of having an evaluation Twitter lunch this year. Eight other evaluators joined me at lunch to discuss the benefits of Twitter, how we use Twitter (especially to inform our practice), and how we can help the field get onto Twitter to invigorate the #eval conversations. Through this lunch, we have decided to put out numerous blog posts, host a Twitter chat, and even propose a panel or demonstration session for Eval18 to educate evaluators on the benefits of Twitter and how to use it for professional development.