EvaluLand

After chatting with folks from EvalYouth about the Global Mentoring Program, they suggested a series of podcast episodes on what evaluation looks like around the world.

This episode is one in which I chat with evaluators from Europe about what evaluation looks like in their region. I chat with Tom Ling and Oto Potluka about European evaluation. They are both board members of the European Evaluation Society, the professional association for evaluators in Europe. We discuss a variety of topics, including the streams of evaluation happening in evaluation, the role of evaluators in program design, reflective practice, stakeholder involvement, impact evaluation, evaluation approaches like theory-driven evaluation, how the GDPR affects evaluation, and more.

Special thanks to Lauren Weiss, the EES Adminstration and Communications Manager, for helping coordinate this episode.

Oto Potluka: Oto Potluka (Ph.D.) is a senior researcher at the Center for Philanthropy Studies, University of Basel (Switzerland). He has studied at the University of Economics Prague, Faculty of Economics and Public Administration. Than, he obtained Ph.D. at the Charles University Prague, Faculty of Social Sciences in 2009. Since 2001, his main activities have concerned IREAS, a non-profit think tank in regional development. His responsibility was mainly evaluation work in the field of the regional and economic development, and civil society.

He has participated in dozens of Czech and international evaluation, research, and training projects in the field of economic development and the impact of public regulation and public expenditure programmes. He has experience with evaluations of public expenditure programmes in regional development, especially those co-financed by EU cohesion policy.

He has experience from several professional associations. He had been actively working in the board of directors of the Czech Evaluation Society for two terms between years 2012-2015. Among the other associations belong European Evaluation Society, American Evaluation Association, Czech Economic Association, and Regional Studies Association. As a member of these organisations, he has always actively participating in building of evaluation culture and evaluation capacities.

Tom Ling: Tom Ling has over 25 years of experience in designing, managing and delivering evaluations. He is a senior research leader at RAND Europe where he is head of evaluation. He leads evaluations and applied research focused on the key challenges facing organisations in health, well-being, and international development. Before re-joining RAND, Ling was head of Impact Innovation and Evidence at Save the Children where his responsibilities included ensuring that evaluations contribute to policy and change in the challenging environment of international development. Prior to Save the Children, Ling spent ten years at RAND Europe as director for Evaluation and Performance Audit following four years as senior research fellow at the National Audit Office in the UK. Before that he taught and researched in various universities.

He has over 20 years’ experience contributing to research projects with the European Commission, Save the Children, UK government departments, the National Audit Office, the Health Foundation in the UK and many others. He has published widely on evaluation, accountability, and related topics. He recently co-edited Performance Audit: Contributing to Accountability in Democratic Government (London: Edward Elgar), following his Performance Audit Handbook and The Evidence Book, a critical examination of the use of evidence in public policy and service delivery. He is a senior research associate with Cambridge University and a Professor (Emeritus) with Anglia Ruskin University.

In any spare time Tom is a keen musician playing the fiddle in both Scottish traditional and jazz bands with many recordings, compositions and performances to his name. He has a wife and two daughters and lives in Cambridge, England.

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

We’re getting meta today… looking at the influence of a major influencer in evaluation: Carol Weiss! I chat with Gregory Greenman II about his dissertation examining citation metrics of her work across disciplines.

Gregory spends a lot of his time thinking about the nature of evidence and its role in policy decisions. He believes that applying sound evaluative thinking and appropriate research methods helps us down the path toward building a just and equitable society. His current research examines the "how and why" of evidence-based policymaking and how we evaluate interdisciplinary research.

Gregory currently works with Eval4Impact, helping organizations understand which aspects of their interventions are successful. He is also an honorary fellow with the Centre for Program Evaluation at the Melbourne Graduate School of Education, where he co-taught evaluation for the past four years. Before making his way to evaluation, he managed legislative research and policy analysis on barriers to prisoner reentry and drug policy at a university in Illinois.

Gregory earned his PhD in Interdisciplinary Evaluation from Western Michigan University in 2020. He is the website coordinator for the RoE TIG and an associate editor for the Journal of MultiDisciplinary Evaluation.

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

I chat with Helen Salmon from SAGE Publishing about book publishing! We go over how to generate ideas for books, what the book publishing process looks like, and about some of the most recent evaluation books from SAGE!

Sage has been a friendly publisher for evaluation books for decades. As Patton stated in the third edition of Qualitative Research and Evaluation Methods (2002): "[Sara Miller McCune's] vision and follow-through have made Sage publications the leading publisher of both evaluation and qualitative inquiry books." (p. xxiv).

Some recent evaluation books mentioned:

More information for interested book authors and editors: https://us.sagepub.com/en-us/nam/book-author-editors

Helen Salmon is an Executive Editor at SAGE Publishing where she signs and develops textbooks in research methods, statistics, and evaluation across the social and behavioral sciences. Originally from the U.K., Helen has worked for SAGE for 22 years, moving to their U.S. office in Thousand Oaks, California in 2005. She is passionate about helping authors develop books that will meet the needs of today’s students, and publishing the voices of a more diverse group of authors at SAGE.

You can find her blog here: [helen.salmon.net](helen.salmon.net)

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

In this week's episode, I chat with Sawsan Al-Zatari, Zach Tilton, and Teddy Theoharis about the EvalYouth Global Mentoring Program (EY-GMP).

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

This week I chat with Tiffany Smith and Libby Smith about our experiences teaching a course on interpersonal effectiveness in the MS in Applied Psychology program at the University of Wisconsin-Stout. We also discuss why teaching interpersonal skills is important and why we believe these skills can and should be explicitly taught.

The course is a 3-credit, 8-week course that focuses on developing self-awareness through personal and collaborative reflective practice. It is the third course in the evaluation concentration. Tiffany brought this course to our evaluation program and taught it from 2015-17. Libby then taught it from 2018-19, bringing in new elements. Dana taught it most recently in an online environment due to the COVID-19 pandemic.

Tiffany [Smith] Tovey: Tiffany [Smith] Tovey (she/her) is a researcher, program evaluator, and educator who focuses on the importance of reflective practice, epistemological awareness, and interpersonal communication in her work and life.

She is currently the senior evaluation specialist at the Office of Assessment, Evaluation, and Research Services in the Educational Research Methodology department at the University of North Carolina Greensboro.

At UNCG, she leads and facilitates numerous evaluation and research projects in K12 education, higher education, and community settings. She also teaches workshops and courses in interpersonal skills, reflective practice, research methods (qualitative and quantitative), and evaluation.

She has a Ph. D. in Educational Psychology and Research with a focus on Evaluation, Statistics, and Measurement and a cognate in Communication Studies from the University of Tennessee, Knoxville. She has a Bachelor’s degree in Psychology and Philosophy from the University of Tennessee, Knoxville as well, and thus very much enjoys a good, deep, philosophical conversation.

Libby Smith: Libby Smith (she/they) is an organizational healing facilitator, as an experienced and holistic evaluator and educator she excels at the human component of evaluation and organizational change. Never one to shy away from crucial conversations, Libby deftly balances accountability and compassion.

Their work focuses on building equity and accessibility through personal growth & embodiment practices. Libby uses all of these skills to provide intersectional and liberation-forward guidance to organizations and clients seeking transformative change.

She has an MS in Applied Psychology works for Catalyst at the University of Wisconsin-Stout, where she also serves as Program Director for the MS Applied Psychology program.

She has been practicing breathwork since 2018, primarily with the guidance of Amy Kuretsky, and has trained in breathwork healing with David Elliott She loves to walk in the woods, take photos and is auntie to 11 year old twin girls. You can learn more about her work at workwithlibby.com

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

This week I chat with Dr. Aisha Rios to both get to know her a little better and to talk about social justice in evaluation.

"My name is Aisha Rios. I founded Coactive Change in early 2020 after working for several years as an external evaluator with consulting firms functioning at the federal, state, and local levels. My passion for social change and justice permeates my participatory and collaborative approach to working with partners, and these values ground my belief in the power of evaluation as a means to facilitate learning and change. I have engaged in grassroots organizing and community building that focused on issues ranging from the exploitation of adjuncts who are overworked and underpaid, and most recently the abolition of the prison industrial complex. The work I am most proud of has centered on dismantling structural violence by way of social movement and community organizing, both directly and in supporting other change agents."

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

I've received multiple questions about my podcasting setup and process. Rather than answer the questions individually, I thought I would podcast about it! I discuss why I got into podcasting, my recording technology and room setup, the audio processing my husband (aka audio engineer) does, podcast hosting, the general timeline for me to create an episode, and some extra few questions.

Technology

Also check out this blog post by Echo Rivera: The Ultimate Guide to Microphones

Contact

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

This episode I chat with Ann Webb Price and Susan Wolfe about coalition evaluation, partnering together in evaluation, the independent consulting TIG of AEA, and so much more!

Ann Webb Price: Ann Price is a community psychologist and evaluator based in Georgia. She helps community coalitions, and nonprofit and foundation leaders identify the root causes of social problems and harness their evidence in order to create real community change. Ann is active in AEA, mentors new and emerging evaluators through her office hours and the Independent Consulting IC Chats, and speaks and trains on evaluation and community coalitions. Ann blogs regularly about community coalitions, evaluation use, and evaluation for non-evaluators and in 2021 will launch her new podcast called Community Possibilities.

Susan Wolfe: Susan Wolfe enjoys evaluation work when it is complex and challenging while focusing on equity and social justice. She has been a Community Consultant with Susan Wolfe and Associates since 2009 where much of her work focuses on community coalitions. In 2020 she teamed up with Ann Webb Price and Kyrah K. Brown to co-edit an issue of New Directions for Evaluation – Evaluating Community Coalitions and Collaboratives.

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

In this episode, I chat with Aasha Joshi and Hannah Lantos about their recent chapter "Demystifying data: Strategies and tools for making data more meaningful in OST programs" in the recently released book Measure, Use, Improve! Data Use in Out-of-School Time, edited by Christina Russell and Corey Newhouse. The book will be particularly useful to leadership-level staff in out-of-school time organizations that are thinking about deepening their own learning and evaluation systems, yet aren’t sure where to start.

Aasha Joshi: Aasha’s work is guided by three questions: What’s being done? Is it working? And how can we make it work better? For 15 years, she has been advising, designing, conducting, managing, and integrating robust research and evaluation in organizations so that they can improve their processes, services, and products to meet their goals and make a positive difference in people’s lives. Aasha earned her PhD in Social Psychology from the University of Cambridge (UK), studying social interactions and knowledge sharing, and is a co-founder of Clarity Writing and Research, a company formed with a commitment to helping people achieve better outcomes in the education and social sectors.

Hannah Lantos: Hannah Lantos, Ph.D., is a Research Scientist in the Youth Development research area at Child Trends. She has served as project director on both large and small projects that have focused on social and emotional learning, teen pregnancy prevention, protective factors for youth involved in the justice system, and integrating a positive youth development approach into job skills training programs for young adults. She completed her Ph.D. in Public Health with a focus on social and behavioral trends in adolescent health from Johns Hopkins University in 2015. Her work broadly aims to explore how the social and physical environments that adolescents live in affect their health and well-being. Adolescence is a time of amazing growth, skill-building, and navigating challenges. Learning how to navigate challenges in ways that promote learning, kindness, and health drive her work. In particular, she is interested in how contexts can strengthen adolescents’ resiliency as they learn about and engage in healthy decision-making. She has worked with both large, national datasets and small, evaluations and with both quantitative and qualitative data in both domestic and international settings. Before her graduate studies at Johns Hopkins, where she wrote a dissertation comparing adolescent’s experiences with violence in Baltimore and Johannesburg, she worked primarily in international settings. She served as a Peace Corps Volunteer in northern Zambia before receiving a Masters of Public Administration in International Development (MPA-ID) and speaks Arabic, Hebrew, Spanish and can still say hello in Bemba (one of the languages spoken in northern Zambia!). She is passionate about applied, multidisciplinary research on adolescent health, well-being, positive development, and in considering the “whole child” as we do research and evaluation on programs that serve young people.

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand

Thank you to everyone who participated in the call-in to share their own reflections about the 2020 AEA virtual experience!

In this episode, I start with thanking the conference organizers, sharing my own reflections, and then pass it off to the eight people who called in to share their own reflections:

Music by Matt Ingelson, http://www.mattingelsonmusic.com/

Support EvaluLand