EvaluLand
Download on the App Store

EvaluLand episodes

  • 41: Theory-based approaches for navigating complexity with Michael Moses

    In this episode, I chat with Michael Moses about theory-based approaches for navigating complexity, adaptive management, participatory strategy, and actions to shift power.

    Things and resources mentioned:
    • Systems thinking in evaluation (including a discussion of complexity) - Bob Williams and Sjon Van't Hof
    • Outcome mapping and outcome harvesting and emergent learning
    • Emergent Learning (including before action reviews, after action reviews, emergent learning tables, and more approaches for supporting collective learning and action)
    • Strategy Testing - Deborah Ladner and the Asia Foundation
    • Kat Haugh - visual facilitation, sensemaking, and notetaking
    • Learning to Make All Voices Count Initiative - Summary and full paper
    • Cynara training on Decolonizing M&E and Research, led by Michelle Lokot
    • Contact information:

      Michael Moses

      LinkedIn

      About Michael Moses:

      Michael Moses is a strategist, facilitator, and evaluator with over 12 years of experience working with public and private sector partners to achieve social impact. He advises foundations, companies, governments, and nonprofits in their efforts to design and implement strategies for advancing change, including by capturing and using data to navigate the complex systems in which they work. In doing so, he helps changemakers learn how to improve programs and organizations, adapt, and over time, strengthen their impact.

      Support EvaluLand

      56 min
    • 40: Community-based participatory research with Dr. Tatiana Bustos

      In this episode, I chat with Dr. Tatiana Elisa Bustos on community-based participatory research (CBPR). We talked about what it is, how it compares to research and other similar forms of inquiry, and how to get started doing CBPR.

      Disclaimer: Views expressed here are personal and not reflective of the speaker's respective employers or agencies.

      Contact information

      Dr. Tatiana Elisa Bustos

      @TElisa72
      https://www.linkedin.com/in/tebustos/

      About Dr. Bustos

      Dr. Tatiana Elisa Bustos knows that community partner engagement is key to understanding social issues. She’ll share her experience applying community-based participatory research approaches.

      Dr. Bustos innovates outside the box ways to do research that invite community participation, improving programs through implementation with a social justice lens. As a 1st generation college student and the daughter of Nicaraguan immigrants, equity is deeply important to her. She is an author and award-winning researcher. She leads professional development workshops on implementation science and community based participatory research.

      She received her PhD in Community Psychology from Michigan State University, an MS in Psychology from Nova Southeastern University, and a BA in Psychology from Florida International University. Connect with her on LinkedIn.

      Dr. Bustos also appeared on The Sci-Files on Impact 89FM and Beyond the Manuscript, the podcast of Progress in Community Health Partnerships.

      Resources

      Professional Organizations

      • Society for Community Research and Action
      • American Evaluation Association Connect (CBPR search)
      • Community Psychology TIG
      • Training Institutes

        • https://www.detroiturc.org/programs-expertise/cbpr-capacity-building
        • https://www.detroiturc.org/about-cbpr/online-cbpr-course
        • https://www.mitrainingcenter.org/courses/cbprs0218noce
        • Toolkits

          • https://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/intervention-research/main
          • Journals

            • Global Journal of Community Psychology
            • American Journal of Community Psychology
            • Collaborations: A Journal of Community Based Research and Practice
            • Support EvaluLand

              35 min
            • 39: #Eval22 Conference Preview

              In this episode, I provide a brief overview of the #Eval22 conference, describe the schedule at a glance, and provide tips for getting the most out of the conference. I hope to see you in New Orleans! Say hi to me at the conference to get an EvaluLand nametag ribbon.

              Eval22 Resources
              • Register to attend the AEA Annual Business Meeting on November 3 at 2pm ET; a recording will be made available after the meeting.
              • Register to attend TIG virtual meetings before and after the conference
              • See the conference schedule at a glance or the entire searchable program
              • Learn more about the professional development workshops offered Monday, Tuesday, and Wednesday prior to the conference
              • Attend all the social events at the conference: poster exhibit & meet the authors reception, TIG fair & reception, and the silent auction to benefit international presenters
              • Attend all the plenary sessions and presidential strand sessions focused on the theme of the conference and watch the virtual pre-conference presidential town hall series that occurred in the months leading up to the conference
              • Check out the resources for attendees, including guide to New Orleans, Zoom background options, PowerPoint templates, marketing toolkit, and tips for social media
              • Check out the speaker resources, including information about rooms and materials, printing and shipping, and the potent presentations information
              • Check out the floor plan for the Exhibit Hall, including the exhibitors, posters, and TIG fair
              • This LinkedIn post by Sylvia Pu, PhD has a ton of great comments about how to get the most out of the conference
              • Support EvaluLand

                16 min
              • 38: Subcontracting with Dr. Tamara Hamai

                I talked with Dr. Tamara Hamai of Hamai Consulting about contracting, subcontracting, and independent consulting in evaluation. We discussed the minor differences between contracting and subcontracting, cleared up a common misconception of what subcontracting relationships are like, how to get into subcontracting, and tips for setting up contracts.

                Contact information:

                Tamara Hamai

                Hamai Consulting
                https://sustainableimpact.co

                About Dr. Tamara Hamai:

                Tamara Hamai, Ph.D., has dedicated her career to empowering organizations and rebuilding our global systems to encourage children’s holistic growth and well-being, from prenatal through the completion of higher education – especially those who are most vulnerable and facing the greatest challenges. In 2008, she founded Hamai Consulting as a platform to help organizations increase their impact, stability, and strength to make a bigger impact in children’s lives. Dr. Hamai’s work spans most aspects of child development, such as early childhood education, higher education, child welfare, parenting and discipline, trauma, and adverse childhood experiences. She has previously been featured by KTLA, KPIX CBS San Francisco, ABC 7 News, NBC Radio, American Psychological Association, Western Psychological Association, Institute for Violence, Abuse, and Trauma, American Evaluation Association, National Head Start Association, and several blogs. She is also a reviewer and on the Editorial Board for publications such as the Journal of Aggression, Maltreatment & Trauma, Journal of Child & Adolescent Trauma, Journal of Sexual Abuse, and the Journal of Child Custody.

                Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                Support EvaluLand

                53 min
              • 37: Strategic Planning with Carrie Tanasichuk and Harry Daley

                This episode I chatted with Carrie Tanasichuk and Harry Daley about how they have been using theories of change as their main process for facilitating strategic planning with non-profits.

                In the episode, they also mentioned:

                • The Community Foundation of Saint John’s impact measurement and evaluation principles
                • The Community Builder’s Approach to Theory of Change
                • The Miles Nadal JCC Theory of Change
                • About Carrie and Harry:

                  *Carrie Tanasichuk *(Twitter DrCarrieTee & [[email protected]]([email protected])) has 16 years of experience in evaluation. She is passionate about using her expertise to help organizations demonstrate impact, improve, and innovate. She has worked across sectors (non-profit, for-profit, government) in diverse areas, including poverty reduction, youth development, criminal justice, health promotion and screening, and technology. She has a Ph.D. in Applied Social Psychology from the University of Saskatchewan.

                  Harry Daley ([[email protected]]([email protected])) was born and raised in Saint John, New Brunswick. He has dedicated his personal and professional life to working with organizations that focus on poverty reduction and amplifying the youth voice. He has used evaluation as a tool for designing experiential and participatory youth programming and believes evaluation is an integral aspect of program design and facilitation rather than something that lives outside the programs. Harry has a Bachelor of Philosophy in Interdisciplinary Leadership from the University of New Brunswick’s Renaissance College.

                  Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                  Support EvaluLand

                  1 hr 3 min
                • 36: Evaluation Job Market with Bradlie Nabours

                  This episode I chatted with Bradlie Nabours about how he got into the field of evaluation and his experience applying for evaluation jobs. He also talks about his evaluation experience and provides great tips on applying for jobs in the evaluation space.

                  Job search & application recommendations:
                  • Don’t give up. Know your worth. The job process is difficult, but don’t settle and remember it doesn’t reflect necessarily on you. Keep applying!
                  • Don’t limit yourself in what positions you look for. Expand your horizons.
                  • Network like your life depends on it! This includes having an online presence and building and leveraging connections with people in your field such as through LinkedIn, EvalYouth (LinkedIn & Twitter @evalyouth_na), and your professors!
                  • Be a versatile applicant. Build your resume. Seek out learning opportunities to grow as a professional. Be willing to experiment and try new things.
                  • Contact information:
                    • Twitter: bradn98
                    • LinkedIn: www.linkedin.com/in/bnabours
                    • About Bradlie Nabours:

                      Bradlie is a perinatal health program evaluator at Johns Hopkins All Children’s Hospital in St.Petersburg, Fl. He works for a federal Healthy Start program focused on reducing adverse perinatal outcomes for black women and their families. He holds a Bachelors of Science in Public Health from Georgia Southern University and completed his Master’s in Public Health with a graduate certificate in Maternal and Child Health from the University of South Florida. He started his evaluation journey while in grad school where he worked as a graduate assistance on the evaluation of a non-profit family violence prevention program. While in grad school his research focused racial disparities in birth outcomes which led him to start his career at the intersection of his research interest and evaluation.

                      Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                      Support EvaluLand

                      55 min
                    • 35: International Evaluation with Dr. Tristi Nichols

                      On this episode, I’m chatting with Dr. Tristi Nichols about her work conducting international evaluation and her journey into the international and evaluation spaces.

                      Resources:
                      • Footprint evaluation
                      • OECD DAC Criteria for Evaluation Framework
                      • List of websites to find international evaluation:

                        • https://www.ungm.org/ type “evaluation” in the title bar [for UN only]
                        • https://reliefweb.int/ or https://reliefweb.int/jobs
                        • https://www.evalcommunity.com/
                        • Contact information:

                          Tristi Nichols

                          914-414-8288
                          @tristiempo
                          https://www.linkedin.com/in/drtristinichols/

                          Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                          Support EvaluLand

                          58 min
                        • 34: Evaluation Job Market with Dr. Ayesha Boyce

                          On this episode, I’m chatting with Dr. Ayesha Boyce about the evaluation job market. Ayesha provides a ton of great insights and information about how to search for jobs, what to consider when looking for jobs, and tips for being a strong job candidate.

                          Ayesha was previously on the podcast discussing "teaching evaluation and supporting students and colleagues of color."

                          Contact information:

                          Ayesha Boyce

                          Twitter: @AyeshaBoyce
                          Website: https://education.asu.edu/about/people/ayesha-boyce

                          About GUEST:

                          Ayesha Boyce is currently an associate professor in the Division of Educational Leadership and Innovation at Arizona State University. She also co-directs the STEM Program Evaluation Lab. Boyce’s scholarship focuses on attending to value stances and issues related to diversity, equity, inclusion, access, cultural responsiveness, and social justice within evaluation—especially multi-site, STEM, and contexts with historically marginalized populations. She also examines teaching, mentoring, and learning in evaluation. She has evaluated more than 55 programs funded by the National Science Foundation (NSF), US Department of Education, National Institutes of Health, and Spencer and Teagle foundations. Boyce is a 2019 American Evaluation Association Marcia Guttentag Promising New Evaluator Award recipient. In her teaching and mentorship, Boyce encourages students to develop a strong methodological foundation, conduct studies based on democratic principles, and promote equity, fairness, inclusivity, and diversity.

                          Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                          Support EvaluLand

                          1 hr
                        • 33: Worker Training Evaluation with Dr. Eric Persaud

                          On this episode, I’m chatting with Eric Persaud about evaluation at the National Institutes of Health, including his work in the Worker Training Program at the National Institute of Environmental Health Sciences. We also talked about his dissertation, how he got into the field of evaluation, and his experiences in conducting evaluation in general.

                          About Dr. Eric Persaud:

                          Eric Persaud received his doctorate in Public Health at the Department of Environmental and Occupational Health Sciences, State University of New York-Downstate Health Sciences University. He focuses on evaluating and researching training programs related to preparing workers for emergencies and disasters, and hazardous workplaces. He has been involved in evaluation and research associated with fentanyl and first responders, opioids and the workplace, and protecting workers from COVID-19.

                          You can reach Eric Persaud at [email protected]

                          You can learn more about the NIEHS Worker Training Program at https://www.niehs.nih.gov/careers/hazmat/about_wetp/index.cfm

                          Resources:
                          • Biosafety and infectious disease occupational health training from the NIEHS Worker Training Program: A Historical look at capacity building that supported a COVID-19 response. Eric Persaud, Deborah Weinstock, Demia S. Wright. Journal of Emergency Management. 2022.
                          • Opioids and the Workplace Prevention and Response Awareness Training: Mixed Methods Follow-Up Evaluation - Eric Persaud, Aimee Afable, Laura A. Geer, Paul Landsbergis, 2021
                          • COVID-19 Biosafety Training and Infectious Disease Response Evaluation Report. NIEHS Worker Training Program. 2021.
                          • Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                            Support EvaluLand

                            50 min
                          • 32: Systemic Design Thinking with Jan Noga

                            This episode I chatted with Jan Noga about systemic design thinking. There’s a wealth of resources and information provided below!

                            Contact information:

                            Jan Noga

                            www.pathfinderevaluation.com

                            About Jan Noga:

                            Jan Noga is an independent evaluation consultant based in Cincinnati, Ohio. She holds a bachelor’s degree from Stanford in developmental and counseling psychology with specialization in early and middle childhood and a master’s degree from the University of Cincinnati in instructional design and technology. Jan has worked in the non-profit and public sectors in human services and education for more than 30 years in roles spanning teaching, research, policy, and program planning and evaluation. As a program evaluator, Jan has planned and conducted both large and small-scale evaluations and provided organizational consulting and capacity building support to clients. She has also taught courses and workshops on such topics as systems thinking, systemic design thinking, research methods and techniques, program planning and development, and survey design and analysis. Jan has been a member of AEA since 2000 and was one of the founding members of the Systems in Evaluation TIG, serving as program chair and then TIG chair from 2004-2012. She is particularly interested in the use of systems approaches as a foundation for design, planning, implementation, and evaluation of change efforts in the human service and education arenas.

                            Systems Thinking Resources for Evaluators:

                            Hands on resources:

                            • Williams, Bob. 2020. Systemic evaluation design: A workbook. Available for download from https://bobwilliams.gumroad.com/
                            • Williams, Bob. 2021. Systems diagrams: A practical guide. Available for download from https://bobwilliams.gumroad.com/
                            • Good for starting out

                              • Anderson, V. & Johnson, L. (1997). Systems thinking basics: From concepts to causal loops. Waltham, MA: Pegasus Communications.
                              • Meadows, D.H. (2008). Thinking in systems: A primer. White River Junction, VT: Chelsea Green Publishing.
                              • Ramage, M. & Shipp, K (2009). Systems Thinkers. New York: Springer.
                              • Sweeney, L.B. & Meadows, D. (2010). The systems thinking playbook. White River Junction, VT: Chelsea Green Publishing.
                              • Williams, B. & Hummelbrunner, R. (2011). Systems concepts in action: A practitioner’s toolkit. Stanford, CA: Stanford University Press.
                              • Williams, B. and Imam, I, eds. (2007). Systems concepts in evaluation: An expert anthology. Point Reyes, CA: EdgePress.
                              • Williams, B. and Van’t Hoft, S (2016). Wicked solutions: A systems approach to complex problems. Available at http://bit.ly/1SVoOH3
                              • Good for more advanced reading:

                                • Bamberger, M, Vaessen, J., & Raimondo, E. (eds.) (2016) Dealing with complexity in development evaluation: A practical approach. Thousand Oaks, CA: Sage Publications.
                                • Cabrera, D., Colosi, L., & Lobdell, C. (2008) Systems thinking. Evaluation and Program Planning, 31(3), 299-310.
                                • Cabrera, D. & Cabrera, L (2015). Systems thinking made simple: New hope for solving wicked problems. Odyssean Publishing.
                                • Capra, F & Luisi, PL (2016). The systems view of life: A unifying vision (6th printing). New York: Cambridge University Press.
                                • Checkland, P. (1999). Systems thinking, systems practice. New York: John Wiley & Sons, Ltd. Cunliff, E., (2002) Connecting systems thinking to action, The Systems Thinker, 15(2), 6-7.
                                • Eoyang, G.H. & Holladay, R.J. (2013) Adaptive action: Leveraging uncertainty in your organization. Stanford: Stanford Business Books.
                                • Karach, R, (1997) How to see structure, The Systems Thinker, 8(4), 6-7.
                                • Patton M.Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: Guilford Press.
                                • Patton, M.Q., McKegg, K., & Wehipeihana, N., eds. (2015). Developmental evaluation exemplars: Principles in practice. New York: Guilford Press.
                                • Senge, P. (1990) The fifth discipline: The art and practice of the learning organization. New York: Doubleday.
                                • Stroh, DP (2015). Systems thinking for social change. White River Junction, VT: Chelsea Green Publishing.
                                • Ulrich, W & Reynolds, M (2010). Critical systems heuristics. In: Reynolds, Martin and Holwell, Sue eds. Systems approaches to managing change: A practical guide. London: Springer, pp. 243–292.
                                • von Bertalanffy, Ludwig. (1950). The theory of open systems in physics and biology. Science,
                                • 13, 23-29.
                                • von Bertalanffy, Ludwig. (1968). General systems theory. New York: George Braziller, Inc.
                                • Wolf-Branigin, M. (2013) Using complexity theory for research and evaluation. Oxford: Oxford University Press.
                                • Some other resources:

                                  • International Society for Systems Sciences
                                  • https://aea365.org/blog/systemic-design-thinking-for-evaluation-of-social-innovations-a-pd-for-intermediate-and-advanced-evaluators-by-jan-noga/
                                  • http://www.epreconsulting.com/SETIG%202018%20Principles.pdf
                                  • https://systemic-design.org/
                                  • https://modus.medium.com/what-the-is-systems-design-e005c1e9fef8
                                  • https://rsdsymposium.org/
                                  • Martin Reynolds Open University
                                  • Music by Matt Ingelson, http://www.mattingelsonmusic.com/

                                    Support EvaluLand

                                    1 hr 6 min

                                  About EvaluLand

                                  From the publisher's feed

                                  This podcast discusses the landscape of evaluation. Each episode will feature a guest in or around evaluation to discuss something evaluation-related. Email me at [email protected] if you have…