EvalNetwork

Rakesh Mohan has been the director of the Office of Performance Evaluations (OPE) an independent and nonpartisan agency of the Idaho State Legislature, since 2002. We cover important issues related to evaluation and how evaluation can guiding government and public policy. Under Rakesh’s direction, the office has received many awards, including the American Evaluation Association’s...

Ann K. Emery at Depict Data Studio helps people and organizations visualize data more effectively. This interview covers key evaluation and research reporting strategies, techniques, and resources. Ann is well-known in the evaluation world and beyond. She has been invited to speak in more than 30 states and 10 countries, more than 3,200 people have enrolled in her online training academy, and she has consulted more than 150 organizations, including the United Nations, Centers for Disease Control and Prevention, and Harvard University.

 In this discussion, we talk about: – Why Ann loves data visualization and reporting – Reasons evaluators should be concerning about data visualization and reporting – Resources someone just starting out in evaluation (and others) can use to get proficient in data visualization. This includes Ann’s free course: Soar Beyond the Dusty Shelf Report– Using a DataViz Wall of Fame – Techniques for engaging project stakeholders in the reporting process. She co-authored an article on using Data Placemats that is available here– Common mistakes she sees evaluators make with their reports – How Ann obtains helpful feedback regarding her work – Her inspiration for developing blog posts – Books that Ann recommends – How she structures her day/week to enhance productivity Enjoy!  

Kylie is the 2020 recipient of the Canadian Evaluation Society’s Award for Contribution to Evaluation in Canada. She has spent the past thirty years working in the not-for-profit sector as a consultant, trainer, program manager, board member, and volunteer. 

There are many valuable nuggets from this video. We discuss the following during the interview:

– Reasons why Kylie loves evaluation.

– Why evaluation is relevant for health and human services students who are not necessarily interested in research or evaluation.

– Why she wrote Evaluation Failures: 22 Tales of Mistakes Made and Lessons Learned.  She highlights some of the most important  lessons from the book.

– Kylie talks about how everyone makes mistakes and the key is learning from them.

– She reviews specific strategies for engaging project stakeholders in an evaluation.

– Kylie provides her favorite resources regarding program design.

– How mindfulness factors into her work and how it can inform the work of evaluators.

– Books she recommends.

You can connect with Kylie through https://communitysolutions.ca or @EvaluationMaven at Twitter.

Watch the video recording of our interview here:

James Pann, Ph.D., interviews Holly Rustick to get her insights into the grant writing and management from her years of experience doing this work, her work training others to do it, and her podcast interviews.

We discuss: 

– Common themes or lessons she has learned from the 120+ podcast episodes completed

– Suggested first steps in writing a grant proposal

– Frequent mistakes that grant writers make

– Low hanging fruit (easy wins) in the proposal development process

– Good resources for those starting to write grant proposals

– How an organization should go about finding a grant writer

Thanks and enjoy!

Watch the video recording of our interview here:

James Pann, Ph.D. interviews with Larry Martin, Ph.D. who is co-author of Designing & Managing Programs: An Effectiveness-Based Approach and a  professor at the University of Central Florida about program design and evaluation. OUTLINE: 00:00 – Introduction 00:22 – Background and Larry’s interest in program design and management 03:13 – Reasoning behind publishing the book...

In this episode, I interview Michael Quinn Patton on Utilization-Focused Evaluation.   “What don’t you know, that if you did know, would make a difference to what you do.”   Utilization-Focused Evaluation with Michael Quinn Patton, Ph.D. “What don’t you know, that if you did know, would make a difference to what you do?” The importance of good questions and careful thought with Michael Quinn Patton, Ph.D., interview by James Pann, Ph.D. Michael Quinn Patton is the Founder and CEO of Utilization-Focused Evaluation, an independent organizational development and program evaluation organization. He has authored numerous books on evaluation, including Blue Marble Evaluation (2019), Principles-Focused Evaluation (2018), Facilitating Evaluation (2018), Developmental Evaluation (2010) and Utilization-Focused Evaluation (2008). During the interview Michael answers the following questions and addresses other areas:   OUTLINE:00:00 – Introduction02:10 – Definition of Utilization-Focused Evaluation (U-FE)04:11 – How developmental evaluation, principles-focused evaluation, and blue marble evaluation fit in with U-FE07:44 – What’s emphasized in U-FE that is not in other evaluation approaches10:03 – How U-FE can be used in conjunction with other evaluation models11:49 – Specific strategies and tactics for engaging project stakeholders in an evaluation15:00 – Lessons from social sciences research literature that inform Michael’s evaluation work17:35 – Common things that evaluators think they already know, but often don’t18:45 – Best way to really learn how to do evaluations and recommendations for students and beginning evaluators23:36 – Relationship between mindfulness and evaluation26:52 – Resources to obtain familiarity with using U-FE28:13 – Books that Michael likes to give as gifts31:04 – How to connect with Michael   

CONNECT WITH JAMES:

– Subscribe to this YouTube channel

– LinkedIn: https://www.linkedin.com/in/pannjames/

– Twitter: https://twitter.com/jpann

Please reach out with comments and feedback! Thanks!

Watch the video recording of our interview here:

In this episode, I interview David Fetterman about Empowerment Evaluation.  David Fetterman introduced empowerment evaluation to the field of evaluation during his presidential address 21 years ago. Since that time it has been used in over 16 countries, ranging from corporate offices of Google and Hewlett-Packard to squatter settlements and townships in South Africa.  Empowerment evaluation has been used by the W.K. Kellogg Foundation, Centers for Disease Control and Prevention, US. Department of Education, Stanford University School of Medicine, and Native American tribes in reservations stretching from Michigan to San Diego.  

For more information about empowerment evaluation see the following:

Better Evaluation: https://betterevaluation.org/en/plan/…

Blog Postings: Stanford Social Innovations Review – This is a brief article about another empowerment evaluation book: Fetterman, D.M. (2103). Empowerment Evaluation in the Digital Villages: Hewlett-Packard’s $15 Million Race Toward Social Justice. Stanford: Stanford University Press.

https://ssir.org/articles/entry/empow… Empowerment Evaluation Blog.

eevaluation.blogspot.com

Videos:

Brief video describing empowerment evaluation. See our Ignite Lecture (20 slides in 5 minutes).

https://www.youtube.com/watch?v=fjUvV… Video of an empowerment evaluation exercised used to facilitate an NSF funded computer science education evaluation initiative. It was used specifically to help them assess their efforts and move their efforts forward, including building a repository of STEM and CS evaluation tools and instruments.

https://www.youtube.com/watch?v=ploz0…

Articles:

Fetterman, D.M. (2011). Empowerment Evaluation and Accreditation Case Examples: California Institute of Integral Studies and Stanford University. In Secolsky, C. and Denison, D.B. (eds.) Handbook on Measurement, Assessment, and Evaluation in Higher Education. New York: Routledge.

Academic Medicine published an article about the use of empowerment evaluation to he...

ALBERTO CAIRO, PH.D., INTERVIEWED BY JAMES PANN, PH.D.

 

 

Alberto responds to the following questions during the recording:

00:00 – Introduction

01:24 – Data visualization and it’s importance

07:18 – Patterns that you find yourself repeating when creating visual data

15:30 – Situations in which annotations would be used to explain findings

18:53 – Signs that your chart is overcomplicated and methods to make complicated charts more understandable

24:18 – There is the famous quote by Feynman- “The first principle is that you must not fool yourself and you are the easiest person to fool.” Suggestions for chart developers to help them be honest with themselves.

28:33 – Importance of the cognitive domain of attention from a designer point of view

34:34 – Resources and trainings that you suggest to become literate on the topic or gain “graphicacy”

35:37 – Your books and how to keep up with your work

   

BLOG POST:

https://evalnetwork.com/blog/data-talks-with-james-pann-episode-6-alberto-cairo/

 

 

CONNECT WITH JAMES:

– Subscribe to YouTube channel: https://www.youtube.com/user/jamesmpann/videos

– LinkedIn: https://www.linkedin.com/in/pannjames/

– Twitter: https://twitter.com/jpann

Please reach out with comments and feedback! Thanks!

 

 

Interview with Sheila Robinson Ed.D. about survey development and use by James Pann, Ph.D.

 

 

OUTLINE:

00:00 – Introduction

01:34 – Creating surveys with a “Respondent-Centered Approach”

04:53 – Insightful tips to create better surveys

08:47 – Making sure respondents understand questions as intended

11:03 – Example of misleading results due to poorly created survey

13:25 – Biases that show up when surveys are used in program evaluations. Biases show up when survey designer inputs their own beliefs or demographic in the creation of the survey

18:00 – Using either an existing survey or creating a new one depending on what needs to be measured 22:41 – When would a survey would not be the right tool to measure data

26:02 – Choosing a survey software

29:30 – Direction or tips for new survey creators

33:00 – How to connect with Sheila

 

PODCAST LINKS:

* Twitter

* sheilabrobinson.com

 

 

CONNECT WITH JAMES:

* Subscribe to this YouTube channel

* LinkedIn

* Twitter

Please reach out with comments and feedback! Thanks!

 

 

Watch the video recording of our interview here:

Data Talks with James Pann, Episode 4 is with Sameet Kumar, Ph.D., a psychologist at the Memorial Cancer Institute, serving the patients of the Memorial Healthcare System in south Broward County. We focus on Sameet’s use of data to make professional and personal decisions.

Sameet has specialized in working with adults who have cancer as well as their caregivers and families for over 15 years. In addition to being a clinical psychologist, he has also studied with numerous Hindu and Buddhist teachers as part of his training. His professional interests include mindfulness meditation, resilience, well-being, and grief and bereavement.

Sameet is the author of the bestselling Grieving Mindfully: A Compassionate and Spiritual Approach to Coping with Loss, and The Mindful Path Through Worry and Rumination, which is available in several languages. His most recent book is Mindfulness for Prolonged Grief.

He can be followed on Twitter @sameetkumar, Facebook page Sameet Kumar, Ph.D., and [email protected].