Improving Development Evaluation

Improving Development Evaluation

Download on the App Store

Improving Development Evaluation episodes

  • Colleges & Institutes Canada: Episode 6 Part 2 Evaluation experts discuss flaws in their PMF and provide solutions on how the evaluation of the project can be improved

    Sign the petition here to require Colleges & Institutes Canada to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 6, part 2, Dr. Wand discusses with evaluation expert Dr. Jenny Jorgenson Colleges & Institutes Canada www.collegesinstitutes.ca Performance Measurement Framework (PMF) for a $18 Million project in Senegal entitled A Thousand women: I am woman, I exist, I participate. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P006852001. The experts conclude that Colleges & Institutes Canada cannot make the claim that their project is achieving its outcomes because only 1 of their 17 outcome indicators in their PMF properly measures the project outcomes. This discussion focuses on the flaws with the PMF and provides solutions to how the PMF and the overall evaluation of the project can be improved. In brief, those solutions include:

    • Replace your self-reporting bias of your project beneficiaries (i.e. vulnerable women) from your focus groups with measuring others (e.g. community members, others within their households) on awareness of the need for women to participate in decision-making bodies;
    • Measure women's incomes and their ability to make decisions that show their power before you claim that your project women have increased their 'empowerment'. Your current measures of empowerment in the PMF are not valid measures of empowerment.
    • Either introduce comparison groups or measure your target groups more frequently to support your claims that awareness levels or empowerment levels have increased. Currently, you only measure your target groups once per year in your PMF. This is inadequate to support your claim that your leadership training and entrepreneurship training are responsible for increases in women empowerment.

    The PMF and complete summary of all 17 outcome indicators for this project is available by emailing [email protected].

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    43 min
  • Colleges & Institutes Canada: Flaws in its Performance Measurement Framework PMF

    Sign the petition here to require Colleges & Institutes Canada to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 6, part 1, Dr. Wand critiques the Colleges & Institutes Canada www.collegesinstitutes.ca Performance Measurement Framework (PMF) for a $18 Million project in Senegal entitled A Thousand women: I am, I exist, I participate. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P006852001 He concludes that the PMF is flawed and that Colleges & Institutes Canada cannot make the claim that its project services have achieved its project outcomes. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique. Finally, he invites a Colleges & Institutes Canada representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project.

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/NE935D6QQ7WLU

    Or at buymeacoffee.com/davidwand

    15 min
  • Mennonite Economic Development Associates (MEDA): Episode 5, Part 2 Evaluation experts and MEDA discuss flaws in their PMF and provide solutions.

    Sign the petition here to require MEDA to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 5, Part 2, Dr. Wand discusses with evaluation expert Benjamin Serebour and MEDA representative Yasir Dildar the Mennonite Economic Development Associates www.meda.org Performance Measurement Framework (PMF) for a $19,040,307 Million project in Africa, Americas, and Asia entitled A New Partnership for Sustainable Impact Investing in Frontier Markets. You can learn more about this project at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/D000120001. This discussion focuses the flaws with the PMF and provides solutions to how the PMF can be improved. In brief, those solutions include:

    • Stop using self-reporting/Most Significant Change from individuals trained/mentored by MEDA as a method to claim that these individuals have experienced increased levels of skill/knowledge/technical ability. Replace this with objective, technical measures of technical ability. If this is too expensive, take a sample. If these individuals and organizations refuse this, do not provide the training and mentoring. You have that leverage.
    • For organizations or 'entities' that MEDA claims to be increasing their standards of reporting on gender and environmental issues due to their training/mentoring, replace the number reporting on these standards with the percentage achieving this minimum standard using your technical audits and put this in your PMF.
    • MEDA cannot find comparison groups that wish to participate in being measured on their project outcomes. Fair enough. Take your treatment group and use half of them as your comparison group and deliver your training/mentoring to that group later. See Chapter 5 Quasi-Experimentation Handbook of Practical Program Evaluation Second Edition ISBN 0-7879-6713-0 for other options without using comparison groups.
    • Include country level reporting in your PMF as there are bound to be variations in performance between countries.https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    59 min
  • Mennonite Economic Development Associates: Flaws in its Performance Measurement Framework PMF

    In this episode 5, Part 1, Dr. Wand critiques the Mennonite Economic Development Associates www.meda.org Performance Measurement Framework (PMF) for a $19,040,307 Million project in Africa, Americas, and Asia entitled A New Partnership for Sustainable Impact Investing in Frontier Markets. He concludes that the PMF is flawed and that the Mennonite Development Associates cannot make the claim that its project services have achieved its project outcomes. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique. Finally, he invites a Mennonite Development Associates representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project.

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    10 min
  • Ghana Rural Integrated Development Episode 4, Part 2: Dr. Wand discusses flaws in their Performance Measurement Framework and provides solutions.

    Sign the petition here to require Ghana Rural Integrated Development to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this Part 2 of Episode 4, Dr. Wand discusses flaws in the Ghana Rural Integrated Development international development organization https://grid-nea.org/ Performance Measurement Framework PMF for its $1,337,797 project in Ghana entitled Improving the Well-Being of Rural Women and Girls in Ghana. Dr. Wand concludes that their PMF is not sufficient to support GRID's claim that it has achieved its project outcomes. He propose solutions on how to improve the evaluation of the project. Finally, Dr. Wand invites listeners to receive a copy of the PMF and his summary critique of the project's outcome indicators by emailing him at [email protected]. In order to improve development evaluation, Dr. Wand recommends that all PMFs should be made available to the public on the Global Affairs Canada project browser and that all organizations use the solutions discussed on this episode when designing their PMFs. Further details about this GRID project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P005964001. Notice that the PMF for this project is not available on the Global Affairs Canada project browser!!!

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    23 min
  • Ghana Rural Integrated Development: Flaws in its Performance Measurement Framework PMF

    In this episode 4, Part 1, Dr. Wand critiques the Ghana Rural Integrated Development www.grid-nea.org Performance Measurement Framework (PMF) for a $1.337 Million project in Ghana entitled Improving the Well Being of Rural Women and Girls in Ghana. He concludes that the PMF is flawed and that theGhana Rural Integrated Development cannot make the claim that its project services have achieved its project outcomes. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique. Finally, he invites a Ghana Rural Integrated Development representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project.

    You can make a donation to this podcast if you wish Dr. Wand to continue monitoring the proper evaluation of foreign aid projects here: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    17 min
  • UNFPA South Sudan Episode 3 Part 2: Evaluation experts discuss flaws in their Performance Measurement Framework and provide solutions.

    Sign the petition here to require UNFPA to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this Part 2 of Episode 3, Dr. Wand discusses with evaluation expert Benjamin Serebour flaws in UNFPA South Sudan's https://southsudan.unfpa.org/ Performance Measurement Framework PMF for its $37,000,000 project in South Sudan entitled Human Resources for Sexual Reproductive Health in South Sudan. They conclude that their PMF is not sufficient to support UNFPA's claim that it has achieved its project outcomes. They propose solutions on how to improve the evaluation of the project. Finally, Dr. Wand urges listeners to contact the Minister for International Development at [email protected] as well as the shadow critics [email protected], [email protected], and [email protected] to require all organizations to post their PMFs on the Global Affairs Canada project browser and have all organizations implement the solutions discussed when designing their PMFs. Further details about this UNFPA South Sudan project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P007519001.

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    44 min
  • Canadian Bureau for International Education Episode 2 Part 2 Evaluation experts discuss flaws in their Performance Measurement Framework and provide solutions.

    Sign the petition here to require Canadian Bureau for International Education to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this Part 2 of Episode 2, Dr. Wand discusses with evaluation expert Hayat Askar flaws in the Canadian Bureau for International Education http://www.cbie.ca/ Performance Measurement Framework PMF for its $5,000,000 project in Canada entitled African Leaders of Tomorrow Scholarships Fund. They conclude that the Outcome Indicators are not sufficient to support the claim that the Canadian Bureau for International Education has achieved its project outcomes. They propose solutions on how to improve the evaluation of the project. Finally, Dr. Wand urges listeners to contact the Minister of International Development at [email protected] as well as the shadow critics at [email protected], [email protected], and [email protected] to post online all Performance Measurement Frameworks and relevant evaluation plans on the Global Affairs Canada project browser for all organizations that they fund. Further details about this Canadian Bureau for International Education project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/D000980001. You will notice that the Performance Measurement Framework is not there!!

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    32 min
  • CARE Canada Episode 1 Part 2: Evaluation experts discuss flaws in their Performance Measurement Framework and provide solutions.

    Sign the petition here to require CARE Canada to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this Part 2 of Episode 1, Dr. Wand discusses with evaluation experts Donald Cole and Dr. Mark Degner flaws in CARE Canada's Performance Measurement Framework PMF for its $3,400,000 project in Vietnam entitled Advancing Women's Economic Empowerment. They conclude that 'stories of change' may be necessary but are not sufficient to support CARE Canada in its claim that it has achieved its project outcomes. They propose solutions on how to improve the evaluation of the project. Finally, Dr. Wand urges listeners to contact CARE Canada at [email protected] to request they post all their project PMFs on their website and to implement the solutions discussed. Further details about this CARE Canada project can be found at https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P007336001

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/NE935D6QQ7WLU

    Or at buymeacoffee.com/davidwand

    40 min
  • United Nations Population Fund UNFPA South Sudan: Flaws in its Performance Measurement Framework PMF

    In this episode 3, Part 1, Dr. Wand critiques a United Nations Population Fund UNFPA www.unfpa.org Performance Measurement Framework (PMF) for a $37 Million project in South Sudan entitled Human Resources for Sexual Reproductive Health Rights and Gender-Based Violence in South Sudan. He concludes that the PMF is flawed and that the United Nations Population Fund UNFPA cannot make the claim that its project services have achieved its project outcomes. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique. Finally, he invites a United Nations Population Fund UNFPA representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project. An evaluation expert will also be attending Part 2 of the episode to provide comments on the UNFPA PMF. More details about the UNFPA South Sudan project funded by the Government of Canada can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P007519001

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    27 min

About Improving Development Evaluation

From the publisher's feed

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations toโ€ฆ