Improving Development Evaluation

Improving Development Evaluation

Download on the App Store

Improving Development Evaluation episodes

  • Oxfam Canada (www.oxfam.ca) fails to evaluate its 2 projects worth $13,140,000: Care Connect and Camino Verde

    Sign the petition here to require Oxfam to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode, Dr. Wand reviews the Performance Measurement Frameworks (PMFs) for 2 Oxfam Canada (www.oxfam.ca) projects: Care Connect https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p012493001 at a cost of $5,950,000 and Camino Verde https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p005232001 at a cost of $7,190,000. He concludes from his review that Oxfam Canada has failed to design proper outcome indicators (with the exception of 2 outcome indicators out of 18 outcome indicators for the Care Connect project) to meet the Development Assistance Committee (DAC) criteria of achieving effectiveness or impact for these 2 projects. In addition, Oxfam Canada has failed to provide any outcome indicators in its PMF for 10 of the 14 outcomes listed in its Camino Verde project. Therefore, Oxfam Canada cannot make the claim that their projects are achieving their expected outcomes of effectiveness and impact.

    Dr. Wand has sent this episode along with his critique of the PMFs and the 2 original PMFs to the Secretary of State for International Development as well as the shadow critics for the Conservative party, the New Democratic Party, and the Bloc Quebecois party. He has recommended to all of these individuals that Government of Canada funding for these 2 projects be stopped immediately until proper outcome indicators are designed so that proper evaluation of the effectiveness and impact of these projects can be achieved.

    Oxfam Canada has also been sent this episode as well as the critique of the PMFs and the 2 original PMFs inviting them to respond to the critique, including an invitation to attend a future podcast to discuss their flaws in their project PMF outcome indicators as well as their proposed actions to address these flaws.

    If you wish to receive your discount on the purchase of sun hats, sunglasses, or polo shirts designed for the Global South, select the link below:

    https://mydeals.page/1hjl

    If you wish to make a donation to this podcast click on either of the links below:

    https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    https://buymeacoffee.com/davidwand

    Listeners can receive the PMFs and my critique of the PMFs by sending an email to: [email protected]

    46 min
  • Global Community Engagement Resilience Fund (www.gcerf.org) fails to evaluate its $5 Million USAID funding

    In this episode, the Global Community Engagement Resilience Fund (www.gcerf.org) fails to evaluate its $ 5 Million USD received from the United States Agency for International Development. Although GCERF claims on its website to be achieving its 4 expected outcomes through funding its implementing partners in developing countries to deliver services to achieve those outcomes, they provide no evidence in the form of outcome indicators and any data related to those outcome indicators on its website. The 4 expected outcomes that GCERF claims to be achieving with this $5 Million USAID funding are:

    1. Increased social cohesion
    2. Increased ability to mobilize, organize, represent their own interests (agency)
    3. Increased knowledge, skills, as well as subsequent increases in income/access to livelihoods.
    4. Increased confidence, critical thinking, life skills, self-worth, resilience (sense of purpose).

    In addition, letters from the USAID and U.S. State Department indicate that no records exist specific to results frameworks/outcome indicators and related data for these expected outcomes for this $5 Million USD given to GCERF from the US AID.

    GCERF is falsely claiming to the American people on their website with their taxpayer funds that they are achieving these expected outcomes because they provide no evidence on their website of achieving these outcomes in the form of outcome indicators and data to show that these expected outcomes have been achieved.

    As a result, the Improving Development Evaluation podcast has mailed a letter along with copies of the letters from the USAID and U.S. State Department to the Chairman of the Foreign Affairs Committee, Brian Mast, requesting that he request GCERF to post all their results frameworks, including all outcome indicators and data for those outcome indicators that measure the achievement of their 4 expected outcomes, from all the grants/projects produced by GCERF in collaboration with their implementing partners that have come from the original $5 Million USD that GCERF received from USAID up to September 15, 2023 specific to Activity ID; 77266; Project #: SCTOOP21VC004; International Sector: Government & Civil Society; USG Sector: Counter-Terrorism; Managing Agency: Department of State; Funding Agency: USAID

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    15 min
  • Doctors of the World: Flaws in its Performance Measurement Framework - Access to health rights Burkina Faso

    Sign the petition here to require Doctors of the World to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode, Dr. Wand reviews the Government of Canada-funded $14 Million Doctors of the World https://medecinsdumonde.ca/ project entitled Access to health rights in Northern Burkina Faso. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P008979001

    From a review of the project's Performance Measurement Framework (PMF), he concludes that Doctors of the World cannot claim that their project is responsible for achieving any of the 8 expected outcomes due to the absence of control or comparison groups as well as invalid or missing outcome indicators for some of the 8 expected outcomes for the project.

    Dr. Wand has emailed the link to this episode as well as the PMF and summary Excel critique to the Minister of International Development, Doctors of the World, and the shadow critics for international development for the Conservative Party, the NDP, and the Bloc Quebecois.

    Dr. Wand has also invited Doctors of the World to respond to his critique in writing and/or be attending his podcast.

    Listeners can receive the PMF and summary critique by emailing [email protected].

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #international development

    #evaluation

    32 min
  • UNFPA Somalia reproductive health project: Flaws in the Performance Measurement Framework

    Sign the petition here to require UNFPA to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode, Dr. Wand reviews the Government of Canada-funded $10 Million UNFPA Somalia project entitled Strengthening Sexual Reproductive Health and Rights through Midwives. Details about the project can be found here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p007483001

    He reviews the Performance Measurement Framework (PMF) for this project and notes flaws in the outcome indicators whereby the UNFPA Somalia country office cannot claim that they have achieved the following project outcomes:

    • Reductions in maternal mortality;
    • Increased quality of midwifery training
    • Increased skill of midwifery associations to lead and advocate;
    • Increased knowledge and awareness of maternal nutrition.

    However, properly designed outcome indicators in the PMF do support the opportunity to show the achievement of increased access to and use of sexual reproductive health services, although actual data needs to be provided.

    Listeners can receive the PMF and the full summary of the Excel outcome indicators by emailing [email protected]. Those documents and this podcast will be shared with the Minister for International Development as well the shadow critics for the Conservative party, the NDP, the Bloc Quebecois, and the Green Party.

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #international development

    #UNFPA Somalia

    #evaluation

    27 min
  • Season 2: Trailer

    In this trailer, Dr. Wand outlines the following focus and changes in format and content for Season 2:

    • The focus of the this podcast is to expose those Government of Canada funded international development organizations that are falsely claiming that their development outcomes are being achieved by reviewing their Performance Measurement Frameworks (PMFs). The podcast will not be reviewing charity or emergency projects such as the payment of school fees or cash transfers or the delivery of emergency services such as food, water, medical care, and shelter.
    • Each episode will identify which of the following OECD Development Assistance Committee evaluation criteria have not been achieved: Relevance, Coherence, or Impact.
    • The podcast will begin reviewing United States Agency for International Development projects by looking at Activity Monitoring Evaluation Learning Plans that are equivalent to the Canadian PMFs.
    • Format changes will include: Combining the project description and a critique of its PMF/Activity MEL Plan into 1 episode; Focusing on a single international development organization with multiple projects that have poor evaluation design to invite to respond to our critique with evaluation experts.
    • Dr. Wand invites experts in Outcome Harvesting, Most Significant Change, and Appreciative Inquiry to be interviewed to explain their contribution to proper evaluation by emailing him at [email protected].
    • Dr. Wand will continue to email his critiques and episodes to the Minister for International Development and all other political parties as well as the international development organizations as proper evaluation is an issue for all Canadians.
    • All PMFs/US AID Activity MEL Plans and their critiques are available upon request through [email protected].

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Buy Me A Coffee here to support the podcast: buymeacoffee.com/DavidWand

    16 min
  • UNDP Bangladesh: Episode 8 Part 2 Evaluation experts discuss flaws in the PMF and provide solutions on how the evaluation of the project can be improved

    Sign the petition here to require UNDP to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 8, part 2, Dr. Wand critiques the UNDP Performance Measurement Framework (PMF) for a $30 Million project in Cox's Bazar District Bangladesh entitled Fighting environmental degradation and promoting greener energy. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010741001. He concludes that the PMF is flawed and that UNDP www.undp.org needs to improve the design of its outcome indicators since only 2 out of the 9 outcome indicators are properly designed.

    He provides alternative indicators for the remaining 7 outcome indicators that are flawed that will actually evaluate the project properly. Finally, he suggests that even the Liquefied Petroleum Gas (LPG) should be replaced by either solar cookers or biogas which has been used in other refugee camps as an even greener energy compared to LPG.

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #UNDP

    #internationaldevelopment

    #evaluation

    22 min
  • UNDP Bangladesh- Flaws in its Performance Measurement Framework PMF

    Sign the petition here to require UNDP to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 8, part 1, Dr. Wand critiques the UNDP Performance Measurement Framework (PMF) for a $30 Million project in Cox's Bazar District Bangladesh entitled Fighting environmental degradation and promoting greener energy. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/p010741001. He concludes that the PMF is flawed and that UNDP www.undp.org needs to improve the design of its outcome indicators since only 2 out of the 9 outcome indicators are properly designed. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique by emailing [email protected] . Finally, he invites a UNDP representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project.

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #internationaldevelopment

    #evaluation

    16 min
  • Equitas: Episode 7 Part 3 Evaluation experts discuss flaws in their PMF and provide solutions on how the evaluation of the project can be improved

    Sign the petition here to require Equitas to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 7, part 3, Dr. Wand continues his discussion with evaluation expert Yvonne Okeke about Equitas www.equitas.org Performance Measurement Framework (PMF) for a $18 Million project in Burkina Faso, Haiti, Kenya, Tanzania, Senegal entitled Achieving Equality through Human Rights Education. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P006852001 The experts conclude that Equitas cannot make the claim that their project is achieving its outcomes because only 1 of their 20 outcome indicators in their PMF properly measures the project outcomes.

    This discussion focuses on why Equitas chose to promote gender equality in 5 countries that are not democracies, according to the 2023 Democracy Index with the 2023 Freedom House index noting that Kenya, Tanzania, and Senegal were only 'partly free.'

    They recommend that this project promoting gender equality might wish to focus on democratic countries in Africa such as Ghana, Lesotho, Namibia, South Africa, and Botswana, according to the 2023 Democracy Index.

    They also discuss the 2 outcome indicators that, in combination, properly measure and evaluate the achievement of improvements in leadership skills to mobilize communities in promoting gender equality.

    The PMF and complete summary of all 20 outcome indicators for this project is available by emailing [email protected].

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #internationaldevelopment

    #evaluation

    28 min
  • Equitas: Episode 7 Part 2 Evaluation experts discuss flaws in their PMF and provide solutions on how the evaluation of the project can be improved

    Sign the petition here to require Equitas to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 7, part 2, Dr. Wand discusses with evaluation expert Yvonne Okeke Equitas www.equitas.org Performance Measurement Framework (PMF) for a $18 Million project in Burkina Faso, Haiti, Kenya, Tanzania, Senegal entitled Achieving Equality through Human Rights Education. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P006852001 The experts conclude that Equitas cannot make the claim that their project is achieving its outcomes because only 1 of their 20 outcome indicators in their PMF properly measures the project outcomes. This discussion focuses on the flaws with the PMF and provides solutions to how the PMF and the overall evaluation of the project can be improved. In brief, those solutions include:

    • Replace your self-reporting bias (i.e. perceived capacity, levels of confidence, transferring knowledge) from your project beneficiaries (i.e. women, men, women organizations, civil society organizations, others) with external, objective measures of their technical capacity to promote and lobby for gender equality.
    • Replace your subjective and frequency measures reported by the duty bearers with objective behaviours such as increased expenditures on gender equality programs 0r legislation passed that reflects better gender equality. Your current measures of empowerment in the PMF are not valid measures of empowerment.
    • Either introduce comparison groups or measure your target groups more frequently to support your claims that your project outcomes have been achieved due to the project.

    The PMF and complete summary of all 20 outcome indicators for this project is available by emailing [email protected].

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    #internationaldevelopment

    #evaluation

    32 min
  • Equitas - Flaws in its Performance Measurement Framework PMF

    Sign the petition here to require Equitas to put all its PMFs and related data on its website:

    www.change.org/EvaluateCanadaAid

    In this episode 7, part 1, Dr. Wand critiques the Equitas Performance Measurement Framework (PMF) for a $18 Million project in Burkina Faso, Senegal, Kenya, Tanzania, and Haiti entitled Achieving Equality through Human Rights Education. You can learn more about the project here: https://w05.international.gc.ca/projectbrowser-banqueprojets/project-projet/details/P006852001 He concludes that the PMF is flawed and that Equitas www.equitas.org cannot make the claim that its project services have achieved its project outcomes. He also describes the services that the project delivers and invites listeners to request a copy of the PMF and his critique. Finally, he invites a Equitas representative to attend Part 2 of the episode to respond to his critique and provide solutions as to how to improve the evaluation of the project.

    https://mydeals.page/1hjl

    Donate here to increase the number of organizations that receive performance audits like this one: https://www.paypal.com/ncp/payment/ZAQD8888DEDXL

    Or at buymeacoffee.com/davidwand

    16 min

About Improving Development Evaluation

From the publisher's feed

Join Dr. David Wand as he takes you on a provocative, investigative journey challenging Canadian & American international development organizations as well as international organizations toโ€ฆ