This episode accompanies the last module of the course – module 15, which is all about building an evidence-based management capacity in organisations. 

There are 3 levels – you as an evidence-based manager; your colleagues who you can influence and guide in evidence based approaches; and the skills, practices and processes in your organisation.

We hear from several experts about ways to build on what you’ve learned, and how others have approached the development of evidence-based practices in their organisations, including the CIPD, the professional HR body, about how it promotes this practice among its membership.

And you’re never alone, there are always others forging a path in evidence-based management who you can reach out to, learn from and share with.

Host: Karen Plum

 Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Jonny Gifford, Senior Researcher, CIPD, the professional body for HR and people development with a worldwide community of 160k members
• David Creelman, CEO of Creelman Research and an expert of HR strategy, analytics and learning
• Stefanie Nickel, Global Head of Diversity & Inclusion, pharmaceutical manufacturer Sandoz, part of the Novartis Group
• Steven ten Have, Partner TEN HAVE Change Management & Professor of Strategy and Change at VU University Amsterdam

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 14 of the course, which is about evaluating the outcome of the decision we’ve taken or the solution we implemented. This is the last stage of our evidence-based management process and is vital to ensuring that we learn from what we’ve done.

Assessing outcomes is vital, because otherwise, how do we know if what we did was effective, and how can we learn and develop our approach to decision making?  Did we capture a baseline before the decision was implemented? And was the decision implemented as planned? If we assess outcomes without these two, then conclusions could be very suspect.

We continue our case study of the large trial at pharmaceutical organization Sandoz (part of Novartis Group), and find out how the D&I interventions were assessed.

Denise Rousseau's 2020 paper: Making Evidence based organisational decisions in an uncertain world 

After action review paper: https://www.cebma.org/wp-content/uploads/Guide-to-the-after_action_review.pdf 

Host: Karen Plum

 Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Stefanie Nickel, Global Head of Diversity & Inclusion, pharmaceutical manufacturer Sandoz, part of the Novartis Group

Additional speaker, courtesy of CIPD: Niamh McNamara, Global Head of People & Organisation, Novartis 

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 13 of the course, which is about incorporating evidence into the decision making process.

Is the evidence appropriate for our situation (is it generalizable). Can we (and should we) action  the evidence? And what are the practical aspects associated with using the evidence that we’ve gathered? The real world is messy and complex, so there are many things to take into consideration.

We share real world examples  from professional HR body CIPD, Creelman Research and a large trial at pharmaceutical organization Sandoz (part of Novartis Group), which shows not only the power of capturing stakeholder experiences, but the value of keeping interventions simple and practical.  

Host: Karen Plum

 Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Jonny Gifford, Senior Researcher, CIPD, the professional body for HR and people development with a worldwide community of 160k members
• David Creelman, CEO of Creelman Research and an expert of HR strategy, analytics and learning
• Stefanie Nickel, Global Head of Diversity & Inclusion, pharmaceutical manufacturer Sandoz, part of the Novartis Group

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 12 of the course, which is about how we bring together the various sources of evidence that we've gathered.

The purpose of taking an evidence-based approach is to reduce uncertainty in our decision making, looking at likelihoods and probabilities to guide our thinking and discussions. The use of Bayes rule and Bayesian thinking are explored, so that we continue to protect ourselves from falling prey to bias (particularly confirmation bias), but that we consider alternative explanations for the evidence that we found - if our initial belief is either true or false.

The use of probabilities isn't something our brains take to easily, so there is some challenge inherent in this approach, but it is simply an extension of the overall evidence-based management approach, where we look at each type of evidence and consistently question whether it is trustworthy, robust and reliable. Once we reach the 'aggregate" stage, it's time to ask how likely is it that the claim or hypothesis we are investigating is true (or false).


Further reading / sources mentioned during the episode:

• Michal Oleszak - "On the Importance of Bayesian Thinking in Everyday Life"
• Daniel Kahnneman - "Thinking Fast and Slow"
• Nate Silver - "The Signal and the Noise"

Host: Karen Plum

 Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University

Additional material with thanks to:

• Julia Galef - President and co-founder of the Center for Applied Rationality - YouTube videos 

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 11 of the course, which is about appraising evidence from stakeholders – the people affected by our decision or the people who are able to influence it. The acquisition of this evidence is covered in Module 10 and its corresponding podcast episode.

In this episode we are reminded that sometimes we engage with certain parties as practitioners (those with experience in the problem or solution we’re working with) and sometimes as stakeholders – with feelings and perceptions about how our decision will impact them, and how they might want to affect it, if they have sufficient power. 

We hear several examples from the guests about stakeholder situations  which show how things can be quite complicated, with lots of things at stake. We consider a number of ethical considerations, because at the end of the day, even though it may be possible to make a decision and implement a solution, it is also necessary to ask – should we? 

Stakeholder evidence has the potential to throw us off course, perhaps slowing down our timetable and complicating matters. But consider the consequences if we don’t know their perspectives and concerns or if we haven't weighed up whether their views are representative. There may be ethical, political, organisational and societal outcomes that could do irreparable harm. 

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Steven ten Have, Partner TEN HAVE Change Management & Professor of Strategy and Change at VU University Amsterdam
• Dr Lisa J Griffiths, CEO, OzChild National Support Office

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 10 of the course, which is about acquiring evidence from stakeholders – the people affected by our decision or the people who are able to influence it. The appraisal of this evidence is covered in Module 11 and its corresponding podcast episode.

In this episode we explore the nature and extent of stakeholder individuals and groups – from governments and regulators, to organisational shareholders, the people implementing the decision and those that will live with the consequences. Rather than assuming that we know how these groups will react to our plans or solutions, we have to consult them. 

In the experience of the guests, there is much to be learned which will inform and enrich the decision making process – including gaining an understanding of the complexities of implementing solutions and unearthing consequences we might not have anticipated.

Stakeholder engagement, assessment and management are all critical aspects of successfully implementing change – and understanding their points of resistance can help to unblock obstacles or just make the process smoother and the outcome more effective.

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Dr Lisa J Griffiths, CEO, OzChild National Support Office
• Steven ten Have, Partner TEN HAVE Change Management & Professor of Strategy and Change at VU University Amsterdam

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 9 of the course, which is about the appraisal of the data, information and evidence gathered within the organisation. The acquisition of this evidence is covered in Module 8 and its corresponding podcast episode.

In this episode we continue to consider some of the wider aspects associated with the use of organisational data. Why we can’t take it’s quality for granted; how it can be highly misleading if we don’t know where it comes from and how it’s created, and how people can continue using it for years in ignorance of its shortcomings.

The excitement around Big Data and data analytics is also discussed, to see whether these are all they are cracked up to be!  And we look at the complexity associated with key performance indicators (KPIs) which again can lock us into repetitive behaviour, without questioning their value.

Once again, the importance of taking decisions based on multiple sources of evidence is reinforced. 

Link to the video mentioned by Jeroen Stouten

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Martin Walker, Director of Banking & Finance at Center for Evidence-Based Management
• Jeroen Stouten, Professor of Organisational Psychology, KU Leuven University
• Ravishanker Jonnalagadda, Senior Expert Data Science, People  Analytics , Novartis Healthcare 

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 8 of the course, which is about the data, information and evidence gathered within the organisation to aid our multi-source evidenced-based decision making. The appraisal of this evidence is covered in Module 9 and its corresponding podcast episode.

In this episode we consider some of the things we need to think about before either using existing organisational data or collecting data specifically geared to address our question or area of interest. As with other sources of evidence, there are a myriad of obstacles and roadblocks to be aware of in our search for organisational data and information to inform our decision making. 

There is a tendency for people to place organisational evidence in high regard, but as we will see, this faith can often be misplaced – not for nefarious reasons, but in some cases simply due to a lack of understanding when designing systems, or using data that has been churned out for years without anyone just asking where the data comes from.

As ever, we keep asking questions!

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Martin Walker, Director of Banking & Finance at Center for Evidence-Based Management
• Jeroen Stouten, Professor of Organisational Psychology, KU Leuven University

 Article by Jeffrey Pfeffer & Robert Sutton 

Find out more about the course here:   https://cebma.org/resources-and-tools/course-modules/ 

This episode accompanies Module 7 of the course, which is about judging the quality of the evidence from the scientific literature that we’ve identified during our earlier searching. This module aligns to chapter 7 of the Evidence-Based Management book.

Modules 5, 6 and 7 all focus on the scientific literature, so when you listen to the corresponding podcast episodes, the picture will hopefully be complete. 

In this episode we look at the process of learning how to appraise the studies, and what their findings mean in the context of the real world that we are faced with. If something is statistically significant and there is a large effect size measured by the study, does that mean that the finding is practically relevant for our organisation? 

Evidence from the scientific literature is undoubtedly important, but shouldn’t be emphasised more than other sources of evidence and it’s good to be reminded that it’s only when these come together than we can really make an evidence-based decision.

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Rob Briner, Professor of Organizational Psychology, Queen Mary University of London
• Lisa J Griffiths, CEO, OzChild National Support Office

Find out more about the course here.

This episode accompanies Module 6 of the course, which is an introduction to the scientific literature that can be used as part of our evidence-based management decision making. This module aligns to chapter 5 of the Evidence-Based Management book.

Modules 5, 6 and 7 all focus on the scientific literature, so when you listen to their corresponding podcast episodes, the picture will hopefully become more complete. 

In this episode we look at different aspects of the scientific world – what motivates academics to study the topics they research; the pros and cons of lab and field studies; and how to approach academic studies to get the most from them (don’t try to read them cover to cover!). We also discuss the importance of statistical significance and effect sizes in research and their practical relevance in the real world.

Host: Karen Plum

Guests:

• Eric Barends, Managing Director, Center for Evidence-Based Management 
• Denise Rousseau, H J Heinz University Professor, Carnegie Mellon University
• Rob Briner, Professor of Organizational Psychology, Queen Mary University of London

Find out more about the course here.