Sign up to save your podcasts
Or
Share Naturalistic Decision Making
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Naturalistic Decision Making
Naturalistic Decision Making episodes
- Episode #9; Interview with Peter Fadde and Olivia Brown
Show Description:
Peter Fadde is professor and director of the Learning Systems Design and Technology (LSDT) graduate program at Southern Illinois University.
Olivia Brown is a Research Associate at the University of Bath, School of Management.
Where to find Peter:
Accelerating the Acquisition of Intuitive Decision-Making through Expertise-Based Training (XBT)
Instructional Design for Accelerated Macrocognitive Expertise in the Baseball Workplace
TEDx talk "Frugal Engineering in Baseball and Beyond"
Association for Applied Sport Psychology: Virtual Conference on Technology in Applied Sport Psychology Practice
Where to find Olivia:
University of Bath
Teamwork in extreme environments: identifying challenges and generating solutions
CREST Research
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
“Extreme team” research premise [2:30]
Forming expedition teams for team research [4:45]
Team research data and observations [6:15]
Prepping research participants on how to share their data [9:05]
Peter describes the background of his award-winning research in expertise-based training [10:10]
Explaining the video occlusion method [21:30]
How good are experts at articulating their expertise? [27:00]
Debate between immersive experience training vs. targeted skill training [31:00]
How Olivia got into Naturalistic Decision Making [33:00]
Justifying NDM’s unconventional research methods [35:10]
Advice for those looking to conduct similar NDM studies [36:00]
Olivia's background in extreme environments and what drew her to “extreme team” research [38:15]
What "extreme team" research can teach us about how people work together generally [40:45]
Applying sports-based NDM research to real life [42:15]
Peter compares STEM training to sports training [46:45]
None of us are immune, and maybe de-stigmatizing racism will lead more people to confront it in ourselves [53:15]
"If both of you could instantly become an expert in something, what would it be?" [56:30]
1 hr - Episode #8: Interview with Jan Maarten Schraagen
Date: 5/22/2020
Show Description:
“Jan Maarten is Principal Scientist at TNO and Professor of Applied Cognitive Psychology at the University of Twente, The Netherlands. His research interests include resilience engineering, team communication processes, and human-machine teaming. He is the lead editor on two influential volumes: Cognitive Task Analysis (2000) and Naturalistic Decision Making and Macrocognition(2008). He is co-editor of the recently released Oxford Handbook of Expertise (2020). He is editor in chief of the Journal of Cognitive Engineering and Decision Making. Dr. Schraagen holds a PhD in Cognitive Psychology from the University of Amsterdam, The Netherlands.”
Where to find Jan Maarten:
University of Twente
TNO
LinkedIn
JCEDM
The Oxford Handbook of Expertise
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
What is the first paper you ever published? [2:00]
Experience conducting research with children [4:11]
Nature of Jan Maarten's 1993 research study on expertise [6:15]
Working with Herb Simon at Carnegie Mellon [10:57]
What led Jan Maarten to attend the 1994 NDM conference in Dayton [13:00]
Discussing the birth and significance of the book "Cognitive Task Analysis" [17:25]
Experience and advice for navigating difficult research interviews [20:35]
Interviewing technique advice for students [23:52]
The most exciting project Jan Maarten is working on right now [27:47]
Goals and directions for future work [36:50]
What is a "cyber-physical system"? [40:20]
Industries that could benefit from an NDM perspective [42:30]
Which project has been the most rewarding for you? [47:50]
Evaluating the success of Dutch navy training simulations [54:50]
One questions that can reveal if someone is an NDM practitioner [57:06]
Two truths and a lie [58:30]
1 hr 3 min - Episode #7: Interview with Larry Shattuck
Date: 5/15/2020
Show Description:
“Colonel Shattuck is Senior Lecturer and Director of the Human Systems Integration Program at the Naval Postgraduate School in Monterey, California. Dr. Shattuck holds a faculty appointment in the Operations Research Department where he teaches human systems integration and human factors engineering. Colonel Shattuck graduated from the United States Military Academy in 1976, and following distinguished service as a signal officer and receiving his PhD from the Ohio State University in Cognitive Engineering, he returned to his alma mater to teach Engineering Psychology until his retirement from the Army in 2005. He has been at NPS since, he co-directs the Human Systems Integration Program. Larry has been an active researcher in the domain of military command and control for nearly two decades. He is unique among his peers in the NDM community for his ability to draw on deep operational experience when studying processes such as how commanders communicate their intent, the manner in which decision makers fuse data in tactical environments, and the ways in which data and information flow through technological system elements to the human agents in that system.”
Where to find Larry:
Naval Postgraduate School
HSI DL Certificate ProgramHSI DL Master’s Degree ProgramLearn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
How Larry's military operational experience has informed his research findings [2:20]
Making the shift to engineering psychology [7:47]
Takeaways from the Army research laboratory's advanced decision architecture program [10:12]
Examples of the program's contributions to the Army [13:58]
Making progress with NDM solutions within a government context [19:10]
Explaining the relationship between NDM and Human Systems Integration [29:07]
Making the NDM research process more efficient [34:40]
How NDM perspectives have influenced Larry's approach to teaching [38:26]
The aspect of Larry's work that has been most rewarding [40:10]
Lessons learned from students [43:45]
Larry's top three mentors [45:46]
Influences outside of NDM [49:25]
One question that can determine if someone is an NDM practitioner [51:07]
Two truths and a lie [52:08]
56 min - Episode #6: Interview with Julie Gore
Date: 5/15/2020
Show Description:
“Today we welcome Julie Gore. Julie Gore is a Reader in Organizational Psychology, at the School of Management, University of Bath in the UK. She is a Chartered Psychologist and Fellow of the British Psychological Society. Her research focus is on the psychology of expertise and Naturalistic Decision Making (NDM) across a range of professions working under uncertainty. She is Associate Editor for Journal of Occupational and Organizational Psychology and serves on the boards of the British Journal of Management and Frontiers in Organizational Psychology. Julie is also an academic advisor for Nasdaq’s Behavioural Science Lab. Dr Gore received one of the world’s first NDM PhD degrees in Applied Cognitive Psychology from Oxford Brookes University, UK.”
Where to find Julie’s work:
The Oxford Handbook of Expertise
The University of Bath
Naturalistic Decision Making and Uncertainty
Naturalistic decision making: navigating uncertainty in complex sociotechnical work
SPRITE+
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Experience as one of the first NDM PhD program participants [1:46]
How Julie initially heard about NDM [3:03]
Julie's PhD research question [3:55]
Discoveries made during PhD research [5:18]
Advice for young NDM PhD students [6:15]
Common barriers students need help managing [8:02]
Surprises and concerns about today's generation of NDM researchers [10:10]
Examples of PhD students whose perspectives have been transformed by NDM models [12:05]
Making space for NDM research papers in journals and publications [14:52]
Nature of Julie's work with NASDAQ [16:32]
Exciting research on decision-making ethics and accountability in the digital space [19:23]
Organizations' and outsiders’ attitudes toward NDM [23:17]
Showcasing the utility of NDM methods [25:18]
A particularly rewarding project [27:10]
An explanation of "adaptive expertise" [29:12]
Opportunities to advance NDM in the context of today's novel circumstances [32:37]
Three people who influenced Julie's career [33:47]
Influences from outside NDM [34:43]
Changes in the NDM community [36:35]
One question that can determine if someone is an NDM researcher [37:30]
Directions for future research [39:37]
Two truths and a lie [41:02]
45 min - Episode #5: Interview with Robert Hutton
Date: 5/7/2020
Show Description:
“Today we welcome our friend, Rob Hutton. Rob is co-founder and director of Trimetis Ltd, a four-person cognitive systems engineering consultancy in Bristol, England. Rob and I started our careers together as young research assistants at Klein Associates in the early 1990s. His thinking contributed to early cognitive task analysis methods and he helped articulate models of macrocognition. Rob is a Fellow of the Chartered Institute of Ergonomics and Human Factors. He is also a part-time lecturer in Nottingham Trent University’s psychology department.”
Where to find Robert:
Trimetis
Nottingham Trent University
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Starting an NDM business in England [1:30]
Types of client organizations [3:30]
A project that Rob is excited to be working on [6:05]
Challenges conveying the value of NDM theory to students [8:45]
What’s the craziest situation you’ve found yourself in, in the name of science? [16:40]
Challenges of cognitive task analysis field research [23:55]
A rewarding impactful project [25:10]
Success of Co-authored Applied Cognitive Task Analysis paper [29:10]
What is the most valuable proposition that you offer your customers? [29:50]
Cultural differences of NDM research in the US vs. the UK [33:05]
Who are three people who have influenced your approach? [41:40]
What question would you ask someone to determine whether they practice NDM? [46:30]
Two truths and a lie [50:00]
57 min - Episode #4: Interview with Jennifer Phillips
Date: 5/1/2020
Show Description:
“Today we welcome our pal, Jennifer Phillips. Jenni is the CEO of the Cognitive Performance Group, which she cofounded in 2006. Her work focuses expert decision-making, primarily in the military training community. She and her colleagues at CPG have pioneered research into the development of skilled performance, and designed innovation in the areas of decision-centered training and assessment, including the development and application of mastery models. Among her many publications is a chapter in the Oxford Handbook of Expertise exploring mastery models and their value in supporting the acquisition and assessment of expertise.”
Learn More About Jenni:
Cognitive Performance Group
Developing Mastery Models to Support the Acquisition and Assessment of Expertise
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Jenny describes her focus of expertise, including tactical thinking in the marine corps and mastery models [1:30]
Jenni breaks down the term “mastery model” and its stages [3:45]
Jenni gives a mastery model example from her work with marine corps instructors [6:05]
Work on assessment of decision skills, performance rubrics, and the process of adaptability [9:50]
Major influences in Jenni’s career in NDM [14:05]
Other people and organizations outside the NDM community who have impacted Jenni’s work [16:30]
What makes Jenni’s approach and expertise unique in her field [19:15]
Challenges and nuances of working in a military environment [21:30]
Difficulty getting buy-in to conduct research in law enforcement [23:45]
A favorite anecdote from research in IED defeat work [24:45]
Some distinct and difficult nuances of IED defeat work in Afghanistan and Iraq [29:45]
Meaning and application of “cognitive fidelity” [32:50]
A surprising characteristic of young researchers [35:20]
One single question that can determine if someone is an NDM practitioner [39:05]
Two truths and a lie [39:45]
Excitement about research on making mastery models work at a large scale [42:10]
47 min - Episode #4: Interview with Jennifer Phillips
Date: 5/1/2020
Show Description:
“Today we welcome our pal, Jennifer Phillips. Jenni is the CEO of the Cognitive Performance Group, which she cofounded in 2006. Her work focuses expert decision-making, primarily in the military training community. She and her colleagues at CPG have pioneered research into the development of skilled performance, and designed innovation in the areas of decision-centered training and assessment, including the development and application of mastery models. Among her many publications is a chapter in the Oxford Handbook of Expertise exploring mastery models and their value in supporting the acquisition and assessment of expertise.”
Learn More About Jenni:
Cognitive Performance Group
Developing Mastery Models to Support the Acquisition and Assessment of Expertise
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Jenny describes her focus of expertise, including tactical thinking in the marine corps and mastery models [1:30]
Jenni breaks down the term “mastery model” and its stages [3:45]
Jenni gives a mastery model example from her work with marine corps instructors [6:05]
Work on assessment of decision skills, performance rubrics, and the process of adaptability [9:50]
Major influences in Jenni’s career in NDM [14:05]
Other people and organizations outside the NDM community who have impacted Jenni’s work [16:30]
What makes Jenni’s approach and expertise unique in her field [19:15]
Challenges and nuances of working in a military environment [21:30]
Difficulty getting buy-in to conduct research in law enforcement [23:45]
A favorite anecdote from research in IED defeat work [24:45]
Some distinct and difficult nuances of IED defeat work in Afghanistan and Iraq [29:45]
Meaning and application of “cognitive fidelity” [32:50]
A surprising characteristic of young researchers [36:20]
One single question that can determine if someone is an NDM practitioner [40:00]
Two truths and a lie [40:50]
Exciting implications for research on making mastery models work at a large scale [42:50]
47 min - Episode #3: Interview with Emilie Roth
Date: 4/24/2020
Show Description:
“Today we are talking with our friend and colleague, Emilie Roth. I first met Emilie at the second Naturalistic Decision Making meeting in Dayton, OH in 1994, but she had been studying how people manage complexity long before that meeting. In fact, I remember at that meeting thinking: ‘I want to be like her.’ Emilie is one of the people that inspired me to think I really could be a scientist. And I know Emilie has inspired many others over the course of her career. She is one of the architects of the cognitive systems engineering movement. She had an important role in redesigning nuclear power plant control rooms in the wake of Three Mile Island. And that was just the beginning; her contributions have been felt far and wide. She now lives in Palo Alto, CA where she runs Roth Cognitive Engineering.”
Where to find Emily:
Roth Cognitive Engineering
Resources:
Oxford Handbook of Expertise
Designing collaborative planning systems: Putting Joint Cognitive Systems Principles to Practice
Cognitive work analysis
Facets of complexity in situated work
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Emily's first published paper [1:30]
Events that shifted Emilie’s focus from experimental psychology to NDM [7:30]
Developing the observation tools and methods for developing systems of expertise [12:25]
Challenges in creating expert decision-making models for nuclear operations at Westinghouse [17:15]
Experience navigating a world of engineers as a female psychologist [20:00]
The benefits of work culture at Westinghouse [21:45]
Observations of railroad industry culture, practices, and informal procedures [23:00]
Optimistic outlook on specific research and applications of NDM [28:45]
Two things about Emilie’s personal life that others might not know [34:20]
Three books that influenced Emilie’s work [38:00]
Associative thinking exercise [44:15]
Elaborating on automation pros and cons [45:25]
49 min - Episode #2: Interview with Robert Hoffman
Show Description:
“Robert is a luminary in the NDM community, a widely recognized world leader in cognitive systems engineering and Human-Centered Computing, and a senior member and fellow of numerous international science and engineering societies. Since 1999, he has called the Institute for Human and Machine Cognition his organizational home, and he organized the NDM 6 conference in 2003 in his hometown of Pensacola, FL. He has co-authored and co-edited 18 scholarly books – including two with your hosts! – and is co-author on over 100 publications in peer-reviewed journals. For twenty years, he served as Co-Editor for the Department on Human-Centered Computing in IEEE: Intelligent Systems. He was a co-founder of The Journal of Cognitive Engineering and Decision Making, a major publication outlet for NDM research.”
Where to find Robert:
Institute for Human and Machine Cognition
Books by Robert:
Minding the weather: How expert forecasters think.
Cognitive Systems Engineering: The Future for a Changing World.
Perspectives on Cognitive Task Analysis: Historical Origins and Modern Communities of Practice.
Applied concept mapping: Capturing, analyzing, and organizing knowledge.
Learn more about NDM:
NaturalisticDecisionMaking.org
Journal of Cognitive Engineering and Decision Making
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Timestamps:
Robert's first published paper [1:45]
Event’s that led to Robert’s interest in NDM [5:15]
Robert’s shift from psychometrics [14:00]
NDM research project that Robert is most excited about [17:00]
The falsifiability of NDM and research limitations surrounding the coronavirus [22:40]
Shortcomings of graduate psychology and computer programs [25:30]
Advice for people interested in exploring the NDM field [27:45]
Two experiences and the lessons they imparted about eliciting expert knowledge [29:45]
Some things about Robert that others might not know [35:30]
Reasons why Robert loves football so much [37:45]
Athletes and cognitive task analysis [40:17]
Word association exercise [42:00]
45 min - Episode #1: Interview with NDM Pioneer Gary Klein
Show Description:
“Today we are talking with my friend and business partner, Gary Klein. Gary is one of the pioneers of the Naturalistic Decision Making movement. He is known for moving the study of decision making out of the laboratory and into the wild. In 1989, he invited a small group of international researchers to meet in Dayton, OH to talk about decision making in natural environments and how to study this phenomenon. It was that group of 32 researchers who came up with the term Naturalistic Decision Making.
Gary's many contributions include the recognition-primed decision model that has greatly influenced how we all think about decision making and expertise. He and his colleagues developed early cognitive task analysis methods that have been a core set of tools for studying decision making in real world environments. He has mentored countless researchers, including both of us. Gary now lives in Boston, MA. He runs a company called ShadowBox LLC, and is also part-owner and advisor to Applied Decision Science, LLC.”
Where to find Gary
ShadowBox LLC Website
LinkedIn
Psychology Today Column
Learn more about NDM:
Website: https://naturalisticdecisionmaking.org/
Where to find hosts Brian Moon and Laura Militello:
Brian’s website
Brian’s LinkedIn
Brian’s Twitter
Laura’s website
Laura’s LinkedIn
Laura’s Twitter
Resources mentioned during the show:
Sources of Power by Gary Klein
Seeing What Others Don't: The Remarkable Ways We Gain Insights by Gary Klein
Timestamps:
Gary's first published paper [1:36]
How Gary got started in Naturalistic Decision Making [4:48]
Gary's research on Israeli aircraft maintainers' coping methods during Yom Kippur War [7:55]
Current NDM-related research that Gary is most excited about [12:10]
Two of the biggest mistakes of Gary’s career and how he recovered from them [14:53]
Gary shares his favorite research stores:
-A firefighter whose quick thinking saved several of his colleague’s lives [22:53]
-A British naval officer whose quick, life-saving decision was at first mistaken for ESP [28:00]
-A neonatal nurse's expert instincts helped save a baby's life [33:33]
Suggestions for how the NDM community can help during today's coronavirus crisis 37:15]
Where Gary would prioritize funding and research investment right now [44:05]
Associative thinking exercise [46:18]
How the scientific method is “often a trap” that can limit researchers' learning opportunities [48:55]
54 min
About Naturalistic Decision Making
From the publisher's feed
Brian Moon and Laura Militello interview leading NDM researchers who study and support people who make decisions under stress.