Creative Language Technologies

This is episode #21 of the podcast and it’s Thursday, the 7th of July, 2022. 

My guest today is Dr. Matthias Michel, a philosopher working at the Center for Mind, Brain, and Consciousness at the New York University. Although most of his research focuses on the scientific study of consciousness, he is also interested in non-human / animal consciousness. Matthias has also worked in the domain of 'philosophy of measurement' as it applies to the measurement of mental properties, especially in psychiatric research (for instance, in the measurement of fear and anxiety).

We started the show by defining consciousness — discussing the ways in which we can assess our own conscious experience. Since the subjective aspect of consciousness makes its scientific study very challenging, I asked Matthias to summarize for us the current assessment methods used in the field. One such method (the use of subjective reports), although somewhat controversial, proves to be particularly important in applications like mental disorders (specifically, fear and anxiety). Matthias believes that cognitive neuroscience research on consciousness could give us a deeper understanding of mental disorders and their treatments. 

The second part of the interview covered technology where we focused in particular on the questions ‘can AI systems be conscious? And, if yes, how?’ Matthias kindly shares with us his field’s perspective as well as his own opinion on the topic.

Here is the show.

Show Notes:
- Defining consciousness; Is human experience always conscious?
- Does being conscious presuppose being aware? (And what kind of awareness is needed here?)
- What makes the scientific study of consciousness challenging?
- Applications: mental disorders (like fear and anxiety): how can the scientific field of consciousness help?
- Subjective reports in the assessment of conscious experience
- Technology: Can AI be conscious? How?

Note:

Relevant papers:

Michel, Matthias. "The Mismeasure of Consciousness: A problem of coordination for the Perceptual Awareness Scale." Philosophy of Science 86.5 (2019): 1239-1249.

Taschereau-Dumouchel, Vincent, et al. "Putting the “mental” back in “mental disorders”: a perspective from research on fear and anxiety." Molecular Psychiatry 27.3 (2022): 1322-1330.

Lau, Hakwan. In Consciousness we Trust: The Cognitive Neuroscience of Subjective Experience. Oxford University Press. 2022.

 Link to Dr. Michel’s website:
 https://matthias-michel.wixsite.com/michel

This is episode #20 of the podcast and it’s Thursday, the 23rd of June, 2022. 

Today I sat down, virtually, of course, with Dr. Michael Mopas, Associate Professor in the Department of Sociology and Anthropology at Carleton University in Ottawa, Ontario. He is cross-appointed to the Institute of Criminology and Criminal Justice and the Department of Law and Legal Studies, and serves as a member of the Duncombe Studio for Social and Cultural Research. Most of Michael’s work is in the area of science, technology, and law with a focus on 'sound'. In his spare time, he plays upright bass in several jazz bands in the city and participates in long-distance triathlons. He has completed two Ironman races (Lake Placid and Mont-Tremblant) and several other events.

In our discussion, we turned to wearable fitness-trackers and mobile apps that allow athletes to measure, monitor, visualize, and record a variety of training metrics. Dr. Mopas and his collaborators have looked into the deeply embodied and sensory dimensions of self-tracking. The overall insight seems to be that the data generated by self-trackers are not only cognitively processed, but also sensed and felt by users. While we do have some understanding of what exactly self-tracking devices measure and quantify, we know less about how/when do their users know these quantitative metrics work for them. How do we mitigate the potential dissonance between these quantitative metrics and the athletes’ lived experiences?

The second part of the discussion moved toward technology. We talked about the future of wearable self-tracking devices and debated if AI can be employed to better understand the emotional needs of the user.  Here is the show.

Show Notes:

- Do wearable tracking devices take too much of our ability of ‘being in the moment’?
- Should we value quantitative metrics over other ways of knowing and making sense?
- Moments of dissonance: self-tracking device’s quantitative metrics vs. the athletes’ lived experience
- Sharing data (e.g., Strava): potential benefits and limitations
- Technology: Can we develop AI that learns qualitative data that people input into such devices?

Note:

Relevant papers:

Mopas, Michael S., and Ekaterina Huybregts. "Training by feel: wearable fitness-trackers, endurance athletes, and the sensing of data." The Senses and Society 15.1 (2020): 25-40.

Lupton, Deborah, and Sarah Maslen. "The more-than-human sensorium: sensory engagements with digital self-tracking technologies." The Senses and Society 13.2 (2018): 190-202.

Link to Dr. Mopas’ web page: 

https://carleton.ca/socanth/people/mopas-michael/

This is episode #19 of the podcast and it’s Thursday, the 9th of June, 2022. 

I interviewed Dr. Elaine Auyoung, Donald V. Hawkins Professor and Associate Professor of English at the University of Minnesota, and Affiliate Faculty of the Center for Cognitive Sciences. She is the author of “When Fiction Feels Real: Representation and the Reading Mind”, recently released in paperback from Oxford University Press. In addition to the project on “Unselfing” described on her faculty webpage, Elaine is also working on a book project on “Becoming Sensitive”, which uses cognitive research on expertise and perceptual learning to show how training in the arts and humanities prepares learners to notice and respond to information in ways that are important for future problem solving but have been difficult to assess.

We had a fascinating discussion on many important topics covered in Elaine’s book, but the episode went over the usual podcast duration, so I’ve decided to split it into two parts. This is Part II.

We continued our discussion from Part I on how to bridge the gap between readers’ experience and the experience of firsthand perception — i.e., how well can we know what we don’t experience directly? We then moved to the vocabulary of 'bereavement' addressing the question ‘What happens when novels end?”  As always, we concluded with a discussion on technology covering e-books, multimedia experiences, and VR. Elaine was also happy to introduce us to her current project on ‘Becoming Sensitive’ where she debates the importance of the Arts and Humanities in preparing learners to make and represent information in ways that are important for future problem solving.

Here is the show.

Show Notes:

- how well can we know what we don’t experience directly? (and limitations of language)
 - leaving room between imagination and experience 
- the vocabulary of 'bereavement': What happens when novels end? 
- the future of digital books and the future of experiencing fiction in VR: 1st person vs. 3rd person immersion
- the role of language in immersive environments 
- Elaine’s current project, ‘Becoming Sensitive’
- perceptual learning: the ability to differentiate one’s experience (preparing us for noticing or for being sensitive)

Dr. Auyoung’s faculty webpage:
https://cla.umn.edu/about/directory/profile/eauyoung

 Link to paperback edition of Dr. Auyoung book:
 https://www.amazon.com/When-Fiction-Feels-Real-Representation/dp/0197621279/ref=sr_1_1?crid=1MMDE5NRJ6GV4&keywords=when+fiction+feels+real&qid=1650048589&sprefix=when+fiction+feels+real%2Caps%2C96&sr=8-1

This is episode #18 of the podcast and it’s Thursday, the 26th of May, 2022.
Today, I interviewed Dr. Elaine Auyoung, Donald V. Hawkins Professor and Associate Professor of English at the University of Minnesota, and Affiliate Faculty of the Center for Cognitive Sciences. She is the author of “When Fiction Feels Real: Representation and the Reading Mind”, recently released in paperback from Oxford University Press. In addition to the project on “Unselfing” described on her faculty webpage, Elaine is also working on a book project on “Becoming Sensitive,” which uses cognitive research on expertise and perceptual learning to show how training in the arts and humanities prepares learners to notice and respond to information in ways that are important for future problem solving but have been difficult to assess.

We had an amazing time covering many important topics from Elaine’s book, but the episode went over the usual podcast duration, so I’ve decided to split it into two parts.  We started with the definition of Reality Novels and ways to describe the experience of reading. For Tolstoy, for instance, readers’ effort to comprehend the characters’ sensations and emotions as fully as possible is an aesthetic and ethical end in itself. However, the way in which such writers convey the felt experience of the fictional worlds has remained relatively underexplored. 

We then looked into the extent to which literary experience depends on the the knowledge and abilities that readers bring to a text, one one hand, and how much it relies on the set of strategies employed by the skillful writer, on the other hand. Another point we addressed was the importance of the translation process in maintaining the level of immersive experience of reading.

In Part II, we talked about how to bridge the gap between the readers’ experience and the experience of firsthand perception — I.e., how well can we know what we don’t experience directly? As always, we concluded with a discussion technologies like e-books and multimedia experiences. 
This is Part I of the episode.

Show Notes:
- what is realist fiction?
- what is the experience of reading?
- how do we get from words on a page to the reader’s immersive experience of the story
- the knowledge and abilities that readers have vs.the set of strategies employed by the skillful writer
- how cognitively taxing is reading a novel?
- the importance of the sensory properties of a language that inevitably get lost in translation

 Link to Dr. Auyoung's book  (paperback edition) :
https://www.amazon.com/When-Fiction-Feels-Real-Representation/dp/0197621279/ref=sr_1_1?crid=1MMDE5NRJ6GV4&keywords=when+fiction+feels+real&qid=1650048589&sprefix=when+fiction+feels+real%2Caps%2C96&sr=8-1

This is episode #17th of the podcast and it’s Thursday, the 12th of May, 2022. 

I sat down with Dr. Sally Warmington, a retired medical doctor and narrative researcher from Melbourne, Australia with a deep interest in the encounter between health professionals and those seeking care. Her experiences as a doctor, student, teacher and patient inform her research at the intersection of anthropology, cultural theory and linguistics. Her book "Storytelling encounters as medical education: crafting relational identity" demonstrates the key role of storytelling in medical education and practice and its potential to promote collaborative, better quality care.

We had a fascinating discussion on many important topics covered in Sally’s book, but the episode went over the usual podcast duration, so I’ve decided to split it into two parts.  This is Part II.

We continued our discussion from Part I on ethics in clinical practice, focusing on the concept of ‘ethical mindfulness’ — how to bring in and apply ethical values in research. We then moved to the importance of medical students’ reflective writing, process which allows students to become more aware of the emotional and relational aspects of the clinical encounter. And, in this context, we explored various options to the questions “Do patients’ stories matter? And, are doctors aware of their importance?” Dr. Warmington argues for a universal adoption of practices that promote dialogic engagement between medical students, clinical teachers, and patients, giving us lots of examples from her own research and clinical experience.

Here is the show.

Show Notes:

- ethical mindfulness in research and clinical practice

- reflective writing in medical teaching practice

- how should we approach medical care?

- toward a universal adoption for practices that promote dialogic engagement between students, clinical teachers, and patients.

- Can empathy be taught? (The need to look for another way of framing the issues around the clinical encounter)

- detached concern vs. engagement practice in medical encounters

Note:

Link to Dr. Warmington’s book: 

 "Storytelling encounters as medical education: crafting relational identity" is available on the Routledge website: 

https://www.routledge.com/Storytelling-Encounters-as-Medical-Education-Crafting-Relational-Identity/Warmington/p/book/9781032177007#

Her research papers can be found on Google Scholar.

This is episode #16 of the podcast and it’s Thursday, the 28th of April, 2022. 

Today, I sat down with Dr. Sally Warmington, a retired medical doctor and narrative researcher from Melbourne, Australia with a deep interest in the encounter between health professionals and those seeking care. Her experiences as a doctor, student, teacher and patient inform her research at the intersection of anthropology, cultural theory and linguistics. Her book "Storytelling encounters as medical education: crafting relational identity" demonstrates the key role of storytelling in medical education and practice and its potential to promote collaborative, better quality care.

We had a fascinating discussion on many important topics covered in Sally’s book, but the episode went over the usual podcast duration, so I’ve decided to split it into two parts.  

We started talking about the importance of storytelling in clinical context and that of identity work — how storytelling tells something about who we are in relation to others. The focus here is not only how people become doctors, but also what kinds of people emerge from contemporary clinical training and how students play an active part in their own identity construction. We also talked about the role the analyst plays in such research — their training, language, cultural orientation, memories as an investigator influencing the study. As investigators, Sally believes, we should apply what she calls “ethical mindfulness” — paying close attention to the participants, but also to our own sensations, perceptions and emotional responses, process which requires an acute self-awareness and an awareness of others. 

In Part II, we talked about the connection between storytelling and empathy, its role in clinical settings and medical education, and the importance of helping students develop an awareness of the emotional and relational aspects of the clinical encounter. 

This is Part I of the episode.

Show Notes:
- the role of stories and storytelling in clinical practice
- identity construction; identity dissonance
- empathy and the formation of professional identities
- “ethical mindfulness” and cognitive and sensory self-awareness in clinical practice
- the analyst’s role in clinical research

Note:
Link to Dr. Warmington’s book: 

 "Storytelling encounters as medical education: crafting relational identity" is available on the Routledge website:
https://www.routledge.com/Storytelling-Encounters-as-Medical-Education-Crafting-Relational-Identity/Warmington/p/book/9781032177007#

This is episode #14 of the podcast and it’s Thursday, the 24th of March, 2022. 

My guest today is professor Charles Spence, a world-famous experimental psychologist with a specialization in neuroscience-inspired multisensory design. He has worked with many of the world’s largest companies across the globe since establishing the Crossmodal Research Laboratory (CRL) at the Department of Experimental Psychology, Oxford University in 1997. Prof. Spence has published over 1,000 academic articles and edited or authored, 15 books (a sample is provided in the Notes). His work focuses on the design of enhanced multisensory food and drink experiences, through collaborations with chefs, baristas, mixologists, chocolatiers, perfumiers, and the food and beverage, and flavour and fragrance industries. Prof. Spence has worked extensively in the world of multisensory experiential wine and coffee and has also worked extensively on the question of how technology will transform our dining/drinking experiences in the future.

We started the discussion addressing how many senses do humans have, after which we jumped right into important questions related to sense harmony, sensory overload, sense congruency, dominance, and harmony. 

The second part of the discussion moved toward technology (as I usually like to do), brainstorming about how can we use the senses to provide the best immersive experience in augmented, virtual, and mixed reality. Here is the show.

Show Notes:

- How many senses do humans have and use
- Sensory dominance: Are vision and hearing our most important and most complex senses?
- Looking at sensory combinations
- How do senses interact?
- How much do we know about individual differences in somatosensory stimulation and perception?
- Considering cross-modal brain plasticity
- The role of technology (i.e., AI / AR / VR) in shaping our awareness and use of the senses
- Ethical implications

Note:

Links to some of Dr. Spence’s more recent (and popular) books:

1) Spence, Charles. 2017. Gastrophysics: The new science of eating. Penguin Viking.  
[International bestseller; winner of the 2019 Le Grand Prix de la Culture Gastronomique from Académie Internationale de la Gastronomie. ]

2) Spence, Charles. 2021. Sensehacking - How to Use the Power of Your Senses for Happier, Healthier Living. Penguin UK. 

This is episode #13 of the podcast and it’s Thursday, the 10th of March, 2022.

My invited speaker today is Dr. Nicole Redvers, a member of the Deninu K’ue (“Deneh-noo-kweh”) First Nation in Denendeh. She has worked with Indigenous patients, scholars, and communities around the globe her entire career. Dr. Redvers is an assistant professor in the Department of Family and Community Medicine and the Department of Indigenous Health at the University of North Dakota where she helped develop and launch the first Indigenous health PhD program. Dr. Redvers is co-founder and current board chair of Canadian charity the Arctic Indigenous Wellness Foundation based in Yellowknife, NWT, providing traditional Indigenous-rooted Land-based wellness supports to northerners. She has been actively involved at regional, national and international levels promoting the inclusion of Indigenous perspectives in both human and planetary health research and practice. She authored the trade paperback book titled, ‘The Science of the Sacred: Bridging Global Indigenous Medicine Systems and Modern Scientific Principles’.

We started the discussion with the inclusion of indigenous perspectives in planetary health and sustainable healthcare education, and then we narrowed the topic to the revitalization of traditional wellness services in the Canadian north with a focus on the preservation of Traditional Medicines. She hopes her book will help build bridges between traditional knowledge and western medicine. 

We’ve also talked about the considerable interest (and even some initiatives out there) to create immersive experiences to teach indigenous languages and bring awareness to indigenous knowledge and perspective, and to the traditional healing practices in a land-based setting. Here is the show.

Show Notes:

- promoting the inclusion of indigenous perspectives in planetary health and sustainable healthcare education. 
- defining Traditional Medicine(s) 
- building “bridges” between traditional knowledge and western medicine 
- Indigenous languages and their connection to the land and wellness 
- immersive experiences to teach indigenous languages and bring awareness to indigenous knowledge and perspective in a land-based setting

Note:

Link to Dr. Redver’s book:
 https://www.penguinrandomhouse.com/books/599277/the-science-of-the-sacred-by-nicole-redversnd/

Paper mentioned:

Nicole Redvers, Yuria Celidwen, Clinton Schultz, Ojistoh Horn, Cicilia Githaiga, Melissa Vera, Marlikka Perdrisat, Lynn Mad Plume, Daniel Kobei, Myrna Cunningham Kain, Anne Poelina, Juan Nelson Rojas, Be’sha Blondin. The determinants of planetary health: an Indigenous consensus perspective. Lancet Planet Health 2022; 6: e156–63

This is episode #12 of the podcast and it’s Thursday, the 24th of February, 2022.

My guest today is Dr. Jay Friedenberg, Professor of Psychology at Manhattan College where he founded and directs the Cognitive Science Program, and where he had served as Department Chairperson for over a decade. Dr. Friedenberg is a vision researcher and has published articles on symmetry detection, center of mass estimation and empirical aesthetics. In addition,  he has written a number of science books. These include undergraduate texts in cognitive science, artificial intelligence and non-linear dynamics. He is also an artist focusing on pastel landscapes and urban sketching and is serving his third year as President of the Haiku Society of America.

We started the discussion with his book on "Understanding Aesthetics, Creativity and the Arts: An Interdisciplinary Approach”, addressing the challenges in defining beauty, art, and creativity. After considering whether beauty is sensory in nature or more intellectual, we debated if beauty and art can be studied scientifically, and addressed the role of intuition in art creation.

The second part of the discussion moved toward technology. Since AI has found its way into the world of art for quite some time now, we looked at how it has already influenced the art industry. Is AI going to replace artists or is there always going to be some human intervention required to create art? And, if so, is the art ecosystem going to equally welcome both kinds of art creation?  

Looking in the future, Jay shared his opinion on the role of AI in the art space in the next 10-20 years, debating who should own the copyright and the ethical implications of AI in creating art. Here is the show.

Show Notes:

- Defining beauty, art, and creativity
- Is beauty sensory/emotional in nature or more intellectual? What is the role of intuition in art creation?
- Can beauty and art be studied scientifically?
- AI and the art industry
- The role of AI in the art space in the next 10-20 years
- Should AI own the copyright? 
- Ethical implications of AI in art creation

Note:

Links o Jay Friedenberg's books on Amazon:
1) Jay Friedenberg. The Future of the Self: An Interdisciplinary Approach to Personhood and Identity in the Digital Age. 1st Edition. University of California Press. 2020. https://www.amazon.com/gp/product/B08KCW7319/ref=dbs_a_def_rwt_hsch_vapi_tkin_p1_i4
2) Jay Friedenberg. Understanding Aesthetics, Creativity and the Arts: An Interdisciplinary Approach. 2020.
https://www.amazon.com/gp/product/B08G1SG16R/ref=dbs_a_def_rwt_hsch_vapi_tkin_p1_i2 

Jay's art:  
bigapplearts.com