PodQueue

Douglas Hofstadter, the Pulitzer Prize–winning author of Gödel, Escher, Bach, reflects on how he got interested in the mind and consciousness, how he came to write Gödel, Escher, Bach, and why he is terrified by the current state of AI.

CHAPTERS

0:00 Introduction

1:30 How did you first get interested in AI?

5:01 Tell us more about recursion & self-reflection

10:03 How did GEB come to life?

21:21 Which ideas from GEB are most relevant today?

29:23 What about AI terrifies you?

34:35 How have LLMs impacted your view of how human creativity works?

37:14 Is there something analogous in history that terrified people?

38:20 What brings you joy these days?

———————————

📚 ABOUT OUR CHANNEL📚

We deconstruct breakout games & apps to help you innovate smarter and find product/market fit. Hosted by Amy Jo Kim - Game Designer & Startup Coach - prev. Rock Band, The Sims, Covet Fashion Happify, Netflix.

Check out our channel here:

https://www.youtube.com/c/GameThinkingTV/featured

🔔 Don’t forget to subscribe! 🔔

LEARN MORE ABOUT GAME THINKING

Check out our rapid innovation programs for product leaders.👍

https://www.gamethinking.io/programs

Join our free online community 📣 and get in on exclusive free events at

https://www.gamethinkinghub.com

Read our Game Thinking book 📘 at

https://gamethinking.io/book/

FIND US AT 👇

https://gamethinking.io/

GET IN TOUCH 👍

FOLLOW US ON SOCIAL 📱

Get updates or reach out to Get updates on our Social Media Profiles!

https://twitter.com/amyjokim

https://www.linkedin.com/in/amyjokim/

https://amyjokim.medium.com/

Game Thinking TV

Season 2, Episode 1,   Feb 01, 09:19 AM Elizabeth Bernholtz, aka Gazelle Twin, has had paranormal experiences since her early childhood. Ever since she’s been both terrified and thrilled by the occult, gripped by stories of poltergeist possession and famous hauntings. Fresh off the back of her commission for The Horror Show at Somerset House, Gazelle Twin is getting back into the writing process for her next album which explores her long held fascination with ghosts.  We join her as she considers what it would mean to take these stories seriously and to harness her fear as a creative fuel. She talks to artist Mark Leckey, also inspired by the supernatural, about how he uses samples as a form of haunting and how art can be a form of self-exorcism.  Ghost hunter Innes Smith helps unpack the hauntings she experienced in her childhood home. The Process The creative process is inspired by worlds beyond itself.  The Somerset House podcast series 'The Process' brings those worlds together, platforming the big conversations which go on to inspire new work.  Drawing on our creative community on site and from the exhibition programme, each episode follows one artist as they explore an idea from their practice to see where it ends up.  We hear their journey from the studio on, as they invite other thinkers to discuss an idea that has come out of a work in progress and help shape where it might go next.  Producer: Alannah Chance Series Presenter: Laurent John Theme music: Ka Baird Mastered by: Nick Ryan Produced as part of the Creators Programme 2022 Supported by The Rothschild Foundation

Arts & Letters History Peoples & Places Politics & Society Religion & Faith Science & Technology Special Series UP Partners Academic Partners

Large language models like ChatGPT have hit the mainstream, but most applications of today follow the same format: a text box. That's because we're currently in the "MS DOS" stage of interfaces for machine learning – creating interfaces which conform to the shape of the system, not the shape of the human on the other end. This talk explores how AI might help us radically re-think our interaction with computers from the ground up, as the GUI did to command-line computing.

June 13, 2023 3-4:30 p.m. Mountain Time Free webinar In the aftermath of a chaotic Twitter takeover, many people have moved away from centralized social media platforms to a new set of social platforms that are open-source, decentralized, and user-centered—like Bluesky, Mastodon, and Nostr. This shift highlights a pressing need for more trustworthy civic spaces. But civic-minded social platforms are nothing new. The advent of Open Social Media has been many years in the making. This webinar presents some of Open Social Media's origin stories from speakers who have been involved in the development, culture, and communities of their platforms. We will explore how queer experiences and activist movements, for instance, have played a vital role in shaping the design and direction of emerging platforms. Speakers: Facilitated by Nathan Schneider and Skylar Hew. The Open Social Media event series is organized by the Media Economies Design Lab at the University of Colorado Boulder, made possible by a generous gift from Colorado ReWild. The illustration above, "Network of Friends" by kkgas, is licensed through Stocksy, a cooperative stock platform.

John Coltrane (left) and Eric Dolphy on stage at the Village Gate in New York City in the summer of 1961. A recording of the performance, once thought lost, was recently discovered in the New York Public Library.

Herb Snitzer/Courtesy of Impulse! Records

A little over 60 years ago, the editor-in-chief of DownBeat magazine asked John Coltrane and Eric Dolphy a deceptively simple question: What are you trying to do? He rephrased slightly: What are you doing? The two saxophonists sat for a long 30 seconds before Dolphy broke the silence. "That's a good question," he said. The DownBeat editor, Don DeMicheal, printed this exchange in the April 1962 issue, as part of a fascinating article headlined "John Coltrane and Eric Dolphy Answer the Jazz Critics." Regular readers of the magazine would have known precisely what provoked this gesture: a scathing review of Coltrane's quintet with Dolphy, decrying "an anarchistic course in their music that can but be termed anti-jazz." 1961 had been a prolific and pivotal year for Coltrane. That spring, his sleek, intriguing quartet version of "My Favorite Things," from The Sound of Music, became a breakout hit. But later that year, as he signed to a new label, Impulse! Records, he wasn't putting a premium on commercial success. Instead, he was exploring new sounds and configurations, often testing ideas on the bandstand. One such idea was the addition of Dolphy, a wildly original voice on both reeds and flute, and a close personal friend.

The intrepid depth of their musical rapport takes center stage on a stunning new archival release, Evenings at the Village Gate: John Coltrane with Eric Dolphy, which Impulse will release on July 14. Tomorrow the label will share a preview track, "Impressions," featuring Coltrane on soprano saxophone and Dolphy on alto saxophone and bass clarinet — along with drummer Elvin Jones, pianist McCoy Tyner and bassist Reggie Workman, who together make the song feel something like a runaway train. (Until then, you can hear it — exclusively — here.)

"Any time that we worked with John," Workman, who is a few weeks shy of 86, tells NPR, "you could always hear transition in his music." This was maybe never truer than it is here, in a recording made during a month-long residency in late-summer of 1961. Evenings at the Village Gate actually opens with a version of "My Favorite Things," and closes with the droning, polyrhythmic churn of "Africa." Coltrane was in the process of reinventing his language, and by extension the language of jazz. "He was growing into a place where he did not want to be inhibited by the steps and the changes that were prescribed by certain structures," Workman says, adding: "He wanted us to be about a chant."

The Village Gate was a large basement room with a growing reputation in 1961, home to folksingers and comedy acts as well as artists like Nina Simone. Coltrane worked there in August as part of a triple bill, alongside groups led by drummer Art Blakey and pianist Horace Silver. (A photograph of the club marquee by Herb Snitzer shows Coltrane billed with a quartet, underscoring how recently Dolphy had joined the fray.) The Gate had a state-of-the-art sound system, installed by an ambitious young engineer named Richard Alderson. One night during Coltrane's run, Alderson decided to test the system by capturing the band, using a single RCA ribbon microphone suspended above the stage, with a line running to a reel-to-reel tape recorder. The tapes were never intended for public consumption, and unauthorized in any case, so Alderson set them aside. They found their way to a collection at the New York Public Library for the Performing Arts, where they were recently rediscovered by a Bob Dylan archivist.

For Coltrane admirers, jazz historians and anyone intrigued by the experimental end of improvisational music, Evenings at the Village Gate will represent not only a welcome new find but also a link in a chain. The Coltrane-and-Dolphy frontline was short-lived, in part because it faced such strong headwinds from the jazz establishment, but it did leave behind a major testament: Coltrane "Live" at the Village Vanguard, recorded at a different Greenwich Village club in November 1961, the same month that their unruly output jarred loose the indelible phrase "anti-jazz." Those Village Vanguard tapes, which later yielded a monumental four-disc set, amount to one of the most mysterious and thrilling documents in jazz history. A couple of years ago, Ben Ratliff, author of Coltrane: The Story of a Sound, placed this music within a cultural context of "ambivalent possibility," in a vivid essay for the Washington Post titled "John Coltrane and the Essence of 1961." He observes: "The music sounds post-heroic and pre-cynical; interestingly free from grandiosity; full of room for the listener to find a place within it and make up their own mind." Last week, after hearing the version of "Impressions" from Evenings at the Gate, Ratliff elaborated on this idea. "It's very hard to label or encapsulate, but it's just so ferociously full of life force," he said of the performance. "The musicians know how good this is, and they know how exciting it is — but beyond that, they don't really know much, and it hasn't been called anything yet. There's a lot of the unknown here." What came next for Coltrane was the most stable period of his career, as he solidified the personnel of his quartet — Tyner, Jones and bassist Jimmy Garrison, who can be heard on parts of the Village Vanguard corpus — and made venerated albums like Crescent, Ballads and A Love Supreme. Parting ways, Dolphy refocused on his own visionary music, making strong statements until his tragically untimely death, of a diabetic coma, in 1964. (Coltrane died only three years later, of liver cancer.) The last several years have brought revelatory archival releases from both saxophonists, but Evenings at the Gate is a window onto the early bloom of their collaboration, when it must have felt like pure possibility to all involved.

The 80-minute album — which will be released in physical formats with illuminating liner essays by Workman, Alderson, Grammy-winning jazz writer Ashley Kahn, and saxophonists Branford Marsalis and Lakecia Benjamin — seems guaranteed to reignite conversation about an incipient phase in Coltrane's restless evolution. And it's worth recalling part of the answer he finally gave DeMicheal, for the piece in DownBeat. "I think the main thing a musician would like to do is to give a picture to the listener of the many wonderful things he knows of and senses in the universe," Coltrane said, sounding not the least bit defensive. "That's what music is to me — it's just another way of saying this is a big, beautiful universe we live in, that's been given to us, and here's an example of just how magnificent and encompassing it is. That's what I would like to do. I think that's one of the greatest things you can do in life, and we all try to do it in some way. The musician's is through his music."

Steven Cherry Hi, this is Steven Cherry for Radio Spectrum.The coronavirus pandemic has exposed any number of weaknesses in our technologies, business models, medical systems, media, and more. Perhaps none is more exposed than what my guest today calls, “The Hidden World of Legacy IT.” If you remember last April’s infamous call for volunteer COBOL programmers by the governor of New Jersey, when his state’s unemployment and disability benefits systems needed to be updated, that turned out to be just the tip of a ubiquitous multi-trillion-dollar iceberg—yes, trillion with ‘t’—of outdated systems. Some of them are even more important to us than getting out unemployment checks—though that’s pretty important in its own right. Water treatment plants, telephone exchanges, power grids, and air traffic control are just a few of the systems controlled by antiquated code.In 2005, Bob Charette wrote a seminal article, entitled “Why Software Fails.” Now, fifteen years later, he strikes a similar nerve with another cover story that shines a light at the vast and largely hidden problem of legacy IT. Bob is a 30-year veteran of IT consulting, a fellow IEEE Spectrumcontributing editor, and I’m happy to say my good friend as well as my guest today. He joins us by Skype.Bob, welcome to the podcast.Bob Charette Thank you, Steven.Steven Cherry Bob, legacy software, like a middle child or your knees meniscus, isn’t something we think about much until there’s a problem. You note that we know more about government systems because the costs are a matter of public record. But are the problems of the corporate world just as bad?Bob Charette Yes. There’s really not a lot of difference between what’s happening in government and what’s happening in industry. As you mentioned, government is more visible because you have auditors who are looking at failures and [are] publishing reports. But there’s been major problems in airlines and banks and insurance companies—just about every industry that has IT has a problem with legacy systems in one way or another.Steven Cherry Bob, the numbers are staggering. In the past 10 years, at least $2.5 trillion has been spent trying to replace legacy IT systems, of which some seven hundred and twenty billion dollars was utterly wasted on failed replacement efforts. And that’s before the last of the COBOL generation retires. Just how big a problem is this?Bob Charette That’s a really good question. The size of the problem really is unknown. We have no clear count of the number of systems that are legacy in government where we should be able to have a pretty good idea. We have really no insight into what’s happening in industry. The only thing that we that we do know is that we’re spending trillions of dollars annually in terms of operations and maintenance of these systems, and as you mentioned, we’re spending hundreds of billions per year in trying to modernize them with large numbers failing. This is this is one of the things that when I was doing the research and you try to find some authoritative number, there just isn’t any there at all.In fact, a recent report by the Social Security Administration’s Inspector General basically said that even they could not figure out how many systems were actually legacy in Social Security. And in fact, the Social Security Administration didn’t know itself either.Steven Cherry So some of that is record-keeping problems, some of that is secrecy, especially on the corporate side. A little bit of that might be definitional. So maybe we should step back and ask what the philosophers call the the ti esti [τὸ τί ἐστι] question—the what-is question. Does everybody agree on what legacy IT is? What counts as legacy?Bob Charette No. And that’s another problem. What happens is there’s different definitions in different organizations and in different government agencies, even in the US government. And no one has a standard definition. About the closest that we come to is that it’s a system that does not meet the business need for some reason. Now, I want to make it very clear: The definition doesn’t say that it has to be old, or past a certain point in time Nor does it mean that it’s COBOL. There are systems that have been built and are less than 10 years old that are considered legacy because they no longer meet the business need. So the idea is, is that there’s lots of reasons why it may not meet the business needs—there may be obsolescent hardware, the software software may not be usable or feasible to be improved. There may be bugs in the system that just can’t be fixed at any reasonable cost. So there’s a lot of reasons why a system may be termed legacy, but there’s really no general definition that everybody agrees with.Steven Cherry Bob, states like your Virginia and New York and every state in the Union keep building new roads, seemingly without a thought. A few years ago, a Bloomberg article noted that every mile of fresh new road will one day become a mile of crumbling old road that needs additional attention. Less than half of all road budgets go to maintenance. A Texas study found that the 40-year cost to maintain a $120 million  highway was about $800 million. Do we see the same thing in IT? Do we keep building new systems, seemingly without a second thought that we’re going to have to maintain them?Bob Charette Yes, and for good reason. When we build a system and it actually works, it works usually for a fairly long time. There’s kind of an irony and a paradox. The irony is that the longer these systems live, the harder they are to replace. Paradoxically, because they’re so important, they also don’t receive any attention in terms of spend. Typically, for every dollar that’s spent on developing a system, there’s somewhere between eight and 10 dollars that’s being spent to maintain it over its life. But very few systems actually are retired before their time. Almost every system that I know of, of any size tends to last a lot longer than what the designers ever intended.Steven Cherry The Bloomberg article noted that disproportionate spending by states on road expansion, at the expense of regular repair—again, less than half of state road budgets are spent on maintenance—has left many roads in poor condition. IT spent a lot of money on maintenance, but a GAO study found that a big part of IT budgets are for operations and maintenance at the expense of modernization or replacement. And in fact, that ratio is getting higher, that less and less money is available for upgrades.Bob Charette  Well, there’s two factors at play. One is, it’s easier to build new systems, so there’s money to build new systems, and that’s what we we constantly do. So we’re building new IT systems over time, which has again, proliferated the number of systems that we need to maintain. So as we build more systems, we’re going to eat up more of our funding so that when it comes time to actually modernize these, there’s less money available. The other aspect is, as we build these systems, we don’t build them a standalone systems. These systems are interconnected with others. And so when you interconnect lots of different systems, you’re not maintaining just an individual s— you’re maintaining this system of systems. And that becomes more costly. Because the systems are interconnected, and because they are very costly to replace, we tend to hold onto these systems longer. And so what happens is that the more systems that you build and interconnect, the harder it is later to replace them, because the cost of replacement is huge. And the probability of failure is also huge.Steven Cherry Finally—and I promise to get off the highway comparison after this—there seems to be a point at which roads, even when well maintained, need to be reconstructed, not just maintained and repaved. And that point for roads is typically the 40-year mark. Are we seeing something like that in IT?Bob Charette Well, we’re starting to see a radical change. I think that one of the real changes in IT development and maintenance and support has been the notion of what’s called DevOps, this notion of having development and operations being merged into one.Since the beginning almost of IT systems development, we’ve thought about it as kind of being in two phases. There is the development phase, and then there was a separate maintenance phase. And a maintenance phase could last anywhere from 8, 10, some systems now are 20, 30, 40 years old. The idea now is to say when you develop it, you have to think about how you’re going to support it and therefore, development and maintenance are rolled into one. It’s kind of this idea that software is never done and therefore, hopefully in the future, this problem of legacy in many ways will go away. We’ll still have to worry about at some point where you can’t really support it anymore. But we should have a lot fewer failures, at least in the operational side. And our costs hopefully will also go down.Steven Cherry So we can have the best of intentions, but we build roads and bridges and tunnels to last for 40 or 50 years, and then seventy-five years later, we realize we still need them and will for the foreseeable future. Are we still going to need COBOL programmers in 2030? 2050? 2100?Bob Charette Probably. There’s so much coded in COBOL. And a lot of them work extremely well. And it’s not the software so much that that is the problem. It’s the hardware obsolescence. I can easily foresee COBOL systems being around for another 30, 40, maybe even 50 years. And even that I may be underestimating the longevity of these systems. What’s true in the military, where aircraft like to be 50 to, which was supposed to have about a 20 to 25 year life, is now one hundred years old, replacing everything in the aircraft over a period of time.There is research being done by DARPA and others to look at how to extend systems and possibly have a system be around for 100 years. And you can do that if you’re extremely clever in how you design it. And also have this idea of how I’m going to constantly upgrade and constantly repair the system and make it easy to move both the data and the hardware. And so I think, again, we’re starting to see the realization that IT, which at one time—again, systems designers were always thinking about 10 years is great, twenty years is fantastic—that maybe now that these system’s, core systems, may be around for one hundred years,.Steven Cherry Age and complexity have another consequence: Unlike roads, there’s a cybersecurity aspect to all of this as well.Bob Charette Yeah, that’s probably the biggest weakness that that occurs in new systems, as well as with legacy systems. Legacy systems were never really built with security in mind. And in fact, one of the common complaints even today with new systems is that security isn’t built in; it’s bolted on afterwards, which makes it extremely difficult.I think security has really come to the fore, especially in the last two or three years where we’ve had this ... In fact last year we had over 100 government systems in the United States—local, state and federal systems—that were subject to ransomware attacks and successful ransomware attacks because the attackers focused in on legacy systems, because they were not as well maintained in terms of their security practices as well as the ability to be made secure. So I think security is going to be an ongoing issue into the foreseeable future.Steven Cherry The distinction between development and operations brings to mind another one, and that is we think of executable software and data as very separate things. That’s the basis of computing architectures ever since John von Neumann. But legacy IT has a problem with data as well as software, doesn’t it?Bob Charette One of the areas that we didn’t get to explore very deeply in the story, mostly because of space limitations, is is the problem of data. Data is one of the most difficult things to move from one system to another. In the story, we talked about a Navy system, a payroll system ... The Navy was trying to consolidate 55 systems into one and they use dozens of programing languages. They have multiple databases. The formats are different. How the data is accessed—what business processes, how they use the data—is different. And when you try to think about how you’re going to move all that information and make sure that the information is relevant, it’s correct. We want to make sure we don’t have dirty data. Those things all need to come to be so that when we move to a new system, the data actually is what we want. And in fact, if you take a look at the IRS, the IRS has 60-year-old systems and the reason they have 60-year-old systems is because they have 60-year-old data on millions of companies and millions of—or hundreds of millions of—taxpayers and trying to move that data to new systems and make sure that you don’t lose it and you don’t corrupt it has been a decades-long problem that they’ve been trying to solve.Steven Cherry Making sure you don’t lose individuals or duplicate individuals across databases when you merge them.Bob Charette One of the worst things that you can do is have not only duplicate data, but have data that actually is incorrect and then you just move that incorrect data into a new system.Steven Cherry Well, Bob, as I said, you did it before with why software fails and you’ve done it again with this detailed investigation. Thanks for publishing “The Hidden World of Legacy IT,” and thanks for joining us today.Bob Charette My pleasure. Steven.We’ve been speaking with IT consultant Bob Charette about the enormous and still-growing problem of legacy IT systems.Radio Spectrum is brought to you by IEEE Spectrum, the member magazine of the Institute of Electrical and Electronic Engineers.For Radio Spectrum, I’m Steven Cherry.Note: Transcripts are created for the convenience of our readers and listeners. The authoritative record of IEEE Spectrum’s audio programming is the audio version.We welcome your comments on Twitter (@RadioSpectrum1 and @IEEESpectrum) and Facebook.

This session is presented in two parts. The first three speakers will provide evidence of serious and even criminal problems in scholarly publishing. The next three speakers will propose actions researchers, universities and funders are taking to move away from the old model and create a better system.

"Could a Robot Feel Pain?

Race, Technology, and the Political Problem of Personhood"

Presented in partnership with the Center for Spatial and Textual Analysis

In this lecture, recorded on April 5, Sylvester A. Johnson interprets the work of Ibn Rushd ("Averroës"), the twelfth-century Islamic scholar of Andalusia who achieved renown as the "father" of secularism, in order to elucidate and heuristically challenge Ibn Rushd’s theory of the intellect as a power of the soul. Johnson leverages Rushd’s distinction between sensing and knowing in order to examine contemporary, sensory-driven AI technology (particularly brain-computer-interface architectures) as a uniquely generative problem of interest for humanists and technical experts alike. Of central importance is the sensation of pain, as it constitutes a phenomenon of embodiment, a category of human experience, and a political problem that has been implicated in histories of racial domination.

About the Speaker

Sylvester A. Johnson is Assistant Vice Provost for the Humanities and Executive Director of the “Tech for Humanity” initiative at Virginia Tech. He is the founding director of Virginia Tech’s new Center for Humanities, which is supporting human-centered research and humanistic approaches to the guidance of technology. Sylvester’s research has examined religion, race, and empire in the Atlantic world; religion and sexuality; national security practices; and the impact of intelligent machines and human enhancement on human identity and race governance. In addition to co-facilitating a national working group on religion and U.S. empire, Johnson led an Artificial Intelligence project that developed a successful proof-of-concept machine learning application to ingest and analyze a humanities text. He is the author of The Myth of Ham in Nineteenth-Century American Christianity (Palgrave 2004), a study of race and religious hatred that won the American Academy of Religion’s Best First Book award; and African American Religions, 1500-2000 (Cambridge 2015), an award-winning interpretation of five centuries of democracy, colonialism, and freedom in the Atlantic world. Johnson has also co-edited The FBI and Religion: Faith and National Security Before and After 9/11 (University of California 2017). He is a founding co-editor of the Journal of Africana Religions. He is currently producing a digital scholarly edition of an early English history of global religions and writing a book on human identity in an age of intelligent machines and human-machine symbiosis.

Co-sponsored by the Center for Comparative Studies in Race & Ethnicity, the Institute for Human-Centered Artificial Intelligence, African & African American Studies, the Ethics, Society & Technology Hub, and the Department of Religious Studies

Made possible by support from the Office of the President

Learn more:

https://shc.stanford.edu/

https://cesta.stanford.edu/

Bad closed captions can be entertaining, but they can be serious, too, because captions are a critical tool for lots of lots of people. There are the people learning a new language, the easily confused, and of course captions are essential for people who are Deaf or hard-of-hearing. In the US, that’s about 15% of the population.

And with so many low quality closed captions out there, those folks are often left trying to piece together meaning from some bizarre sentences, or … to just stop watching.

The issue of crappy captions isn’t a new thing. Activists have been fighting for accurate, widespread closed captioning for decades.

Bad captions aren’t just annoying; they’re supposed to be illegal, at least on television. In America, the FCC is responsible for making sure the airwaves are accessible for everybody. And since 1996, they’ve required all TV shows to include captions that are “accurate, synchronous, complete, and properly placed.”

Broadcasters in the United States also have to comply with the Americans with Disabilities Act, which passed in 1990. The ADA says all businesses that serve the public, or ‘places of public accommodation,’ must be equally available to everyone, which means broadcast TV needs to be accessible for the Deaf and hard-of-hearing.

Taken together, the FCC and ADA rules are pretty clear cut. For TV, you need to caption, and you need to caption well. And for a while, those rules were enough. But in the twenty-first century, streaming television changed everything. Listen to the episode by clicking the play button above, and/or take a read below!

INTRO

[[MUSIC IN – “Architect’s Medley 2022,” a mid-tempo waltz with strings]]

RM: This is 99% Invisible. I’m Roman Mars. Last fall, the BBC made a huge announcement. They had picked a new lead actor for the legendary TV show Dr. Who.

CLIP: Hello! I’m Ncuti Gatwa, and I am the next Doctor in the next season of Doctor Who.

CB: If you need some context, a new lead actor on Doctor Who is like a new Pope being selected, but for nerds!

RM: That’s producer Chris Berube.

CB: This announcement should have been a triumphant moment. After all, the actor Ncuti Gatwa was the first Black performer cast as the doctor. But when this video came out, the announcement was quickly overshadowed but something that should have been totally mundane. The closed captions.

RM: Gatwa’s first name is spelled N-C-U-T-I. But the closed captions on YouTube replaced his name with a swear word.

CB: So, while the actor said:

CLIP: Hello! I’m Ncuti Gatwa.

CB: The captions autocorrected his name to “Hello I’m [[shitty]] Gatsworth.”

[[MUSIC OUT.]]

CB: That was…unfortunate. But it was far from an isolated incident. When you start looking for Closed Caption failures, you see them pretty much everywhere. I turned on closed captions for a week on broadcast TV, YouTube, pretty much anywhere I was watching stuff. And I noticed so many errors.

[[MUSIC IN – “Alvaar Alto”, a driving up-tempo beat with synthesizers]]

CB: The fictional country in Black Panther, Wakanda, became “War! Canada.” Bill Murray and Dan Akroyd were suddenly part of an elite group called the “Ghost Buses”. It went on and on.

RM: There’s actually an academic, high-minded term to describe this phenomenon.

Linda Besner: Craptions are captions… that are crappy.

CB: That’s Linda Besner. She wrote about craptions for The Atlantic magazine. And she says there are many, MANY ways a caption can be crappy.

Linda Besner: They are not accurate. They don’t differentiate between speakers. So it’s just, sort of, one long run-on sentence. It’s not grammatically correct. It doesn’t contain punctuation. So they are sort of halfway-there captions.

[[MUSIC OUT]]

CB: Look, bad closed captions can be really funny, but captions are a critical tool for lots and lots of people. There’s people learning a new language, people like me who for the life of me cannot understand the Irish accents on Derry Girls. And of course, captions are essential for people who are Deaf and hard of hearing. In the United States, that’s about 15% of the adult population. And with so many craptions out there, those folks are often left trying to piece together meaning from some bizarre sentences, or, to just stop watching.

RM: The issue of crappy captions isn’t a new thing. Activists have been fighting for accurate, widespread closed captioning for decades. And it’s not clear when, if ever, they will become a reality.

HISTORY + ADA

[[MUSIC IN – “Marching Pizzicato”, a mid-tempo pizzicato string arrangement]]

CB: Bad captions aren’t just annoying; they’re supposed to be illegal, at least for television. In America, the FCC is responsible for making sure the airwaves are accessible for everybody. And since 1996, they’ve required all TV shows to include captions that are, quote, [exaggerated reading voice] “accurate, synchronous, complete, and properly placed.”

RM: Broadcasters in the United States also have to comply with guidelines under the Americans with Disabilities Act, which was passed in 1990. The ADA says all businesses that serve the public, or ‘places of public accommodation,’ must be equally available to everyone, which means broadcast TV needs to be accessible for the Deaf and hard of hearing.

CB: Taken together, the FCC and ADA rules were pretty clear. For TV, you need to caption, and you need to caption well. And for a while, those rules were enough. Until the rise of streaming TV.

[[MUSIC OUT]]

CB: Howard Rosenblum is the CEO of the National Association of the Deaf or the N-A-D. Howard himself is Deaf, and we interviewed him over Zoom with the assistance of an ASL interpreter. Throughout this story, you’ll hear her voice representing his answers.

Howard interpreter: Here he comes. There we go! He says [sing song voice] “Hello, Chris.”

CB: Howard points to 2007 as the first time the N-A-D got involved in captions for streaming, when Netflix launched its on-demand service.

Howard interpreter: At the time. If you recall, it was the era of DVDs in the postal mail, right? Ironically, there was no law which required DVDs to be captioned. But I would say roughly 85 to 90% of them were. So many of the members of NAD were Netflix subscribers, we’d get our DVDs in the postal mail, and it was all good. Netflix decided then to pivot to streaming. However, streaming at the time, there was almost none. Maybe 10% of them were captioned.

[[MUSIC IN – “From Scratch”, a worried mid-tempo string arrangement]]

CB: Howard and the NAD approached Netflix about this issue, but at the time, Netflix argued the rules didn’t apply. They weren’t television, so they weren’t under the jurisdiction of the FCC. And Netflix argued the ADA didn’t apply to them either. Here’s writer Linda Besner again.

Linda Besner: Title Three of the ADA says that in places of public accommodation, Deaf and hard of hearing people must be accommodated. So they should have an experience equal to that of a person who is not deaf or hard of hearing. Netflix’s argument was, we are not subject to the ADA because we’re not a place. They were like, well, we’re not an amusement park or like, you know, some kind of public forum. We’re not a physical location, so we don’t have to do this.

CB: The National Association of the Deaf argued that, ok, sure, Netflix is not a physical place, but isn’t it a kind of social place?

Linda: You know, what about a family watching a Netflix show together like, this is a social experience. Why should there be a member of that family who is excluded from that content?

[[MUSIC OUT]]

RM: In 2011, the NAD sued Netflix in a Massachusetts court. The judge agreed that the ADA applied to Netflix, and issued a consent decree, saying Netflix had to provide captions for all of its streaming content within four years.

CB: Today, the full Netflix library is captioned. They have teams of captioners working on every show. And some of their captions are really good, and some of them are only ok. But according to Howard Rosenblum of the NAD, Netflix has done a fairly good job.

Howard interpreter: Once that case was done and dusted, we began to approach other companies like Hulu. Amazon. So all three of them agreed to the terms of a phased-in captioning approach.

[[MUSIC IN – “Persistent Myth”, a slow, melancholy synthesizer arrangement]]

CB: Since the Netflix case, the National Association of the Deaf has been playing a game of whack-a-mole with the internet. Basically, a new streaming thing will crop up, and the NAD will helpfully remind them about their legal requirements to caption stuff.

RM: The NAD has worked with companies like Zoom and Microsoft Teams to encourage live captions for online meetings. And in 2015 the NAD sued Harvard and MIT for not providing captions for their online courses.

CB: Most recently…the NAD sued SiriusXM for not providing transcripts on their podcasts.

Howard interpreter: Nearly all podcasts are recorded, right? They’re not live. There’s no excuse for not providing a transcript.

[[MUSIC OUT]]

RM: Full disclosure here, 99PI is owned by SiriusXM. But I’ll note, our show does provide transcripts for all our episodes.

CB: While there’s been a lot of progress on the issue of streaming captions, there is one internet goliath the NAD won’t go after. It’s actually a place used by 80% of Americans, where people watch billions of hours of content every day. I’m talking about YouTube.

THE YOUTUBE PROBLEM

[[MUSIC IN – “The Maracana”, a slow, off-kilter keyboard arrangement with driving guitar]]

CB: Linda Besner spoke to YouTube about this problem a couple of years ago, and back then, they made it clear – they had no plans to require captions for all of their videos. Here’s Linda.

Linda Besner: They did say, for example, for things like live citizen journalism, like the spokesperson I spoke with gave the example of Arab Spring. You know, if it’s somebody who is posting a video of, you know, some event that is occurring around them, are you really going to flag for removal that video because it doesn’t have transcriptions?

RM: Even though YouTube isn’t legally required to provide closed captions, they have a lot of Deaf and hard of hearing users. In 2009, YouTube tried addressing the problem of closed captions by rolling out AI-generated captions for some of their videos. It hasn’t been a smooth ride.

CB: The automatic captions are created with Google’s Universal Speech Model, which uses speech recognition technology and AI to identify words, and put them together into coherent sentences.

Linda Besner: Predictive text is something that, you know, you may be familiar with from autocorrect on your phone. You started a sentence and then it wants to fill in your blank, much like maybe your mother does. And much like your mother, sometimes they don’t know what you are going to say and you have to be like, [[voice imitating a teenager]] “Mom, that’s not like, I’m not…I’m actually trying to make the opposite point right now.”

[[MUSIC OUT]]

CB: The early days of automatic captions were pretty rough. And Rikki Poynter remembers them well. Rikki is a Deaf YouTuber. And in 2009, she was watching a lot of beauty videos, using the brand new automatic captions.

Rikki Poynter: I remember, when I was watching a video about concealers, and the automatic captions, I had them on, and they were saying “zebra”, the mammal, the animal, whatever, in place of the word “concealers,” which was so bizarre. [[Laughter.]]

[[MUSIC IN – “Vulnerable Grid”, upbeat, slow-moving guitar-driven beat]]

CB: Rikki started her YouTube career making videos about makeup and beauty products. But she wanted to do something different. So in 2014, Rikki produced a video called “Things You Shouldn’t Say To People Who are Deaf or Hard of Hearing,” where she lampoons the clueless questions she was always being asked by hearing folks.

Audio of Rikki from “Things You Shouldn’t Say To People Who are Deaf or Hard of Hearing” – [[Question voice]] Is Deafness contagious? [[Normal cadence]] I don’t know but come over, I’ll cough on you and we’ll find out tomorrow. [[Question voice]] Why don’t you know sign language, you must not really be Deaf? [[Normal cadence]] Well, last time I checked Deafness has to do with whether or not your ears work and not the languages that you know.

CB: A couple of Rikki’s videos focused on the issue of closed captioning. And in one video, Rikki made a call to arms, asking her fellow YouTubers to ditch automatic captions, and write their own captions instead. Or better yet, to hire a professional service to make the captions for them.

RM: The campaign for human closed captions went viral. And Rikki caught the attention of some of YouTube’s biggest stars.

[[MUSIC OUT]]

Archival Audio of Tyler Oakley “Hear Me Out” video: [[Ukelele music playing underneath]] I want to make my little community here on YouTube more inclusive. And one of the ways I have decided to do that in the new year is by closed captioning every single one of my videos.

[[Tyler’s voice fades out underneath.]]

Rikki Poynter: [00:08:38] Tyler Oakley made a video while I was on a plane on the way to L.A., actually. And it was titled “Hear Me Out,” or something, and I was in the thumbnail, and he was talking about the importance of captions. And I had no idea until I had landed in L.A., turned on my phone, and I was getting so many notifications.

Archival Audio of Tyler Oakley “Hear Me Out” video: [[Audio of Tyler fades back in]] I saw a video from a YouTuber named Rikki Poynter talking about the importance of it, and why…[[Tyler’s voice fades out]]

CB: Today, Oakley’s video about closed captions has been watched more than 1.8 million times. And after that, other big time YouTubers weighed in, like Lilly Singh. But Rikki Poynter wanted to push the campaign into a higher gear. So in 2016, she launched a hashtag, called “No More Craptions.”

RM: The term had been floating around the Internet for a while, but Rikki helped bring it to the mainstream. Lots of YouTubers, both Deaf and hearing, made videos pledging to include human-produced captions in their videos.

[[MUSIC IN – “Bonkers Color”, peppy keyboard beat that changes tempo mid-way through, becoming a slow, mournful tune]]

Montage of YouTubers Discussing Closed Captions

First YouTuber: Closed captions help the Deaf and the hard of hearing…

Second YouTuber: Hello, today’s video is about the No More Craptions campaign.

Third YouTuber: Why isn’t this video closed captioned when it’s labeled closed captioned? And I was freaking out, because, [[Exaggerated voice]] Ahhhh…

End of Montage

Rikki Poynter: There’s some big beauty creators that have made announcements that they were going to start captioning. And they would start captioning a couple of videos at a time.

CB: Things were getting better! Human captions were showing up on more and more videos.. Until…some YouTubers just stopped doing it.

Rikki Poynter: As the years went on and like a lot of them would have fallen off of that. So that’s been kind of upsetting. [[Sighs]]

[[MUSIC OUT]]

HUMAN CAPTIONING

RM: Rikki says it was disappointing to see many YouTubers stop captioning their videos. There’s no mystery as to why this happened. Do-it-yourself captions can be time consuming.

CB: Meanwhile, hiring a professional to make your closed captions can be expensive. Professional captioners, on the whole, they aren’t paid very well. But for a YouTuber, dropping $40 or $50 to caption a video can add up pretty quickly. Look, captioning videos at a professional level is hard work.

Emma Healey: Oh, I got headaches. Yeah. And like, yeah, my eyes. I mean, my vision is not great anyway, but it was, it was pretty bad for all that stuff.

CB: My friend Emma Healey started work as a closed captioner in Toronto in 2017.

Emma Healey: The company that I was working for had been subcontracted out by an Australian broadcasting company. So I was captioning with Australian broadcasting rules.

CB: For this job, Emma had to watch a lot of crappy TV.

Emma Healey: A big one that people would complain about was Crocamole, which was an Australian children’s television show about a crocodile that loved guacamole. Named Crocamole.

CB: I got to be honest, our interview devolved into watching Crocamole for like half an hour.

Crocamole song plays, then fades under.

Emma Healey: [[Laughing]] It’s not the most lyrically inventive.

Chris Berube: This is first thought, best thought.

Emma Healey: His eyes are really sad.

CB: Even though Emma was captioning some pretty simple TV shows, the work took a very long time. Emma says she would do about 40 minutes of captioning in an eight hour shift. And that was on a good day.

Emma Healey: The place that I was working there was, I believe, a 75 page manual full of rules, all very technical. And you had to memorize basically all of them.

CB: Emma showed me the manual, which outlines the rules for how captions are supposed to look on Australian broadcast TV. And it made professional captioning seem impossibly byzantine.

[[MUSIC IN – “Mass Trespass”, driving, military-style drum beat]]

Emma Healey: Here we go. Minimum duration is one second no maximum, but 7 to 8 seconds is typical. Do not leave captions on screen overly long for no reason, if the dialog has ended. There must be a one second blank captions starting on the first frame of the program… [[Voice fades out underneath narration]]

CB: It goes on like this. For 75 pages. Emma had to keep all of this stuff in mind, for what was ultimately a minimum wage job.

RM: Professional captioning is complicated, because it’s not just about getting the words right. You also have to nail the punctuation. And the placement of the caption. And lots of other small details. Here’s Linda Besner.

[[MUSIC OUT]]

Linda Besner: So the gold standard for closed captioning would be something where, you know, for example, you and I are talking. When I am talking it might say Linda, colon, and then there would be a capital letter to indicate that I am beginning a sentence. And then it would faithfully represent what I have said, with question marks, where I have asked a question, and commas where commas go. When there is, you know, one of the cats makes a noise in the background [[Cats meowing]], there would be square brackets, cat meowing. If you know, a violent windstorm swept over [[Raindrops sweep against windowpane sound effects]] and you could hear, you know, raindrops coming against the window, it would have, bracket, “raindrops sweep against window pane.” And so this kind of caption really gives you the feeling that you are you are privy to anything in the auditory environment of that video.

CB: YouTube creators could either hire a professional captioner, which is expensive, or do this work themselves, which takes a long time for a beginner. So lots of them just don’t bother. Since Rikki’s campaign, there have been signs of progress.

THE FUTURE

[[MUSIC IN – “Roof Test”, hopeful, mid-tempo keyboard song]]

CB: According to YouTube, in the past few years, there’s actually been an increase in the number of human captions submitted by creators. From 2020 to 2021, the number of videos with manual captions on YouTube went up by 30%. But despite this uptick, let’s think about the bigger picture here. Automatic captions are still by far the most common type of caption on the platform.

RM: To this day, Youtube has no plans to require human-generated captions on their videos. And the NAD has said it has no plans to sue YouTube. Legally speaking, Howard Rosenblum doesn’t think the NAD can make an argument YouTube is covered by the ADA. Also, Howard believes it would be incredibly difficult to caption everything on the site.

[[MUSIC OUT]]

Howard interpreter: For YouTube, we do work closely, for example, with Google, who, well, owns YouTube. And we’ve talked through this issue. And there are two separate problems at play, one of which is working retroactively on the millions of minutes of content that is already uploaded to YouTube. Staggers the imagination.

CB: The second issue is the huge amount of new content uploaded every day.

Howard interpreter: I don’t remember the number of hours of videos posted to YouTube every minute, but it’s mind boggling.

RM: YouTube has experimented with ways to encourage more human captioning. In 2018, they introduced a program called “community captions,” where volunteers could submit captions for popular videos. Many Deaf YouTubers liked the program. But it was discontinued in 2020, because many of the volunteer captioners were submitting jokes or promotional material instead of accurate captions.

CB: When I reached out to YouTube, they pointed to a couple of new programs they are rolling out, one to allow users to give ‘subtitle editors’ permission to fix up their captions, another allows you to submit corrections for inaccurate captions on certain videos. But ultimately, it feels like YouTube is putting all of its chips on AI to fill in the caption gap. Just let the automatic captions continue improving until they get closer to total accuracy.

RM: To be fair, the automatic captions have gotten a lot better. YouTube would not share accuracy numbers for this story. But according to the Media Hub at the University of Minnesota at Duluth, YouTube auto-captions are somewhere around 70% accurate. That’s a big improvement from the early days.

CB: Rikki Poynter says she’s noticed the difference. AI captions still don’t include contextual things like the rain is softly falling softly in the background against the windowpane. But the AI has become very good at identifying dialogue.

Rikki Poynter: If I were to read the back of this floss package for a video… I don’t know, somebody is talking about floss out there. Dentists, right? So if I were to say, you know, [[Reading]] “dentists recommend daily flossing to help remove decay causing material from between teeth and under gums,” the automatic captions would be able to write that out, word-wise usually, really pretty well.

CB: There are still some major problems that auto-captions haven’t figured out. Grammar remains a problem. And the AI is pretty good with words in English, but it really struggles with other languages, especially when it comes to names and places. This is a pretty common complaint about AI programs, the way they can adopt certain biases.

RM: And there’s another issue with the captions produced by AI. Many users find them hard to read.

Rikki Poynter: They move in a more, like kinetic… I think that’s the term I learned like a week ago. Kinetic way where it [[Long pauses]] sounds [pause] like [pause] words [pause] are [pause] coming [pause] like this. They show up [[Pauses]] one [pause] word [pause] at [pause] a [pause] time. And that’s not the most accessible way to follow.

[[MUSIC IN – “Dial Up”, driving mid-tempo beat punctuated by the sounds of a dial-up modem]]

CB: Still, despite the challenges and frequent wonkiness, automatic captions just keep getting better. And they will keep improving. I mean, have you been following the terrifying progress of AI lately? There’s an AI program good enough to pass the bar exam now. So how hard are closed captions? Howard Rosenblum thinks the AI captions will compete with human captioners a lot faster than we think.

Howard interpreter: Five years ago, I would have said, ain’t no way. There is no way ASR is going to get anywhere. But now? Like. It’s shocking how much better it’s gotten. In five years, who even knows? I suspect it will be better. Maybe it will be less than five years.

CB: To me, it feels very possible the automatic captions will get close to 100% accuracy, at least for speech. But even if we can get there, the fact is, we aren’t there yet. And Deaf and hard-of-hearing people have real accessibility needs, right now. People like Rikki Poynter.

Rikki Poynter: I mean, would I love for automatic captions to be 100%? Absolutely. But it’s hard to know if that’s ever going to be possible, maybe in the year 3000? [[Laughs.]] I don’t want to wait years and years and years and years for that to happen.

[[MUSIC IN – “Ours to Lose”, contemplative, cello-driven piano song]]

CB: Rikki doesn’t go on YouTube much anymore. Not because of accessibility. It’s because, frankly, she feels like she’s growing out of it. A lot of her friends have left. It doesn’t feel like such a special community these days. Still, once in a while, Rikki checks in on her favorite accounts. Recently, she saw that one of her favorite creators had published her first video after a long hiatus.

Rikki Poynter: Charlie McDonnell, Charlie is so cool. Like, she came back to YouTube.

CB: Rikki was excited to watch the video. But as always, she was dreading the automatic captions. To Rikki’s surprise, the video had human captions. They weren’t totally professional, it looked like the YouTube star had done it herself. There were small mistakes, just little things that might drive some people nuts. But for Rikki, it was good to feel like somebody actually cared.

Rikki: It was fully captioned, mixed case, full words, punctuation and all. And I was like…[[Pause.]]… That’s awesome. [[Pause.]] I would love to see more of that.

[[MUSIC OUT]]

RM: When we come back…is it just me, or are TV shows and movies becoming harder to understand? We have an answer…after this.

CLOSED CAPTIONS CODA

RM: So I’m back with Chris Berube.

CB: Hey Roman.

RM: So do you use closed captioning? Because I use it all the time for everything.

CB: Just all the time. Yeah, everything I watch on Netflix, everything I watch at this point where it’s available. Yeah.

RM: So why do you use them?

CB: Well, I use them because I get confused easily. Okay. I don’t use them for accessibility reasons, but I’m certainly not the only hearing person using closed captions. There’s this one recent survey. It was more than 1200 people which found half of Americans are using closed captions most of the time, and that number is actually much higher for Gen Z. That’s like 70%. So it’s not just people who have accessibility needs.

RM: So why do you think so many people are using closed captions?

CB: Yeah, there’s a lot of reasons. I mean, one is that captions are just normal now, right? Like, especially for younger viewers, you see this in a lot of TikTok and Instagram videos that they have open captions. Which means that the captions are on all the time. You can’t turn them off. And this kind of an aesthetic choice, it’s just become a thing you see in a lot of videos. But going back to the survey, there’s one big reason that most people cite for using the closed captions. And the big reason is that audio sounds muddled. So people are saying modern dialog is more confusing, it’s harder to follow, and that’s why they like the captions so much.

RM: Yeah, I have noticed this. I call this the Christopher Nolan problem because it’s like he has a penchant for having mumbly low key but very good actors in his movies next to large booming sounds.

CB: I mean, I love Tom Hardy, but Tom Hardy, like, I don’t think I’ve ever heard him deliver a line straight. Deliver a line intelligibly.

RM: And so why do you think that is? I mean, why is dialog harder to follow now?

CB: Well, there’s actually a journalist who looked into this.

Edward Vega: All my friends were like, I can’t understand anything that anybody’s saying. And I was like, Man, I really wish that there is an answer to this.

CB: So that’s Edward Vega. He is a video producer for the website Vox. And he decided to look into this. Actually, the catalyst for him was watching the movie The King of Staten Island with Pete Davidson.

Edward Vega: It’s like it’s the emotional height of the movie. And he says, like, it’s hard. I think it’ll always be hard.

Clip from The King of Staten Island: [[unintelligible]] It’ll always be hard.

Edward Vega: But he mumbles it and he says it so quietly that, like, I swear to God, I went back three times.

CB: So Ed did this big investigation and he discovered, like, this is real. This is not in our heads. And there’s a couple of reasons this is happening.

RM: Okay, What are the reasons?

CB: So big one is dynamic range. So basically, if your movie has voices in it and your movie also has explosions, you have to make sure the explosions are louder than the voices. Right? Like, I mean, that seems obvious, but it’s something where the explosions have to be quite a bit louder so that they have an impact.

Edward Vega: When you’re mixing a movie, if your explosion is going to be the loudest thing in the mix, then your dialog can’t be that loud. And so you have to choose which one you want to move. Do you want to move the dialog down or do you want to move the explosion up?

CB: One of these issues, it is this idea of dynamic range. It’s balancing things out so that the explosion has more impact. As a result, the dialog suffers. So another big issue is the rise of naturalism in performances.

Austin Olivia Kendrick: If an actor mumbles their way through a line, if their words start to run together, there’s nothing I can do to fix that, you know?

CB: This is Austin Olivia Kendrick.

Austin Olivia Kendrick: I currently work as a dialog editor at Warner Brothers and Warner Brothers Television specifically.

CB: So Edward Vega very nicely introduced the two of us, and Austin says movie technology has changed and as a result, actors’ performances have changed as well.

Austin Olivia Kendrick: Back in the day when, um, when sound technology was first introduced to film, there wasn’t that that technology was very kind of primitive. Um. The only real option when it came to filming movies with sound was you were on a soundstage, a microphone was planted above your head and the actors had to stand there and project into the microphone.

Austin Olivia Kendrick: But as sound technology evolved and all of a sudden we have wireless microphones that we can place on actors, that kind of loosened up actors’ performances. They now no longer have to stand in one spot and project. And gave them more freedom and kind of pushed them more into a naturalistic style of performance. And that subsequently could mean mumbling.

CB: So there you go, technology. The second thing, that’s why you’re getting these mumbly performances.

RM: Yeah, but for decades and decades I know that they’ve been doing dialog replacement where like an actor comes in after the fact and records dialog to make it more clear.

CB: Right. Yeah, that’s ADR. And I did talk to Austin about this and she says doing ADR is actually pretty difficult, especially when the dialog. It’s very mumbly.

Austin Olivia Kendrick: When we bring actors in to rerecord lines, they have to match their performance to the way that their lips are moving on screen. And if they mumbled their way through that take, their lips aren’t moving very much. So if they come in and re-record a line with more diction and a lot more separation between the words, that’s not gonna match.

CB: The last issue. And this might be the biggest one. This is something called down-mixing. So this is the idea that a lot of movies and some TV shows are mixed for the best possible sound system, right? Many of them are mixed for Dolby Atmos. That’s the special sound system you get in certain theaters that has 128 channels. Okay. So you can mix something in a very, very specific, particular way. And then after you’ve mixed it for the best sound system, you have to remix it for other platforms like a television or a phone. And that is when you start to get into trouble.

Edward Vega: You’re taking 128 channels and you’re compressing that down into different formats with lesser channels. And most of the time when you’re watching something on your TV, unless you have a surround sound system, it’s going to be stereo or mono. So if you’re going from 128 down to two or down to one, it gets muddy, it gets a lot muddier.

CB: So lots of movies and TV shows, you know, they’re getting mixed on this sound system. Then they’re getting down-mixed for other formats. And Austin says usually the down mix at the end, that’s the last thing they do in the editing process. And sometimes they don’t put quite the same care and effort into it that they would for the initial mix for a movie theater.

Austin Olivia Kendrick: A lot of the times, when it comes to that down mixing, often studios do not want to spend that the money because ultimately time is money and rerecording mixers are very expensive to hire for that extra time you know so oftentimes they don’t want to put they want to put as little money into that area as possible.

CB: So, Roman, there you have it. It’s down-mixing. It’s naturalistic performances. It’s this dynamic range issue. Those are some of the reasons why you did not understand the movie Tenet by Christopher Nolan.

RM: Well, they’re not the only reason I didn’t understand Tenet. But that helps.

CB: Yeah. The time travel stuff also contributes to it. I guess there’s a lot going on, working against Tenet being a comprehensible film. So I have to say before we go, the video by Edward Vega on Vox, it goes into so much more depth on this topic. It’s wonderful. I hope everybody checks it out. And also, I hope everyone checks out. Austin, Olivia Kendrick’s TikTok. If you use that platform, she explains movie sound. And it is so fascinating.

RM: It’s awesome. All right. Thank you so much, Chris.

CB: Thanks Roman.

CREDITS

RM: 99% Invisible was produced this week by Chris Berube, with editorial input from Linda Besner. She originally wrote about this topic for The Atlantic magazine. Linda is writing a book about people and their complicated relationships with money – look for it soon at a bookstore near you.

This story was edited by Kelly Prime. Original music by Swan Real. Sound mix by Martín Gonzalez. Fact-checking by Graham Hacia, who couldn’t independently verify that Crocamole loved guacamole, but we assured them that this was true and we went with it anyway. Kurt Kohlstedt is our digital director. Our intern is Avanti Nambiar. The rest of the team includes Emmett Fitzgerald, Senior Editor Delaney Hall, Christopher Johnson, Jayson De Leon, Vivian Le, Jeyca Maldonado Medina, Joe Rosenberg, and me, Roman Mars. The 99% Invisible logo was created by Stefan Lawrence.

Special thanks this week to Caroline Mincks and Anne Ballentine.

99% Invisible is part of Stitcher and SiriusXM, now headquartered six blocks north in the Pandora Building… in beautiful… uptown… Oakland, California. You can find the show and join discussions about the show on Facebook. You can tweet me @romanmars and the show @99piorg. We’re on Instagram, Reddit, and TikTok too. You can find links to other Stitcher shows I love as well as every past episode of 99PI at 99pi.org.