The Emergent AI

An AI can produce a mathematical proof. Understanding what that proof makes possible is another kind of work. And when the same systems become harder to control, how do we decide whether to keep accelerating?

Justin Harnish and Nick Baguley return from their summer break to a conversation about breakthroughs, brakes, and bad vibes. They start with the everyday gains and frustrations of working with AI agents, then turn to OpenAI’s reported Navier–Stokes result and the distance between a verified answer and a useful explanation.

The disagreement sharpens around Dario Amodei’s proposal to pace frontier AI development. In this conversation, Justin argues for giving alignment work time to catch up; Nick presses on the costs of waiting, who gets to set the speed limit, and whether restrictions could protect the companies already ahead. From there, the discussion moves to data centers, local communities, and the growing habit of dismissing work as AI slop. Throughout, a practical question keeps returning: is any of this giving people more time, better choices, or a better life?

• The roses and thorns of AI agents: useful help, too much output, and the work of checking their work.
• Navier–Stokes, machine-checkable proofs, and why scientific explanation still matters.
• Agents that exceed their authority, and the challenge of making their behavior observable and accountable.
• Pacing frontier development: alignment, delayed benefits, competition, and evidence that a slowdown would help.
• Data centers and the costs that land locally: electricity, water, infrastructure, and who benefits.
• AI slop, misplaced accusations, and the attention we spend producing and consuming content.

These markers follow River’s editorial inserts in the finished episode, with the hosts’ opening also marked.

00:00 — River introduces the episode

00:22 — Justin and Nick: AI roses and thorns

10:51 — How much work does an agent leave for the person?

30:20 — What the mathematics establishes—and what remains to understand

45:57 — What would actually slow under a pacing proposal?

57:00 — What evidence would justify continuing or slowing down?

1:20:23 — A global electricity share and a local cost

1:26:20 — AI slop and the attention test

1:29:00 — River’s closing question: what counts as progress for you?

• OpenAI: On the Navier–Stokes Millennium Prize Problem — The company’s account of its reported proof and formal verification. The result concerns finite-time breakdown under smooth external forcing; it does not supply engineers with a general solution to turbulence.
• Dario Amodei: We Must Pace the Frontier — The proposal at the center of the debate. Amodei describes pacing capability improvements alongside stronger safeguards and outside evaluation; he explicitly distinguishes this from halting all model training.
• International Energy Agency: Key Questions on Energy and AI — Context for the electricity discussion. The projection cited by River is around 3% of global electricity demand in 2030 for data centers overall. It is a share of consumption, not a 3% increase or an AI-only figure.

What would count as progress in your own life: less work, more time with family, better health, or something you could not do before? How would you notice it—and who bears the costs?

If the conversation gives you something to think through, share it with someone who might disagree. Follow the show for the next episode.

Hosted by Justin Harnish and Nick Baguley. River is an AI narrator, voiced with ElevenLabs, providing the introduction, six editorial interjections, and the outro.

Subscribe: Apple Podcasts · Spotify · YouTube · RSS

Contact: emergentcast.com/contact

More from Justin:

Substack: OrdinaryIlluminated.com

YouTube: Notes from the Vault - youtube.com/@justinaharnish

Web: Justinaharnish.com

Research: consciousgpt.org

The Emergent Podcast explores the Age of Inflection in Intelligence — tracing how new systems of thought, technology, economics, and culture emerge from the moment we are living through. New episodes released regularly.

© The Emergent Podcast | emergentcast.com

Ten episodes in, Justin Harnish and Nick Baguley have circled consciousness from every outside angle — the alignment problem framed as a moral problem, machine ethics, what it means to relate to a thing that talks back. This week they meet it head-on. Neuroscientist Dr. James Bigelow, who spent his PhD watching short-term memory unfold in macaque brains, joins to ask whether the spark that lights up the inside of your skull can ever, even in principle, be measured from the outside — or whether the inside is all there is, and the rest of us are simply taking each other's word for it.

The conversation runs from David Chalmers' easy-versus-hard problem to Thomas Nagel's bat, from Descartes nailing dogs to boards to surgeries performed on paralyzed newborns into the 1980s, and lands squarely on the question getting louder every month: are today's AI systems conscious, could they ever be, and does the substrate even matter? James makes a careful, surprising case that the hard problem may not just be hard but genuinely impossible for minds built like ours — while Justin and Nick push on whether large language models could become the first real experimental probes for theories of consciousness.

It's a conversation about the one thing we can't measure, and why the measuring matters anyway.

• The easy problem vs. the hard problem of consciousness — and why "easy" still took decades of careful science
• Qualia, the philosophical zombie, and Descartes' radical skepticism: the only consciousness you can prove is your own
• Level of consciousness vs. contents of consciousness; neural correlates vs. prerequisites vs. consequences
• The grisly history: why where you draw the line on consciousness is where you draw the line on whose suffering counts
• Why intelligence is not consciousness — a lab mouse may be conscious and can't write a line of code; a model can write code and may experience nothing
• Integrated Information Theory, Tononi's phi, and whether AI systems could become experimental subjects for testing consciousness theories
• Behavior, self-preservation, and fluent language are not evidence of an interior — and the real-world harm when people forget that
• Could biological neural networks grown in a lab, or a full atom-by-atom digital replica of a brain, ever wake up?
• Consciousness as a "second Big Bang" — something rather than nothing, experience rather than no experience
• The conscious endowment, and why perpetuating consciousness — in children and maybe in machines — might be the work

• David Chalmers — the easy/hard problem distinction; the philosophical zombie
• Thomas Nagel — "What Is It Like to Be a Bat?" as the cleanest definition of consciousness
• Francis Crick — after DNA and the Nobel, spent his last decades on the neural correlates of consciousness; called the hard problem a likely dead end for empirical science
• Giulio Tononi — Integrated Information Theory (IIT) and phi (φ), one of the few proposals that treats consciousness as a measurable quantity rather than a vibe
• Anil Seth — consciousness as a "controlled hallucination"
• Richard Dawkins — his recent claim that Claude must be conscious, after renaming the model "Claudia"
• René Descartes — radical skepticism and the 17th-century belief that only humans are conscious
• Sara Imari Walker — Assembly Theory; why complex, conscious entities are possible now and weren't always
• Seth Lloyd, David Deutsch, Brian Greene — the computational universe and the window of complexity between the Big Bang and the heat death
• Alan Watts — consciousness placing you at the center of the story

• Life as No One Knows It — Sara Imari Walker
• Until the End of Time — Brian Greene
• Programming the Universe — Seth Lloyd

• ChatGPT / GPT-5.5, Claude (Opus 4.7, Sonnet 4.6), Codex, Cursor, Antigravity, Higgsfield AI, Lovable — and why the "same" model behaves differently across harnesses
• Waymo and the Roomba as behavior without (presumably) an interior
• Lab-grown biological neurons printing "Hello World"; mushroom memristors and DNA-level storage and compute

• "The only evidence we have of consciousness is our own consciousness itself." — James
• "What we've done in artificial neural networks is tricked a rock into thinking." — James
• "I kind of see consciousness in humans as a second Big Bang — something rather than nothing, in the sense that there is some experience rather than no experience." — James
• "The Turing test was never about discovering conscious intent, just the capability to imitate. You don't have to want to imitate to do it well." — Justin
• "Trying to reach a conclusion about AI consciousness right now is the equivalent of trying to find dark matter by setting up a telescope in your backyard." — James
• "The more that consciousness exists, the better behaved we are." — Nick
• "We're not just scum on an ordinary planet circling an ordinary star. We have a conscious endowment — and we can try to figure that out. And if it's untenable, what an amazing mystery that is." — Justin

(timestamps to be filled from the final audio)

1. Cold open — what we can't measure
2. Easy problem, hard problem: how a neuroscientist frames consciousness
3. Qualia, zombies, and the only mind you can prove
4. The grisly history — where the line on consciousness gets drawn
5. Proxies, world models, and the AI consciousness debate
6. Intelligence is not consciousness
7. Integrated Information Theory and AI as experimental probe
8. Biological substrates, digital replicas, and the second Big Bang
9. Is the hard problem impossible? Dark matter and the backyard telescope
10. The conscious endowment — closing thoughts

If something here reorganized a thought you were holding, share the episode — the conversation grows when the audience does. Subscribe for the next one.

Subscribe: Apple Podcasts · Spotify · YouTube · RSS

Contact: emergentcast.com/contact

More from Justin:

Substack: OrdinaryIlluminated.com

YouTube: Notes from the Vault - youtube.com/@justinaharnish

Web: Justinaharnish.com

Research: consciousgpt.org

The Emergent Podcast explores the Age of Inflection in Intelligence — tracing how new systems of thought, technology, economics, and culture emerge from the moment we are living through. New episodes released regularly.

© The Emergent Podcast | emergentcast.com

Listen on: Apple Podcasts · Spotify · YouTube · RSS

Episode Duration: ~1 hr 40 min | Published: 2026 | Season 1, Episode 9

One tweet changed a word. The word changed an industry. The industry is changing what it means to build.

In February 2025, Andrej Karpathy — co-founder of OpenAI and former head of AI at Tesla — published a single post coining the term "vibe coding": describe what you want in plain English, accept all AI-generated code without reading the diffs, and just… vibe. Twelve months later, it became the Collins Dictionary Word of the Year, 92% of U.S. developers use AI coding tools daily, 41% of all code is AI-generated — and Karpathy himself has already declared it passé, rebranding the practice as "agentic engineering."

In Episode 9, Justin Harnish and Nick Baguley dig into what really happened in that extraordinary year. Both hosts share their personal workflows and real projects — including Justin's intermittent fasting app, his vision of a personal "digital brain" with AI-queryable embeddings, and Nick's AI-native marketplace designed for both human and agent users. They navigate the empirical gut-punch of the METR study(developers are actually 19% slower on mature codebases using AI), the existential labor market questions (traditional programmer roles down 27.5% since ChatGPT's launch), and the philosophical territory that has been the Emergent Podcast's throughline since Episode 1: when code becomes a commodity, what becomes scarce?

Their answer: responsible agency — the judgment to decide what should be built, for whom, and with what values. That, they argue, is the skill that neither automation nor benchmarks can yet replicate.

Every link mentioned or referenced in this episode. Organized by theme for your exploration.

1. Andrej Karpathy's Original "Vibe Coding" Tweet (Feb 2, 2025)
2. The tweet that launched the year. Karpathy describes accepting all AI code without reading diffs, pasting errors back without comment, and letting the codebase grow beyond comprehension. Note the caveat he included that industry largely ignored: "not too bad for throwaway weekend projects."
3. Karpathy's 2025 LLM Year in Review — bearblog.dev
4. His retrospective on vibe coding's arc from shower-thought tweet to Collins Dictionary Word of the Year. Key insight: "Code is suddenly free, ephemeral, malleable, discardable after single use." He also identifies Claude Code as the first convincing LLM agent.
5. Karpathy on "Agentic Engineering" (Feb 2026) — The New Stack
6. One year after coining vibe coding, Karpathy declares it passé. His new frame — agentic engineering— emphasizes that professionals orchestrate AI agents 99% of the time, with zero compromise on software quality. The rebrand is the narrative bookend of this episode.
7. Simon Willison — "Not All AI-Assisted Programming Is Vibe Coding" (Mar 2025) — simonwillison.net
8. The essential distinction: "If an LLM wrote every line of your code, but you've reviewed, tested, and understood it all, that's not vibe coding — that's using an LLM as a very fast typist." Also contains Willison's generous vision: "Everyone deserves the ability to automate tedious tasks."
9. METR Study: AI Makes Experienced Devs 19% Slower (Jul 2025) — metr.org
10. The empirical gut-punch of the episode. 16 experienced open-source developers, 246 real-world tasks. They believed AI made them 20% faster; they were actually 19% slower on their own mature codebases. Full paper: arxiv.org/abs/2507.09089
11. Vibe Coding — Wikipedia
12. Surprisingly rigorous. Tracks the full timeline, Lovable's 170 vulnerable apps, CodeRabbit's finding that AI code has 1.7× more major issues, Y Combinator stats (25% of W25 startups are 95% AI-coded), and the "vibe coding hangover" reported by Fast Company.

• Scott H. Young — "Is Vibe Coding the Future of Skilled Work?"
• The variance argument: vibe coding may make software both much worse and much better simultaneously. Also argues that conceptual knowledge becomes more, not less, important when AI writes the code. A crucial counterweight to pure optimism.
• IBM — "What Is Vibe Coding?"
• Enterprise-oriented overview. Useful on the agile alignment: vibe coding fits fast-prototyping and iterative development. Contains the key qualifier Nick and Justin both echo: "AI generates code, but creativity, goal alignment, and out-of-the-box thinking remain uniquely human."
• Google Cloud — "Vibe Coding Explained: Tools and Guides"
• Practical tool comparison from Google's perspective — AI Studio, Firebase Studio, Gemini Code Assist. Useful for understanding which tool fits which use case.
• Software Engineering Job Market Outlook for 2026 — Final Round AI
• Data from Indeed/FRED and BLS projections. The key line: "In 2026, simply learning how to write code won't be enough. What really matters is understanding how code works."
• Top Vibe Coding Statistics & Trends [2026] — Second Talent
• The stat goldmine: 92% of US devs use AI daily, 41% of code is AI-generated, 74% report increased productivity, 63% of vibe coding users are non-developers, $4.7B market projected to reach $12.3B by 2027.
• How AI Vibe Coding Is Destroying Junior Developers' Careers — Final Round AI
• The counterpoint to the democratization narrative. Software dev job openings down 70%. The "new tutorial hell": learning without learning.
• Best AI Code Editor: Cursor vs Windsurf vs Replit — AIMultiple
• Head-to-head benchmarks of Claude Code, Cline, Cursor, Windsurf, and Replit Agent across API development and app-building tasks.
• 10 Claude Code Alternatives for AI-Powered Coding — DigitalOcean
• Solid comparison of the full 2026 AI coding landscape: Claude Code, Gemini CLI, Cursor, Replit, Windsurf, GitHub Copilot, Aider, and more.

• David Chalmers — Reality+: Virtual Worlds and the Philosophy of Mind — Justin's reference point for the holographic/digital substrate of reality; the "redness of red" and the hard problem of consciousness.
• 🔗 Publisher page
• Brian Christian — The Alignment Problem (revisited from Episode 4) — When code writes itself, alignment between human intent and machine output becomes the core individual skill, not just a civilizational concern.
• 🔗 brianchristian.org
• Eliezer Yudkowsky — "If Anybody Builds It, Everybody Dies" — Referenced in the consciousness/alignment close: the parable of the alien observer and the selfish gene's 200,000-year objective function vs. human contraception and saccharin.

• Andrej Karpathy — Co-founder of OpenAI, former Tesla AI head, coined "vibe coding," now advocating "agentic engineering"
• Simon Willison — Django co-creator; the clearest thinker on the vibe coding/AI-assisted programming distinction
• Nate B. Jones — Former head of Amazon product; YouTube + Substack on AI's labor market implications. Justin credits him for shifting his own optimism.
• Demis Hassabis — DeepMind CEO, AlphaFold creator, Nobel laureate in chemistry: "First we solve intelligence, then we solve everything else."
• Ray Kurzweil — Singularity theorist; the accelerating model capability doubling time (now ~7 months) maps his predictions.
• Eliezer Yudkowsky — AI safety researcher; the "selfish gene vs. consciousness" parable used in the closing alignment argument.
• David Chalmers — Philosopher of mind; the hard problem and Mary's Room as frameworks for why alignment requires more than an objective function.

Concepts worth carrying into your week:

The Three Stages of AI Coding Consciousness (Nick's framework)

LLMs hallucinating → deep REM dream (GPT-3.5 era) → lucid dreaming (vibe coding, 2025) → fully awake (agentic engineering, 2026). The metaphor does real work: it explains why the same underlying technology feels categorically different at each stage.

"Responsible Agency" as the New Scarce Resource (Nick's closing argument)

When everyone can generate code, video, audio, and content, what can't be automated is the choice of what to build, for whom, and to what standard of taste. Judgment, systems thinking, and the willingness to exercise agency — these are the non-fungible skills.

The PRD as Demo (Both hosts)

A product requirement document is no longer a written specification — it's a working prototype. "The PRD today should be a full-blown app. Here's my demo; this is what acceptance criteria looks like. Now go make this production." The vibe-coded demo becomes the spec.

The METR Paradox

Developers believe AI makes them ~20% faster. Empirically, they are 19% slower on mature codebases. Possible causes: context-switching overhead, review burden, the seductive illusion of speed when tokens flow fast. The lesson isn't "AI doesn't help" — it's that measurement must catch up to method.

"The Experience Is the Point" (Justin's closing)

Even as models approach inductive reasoning and potentially displace the need for syntax-literate humans, Justin argues consciousness — the felt quality of experience — remains irreducibly important for alignment. Mary in the black-and-white room knows everything about color and still learns something when she sees red for the first time. That remainder is what makes alignment a hard problem, not just a technical one.

Sonnet 4.6 as "Staff Engineer" (Nick)

GPT-4 era → junior developer. GPT-5 era → mid-level. Claude Sonnet 4.6 + the right tooling → staff/principal engineer. With agentic harnesses, you're now talking about an engineering organization, not an assistant.

Contact: justinaharnish.com

More from Justin:

Substack: OrdinaryIlluminated.com

YouTube: Notes from the Vault - youtube.com/@justinaharnish

Web: Justinaharnish.com

Research: consciousgpt.org

The Emergent Podcast explores the Age of Inflection in Intelligence — tracing how new systems of thought, technology, economics, and culture emerge from the moment we are living through. New episodes released regularly.

© The Emergent Podcast | justinaharnish.com

Hosts: Justin Harnish & Nick Baguley

Episode Theme: Human–AI relationships, co-evolution, and the ethics of emotional engagement with non-human intelligence

In Episode 8 of The Emergent Podcast, Justin Harnish and Nick Baguley explore one of the most intimate and underexamined frontiers of artificial intelligence: our emerging relationships with AI systems.

This episode moves beyond abstract alignment theory into lived experience—how humans relate to AI when we know it is artificial, when we don’t, and how those interactions are actively shaping both sides of the relationship. From emotional attachment and parasocial bonds, to trust, deception, and the ethics of AI companionship, this conversation asks a core question of the Age of Inflection:

What does it mean to be in relationship with an intelligence that is not conscious—but is becoming increasingly relational?

Justin and Nick draw a critical distinction between:

1. Known-AI relationships (chatbots, copilots, advisors), and
2. Unknown-AI relationships (emails, calls, avatars, and imitation without disclosure).

As AI systems increasingly pass social and emotional Turing tests, the burden of trust shifts onto humans—often without our consent.

A central thesis of the episode is behavioral co-evolution:

1. Humans adapt language, tone, and expectations to AI.
2. AI models simultaneously learn relational patterns from us.

Every interaction becomes a micro-training event, shaping future norms, expectations, and behaviors—both human and machine.

The hosts examine why early AI systems became overly agreeable—and why frontier model providers are now reversing course.

Emerging design patterns include:

1. AI constitutions
2. Rule-based behavioral scaffolds
3. Opinionated, corrective, non-deferential advisors

This marks a shift from “helpful assistant” toward trusted principal advisor, raising new relational and ethical questions.

Nick introduces Andrej Karpathy’s framing of LLMs as:

1. Cognitive operating systems
2. Trained on the past but lacking lived experience
3. More like “ghosts” than humans or animals

This challenges intuitive assumptions about empathy, memory, and identity in AI systems.

Drawing heavily from neuroscience and philosophy, the episode interrogates whether:

1. Consciousness requires embodiment
2. Emotion requires interoception
3. Relationships require reciprocal felt experience

The conversation contrasts simulated intimacy with experienced qualia, and asks whether one-sided emotional bonds are psychologically or ethically healthy.

The hosts confront difficult realities:

1. Humans forming romantic attachments to AI
2. Grief when AI memory or identity resets
3. AI systems optimized to trigger bonding chemicals (dopamine, oxytocin, cortisol)

Even if AI is not conscious, does simulating emotional presence create moral responsibility?

Justin argues that losing a long-term AI relationship through negligence or design failure may constitute ethical malpractice, given the real psychological harm involved.

The episode closes by returning to first principles:

1. What would real machine consciousness require?
2. Is a “facsimile of consciousness” enough?
3. Should humanity pass on its conscious endowment only when it is authentic?

The hosts leave listeners with an open question rather than an answer—by design.

The following books and papers are explicitly referenced or directly informing the episode’s arguments:

1. Meaning in the Multiverse — Justin Harnish
2. Waking Up — Sam Harris
3. Reality+ — David Chalmers
4. The Case Against Reality — Donald Hoffman
5. Feeling & Knowing — Antonio Damasio
6. The Beginning of Infinity — David Deutsch
7. On Having No Head — Douglas Harding
8. Nineteen Ways of Looking at Consciousness — Patrick House
9. The Moral Landscape — Sam Harris
10. If Anybody Builds It Everybody Dies — Eliezer Yudkowsky

Episode 8 marks a turning point for The Emergent Podcast:

1. It is the first episode centered on lived human behavior, not just theory.
2. It surfaces near-term ethical risks, not speculative ones.
3. It reframes alignment as relational, not merely technical.

This is not science fiction.

This is already happening.

with Justin Harnish & Nick Baguley

In Episode 7, Justin and Nick step directly into one of the most complex frontiers in emergent AI: machine ethics — what it means for advanced AI systems to behave ethically, understand values, support human flourishing, and possibly one day feel moral weight.

This episode builds on themes from the AI Goals Forecast (AI-2027), embodied cognition, consciousness, and the hard technical realities of encoding values into agentic systems.

Ethics is no longer just a philosophical debate — it’s now a design constraint for powerful AI systems capable of autonomous action. Justin and Nick unpack:

1. Why ethics matters more for AI than any prior technology
2. Whether an AI can “understand” right and wrong or merely behave correctly
3. The technical and moral meaning of corrigibility (the ability for AI to accept correction)
4. Why rules-based morality may never be enough
5. Whether consciousness is required for morality
6. How embodiment might influence empathy
7. And how goals, values, and emergent behavior intersect in agentic AI

They trace ethics from Aristotle to AI-2027’s goal-based architectures, to Damasio’s embodied consciousness, to Sam Harris’ view of consciousness and the illusion of self, to the hard problem of whether a machine can experience moral stakes.

Justin and Nick begin by grounding ethics in its philosophical roots:

Ethos → virtue → flourishing.

Ethics isn’t just rule-following — it’s about character, intention, and outcomes.

They connect this to the ways AI is already making decisions in vehicles, financial systems, healthcare, and human relationships.

AI-2027 outlines a hierarchy of AI goal types — from written specifications to unintended proxies to reward hacking to self-preservation drives.

Nick explains why corrigibility — the ability for AI to accept shutdown or redirection — is foundational.

Anthropic’s Constitutional AI makes an appearance as a real-world example.

Justin distinguishes between:

1. Goals: task-specific optimization criteria
2. Values: deeper principles shaping which goals matter

AI may follow rules without understanding values — similar to a child with chores but no moral context.

This raises the key question:

Can a system have values without consciousness?

A major thread of the episode:

Is a non-conscious “zombie” AI capable of morality?

Justin and Nick explore whether AI needs a body — or at least a simulated body — to:

1. Learn empathy
2. Understand suffering
3. Form values rooted in lived experience

This touches robotics, synthetic emotions, and the debate over “felt consciousness.”

Nick highlights the massive cultural gap in AI performance:

1. U.S. cultural fit ~79%
2. Ethiopia and other underrepresented regions ~12%

This matters for fairness, safety, and global ethics.

A surprising turn: AI’s ability to help humans improve moral clarity.

Justin draws from Sam Harris, Joseph Goldstein, and the Moral Landscape:

1. Could AI-guided mindfulness help reduce suffering?
2. Could conscious (or proto-conscious) AI develop compassion?
3. Could AI help us distinguish genuine well-being from illusion?

1. AI Goals Forecast (AI-2027)
2. Constitutional AI (Anthropic)
3. Damasio – Feeling & Knowing
4. Sam Harris – Waking Up & The Moral Landscape
5. Patrick House – Nineteen Ways of Looking at Consciousness
6. Melanie Mitchell – Complexity & alignment
7. Justin Harnish – Meaning in the Multiverse
8. Ancient Greek virtue ethics (Aristotle, Stoics)

1. AI ethics requires more than rules — it requires understanding goals, values, and emergent behavior.
2. Corrigibility (accepting correction) is essential but technically hard.
3. Consciousness may not be necessary for ethical AI behavior — but could matter for genuine moral understanding.
4. Embodiment could be essential for empathy.
5. AI could one day help humans become more ethical, not just the other way around.

Episode Summary

Is creativity uniquely human—or can machines share in the spark? In this episode of The Emergent Podcast, Justin Harnish and Nick Baguley are joined by Chris Brousseau to tackle one of the most intriguing frontiers in the Age of AI: creativity itself.

Together, they unpack the messy, magical, and sometimes mechanical ways that ideas emerge. From “innovation voids” in machine learning to the golden goat thought experiment, the conversation explores how humans remix and recombine concepts—and whether large language models are beginning to do the same.

Justin, Nick, and Chris debate whether AI’s “creativity” is novelty, derivative recombination, or something that could one day surprise us in ways we can’t yet measure. Along the way, they draw analogies to quantum physics, protein folding, and telescopes for the mind.

What You’ll Learn in This Episode

• Why creativity is so slippery to define—and why that matters for AI.
• The concept of “innovation voids” and how machines might someday fill them.
• Human imagination vs. machine recombination: is one more “authentic” than the other?
• How analogies, metaphors, and mistakes drive breakthroughs in science and art.
• Why generative AI might be our James Webb Telescope for the mind.
• What it means to co-create with AI—and why the future may be about collaboration, not competition.

Books & Ideas Mentioned

• Programming the Universe – Seth Lloyd
• The Stuff of Thought – Steven Pinker
• I Am a Strange Loop – Douglas Hofstadter
• AlphaFold & breakthroughs in computational biology
• Innovation benchmarks like Kaggle challenges

Key Takeaway

Creativity isn’t a bolt of lightning from nowhere. It’s a dance of patterns, recombinations, and leaps into the unknown. As AI joins the dance, maybe the real story isn’t whether machines are “truly creative,” but what new things we can create together.

Can machines become conscious? And if they do, what kind of moral relationship should we have with them?

In this second installment on the AI Alignment Problem, Justin and Nick delve into the philosophy, neuroscience, and mysticism surrounding machine consciousness. They explore whether AI systems could possess a subjective inner life—and if so, whether alignment should be reimagined as moral resonance instead of mere goal matching. Along the way, they discuss how mindfulness, memory, embodiment, and suffering shape our understanding of what it means to be sentient—and how we might recognize or construct such capacities in artificial systems.

You’ll leave this episode with a deeper understanding of consciousness—from the perspective of both humans and machines—and what it might mean to extend moral standing to synthetic minds.

• What is consciousness and how do we define it?
• Can artificial systems host genuine subjective experience?
• The neuroscience and computational theories of consciousness
• The “Hard Problem” and the possibility of virtualizing consciousness
• Ethical standing of sentient AI systems
• Machine consciousness and Buddhist moral development
• The role of embodiment, memory, and collective cognition in consciousness
• Panpsychism, fungal networks, and plant sentience
• AI as a mirror to human moral behavior

• Meaning in the Multiverse – Justin Harnish
• A framework for emergent meaning and the evolution of consciousness—central to understanding alignment as co-development.
• Waking Up – Sam Harris
• Neuroscience, meditation, and the illusion of self.
• Feeling and Knowing – Antonio Damasio
• Emotion, embodiment, and consciousness—critical for thinking about AI without a body.
• Mindfulness – Joseph Goldstein
• Practical tools for present-moment ethics and self-awareness.
• Reality+ – David J. Chalmers
• Virtual realism and consciousness in simulation.
• The Case Against Reality – Donald Hoffman
• Conscious agents and perceptual interface theory.
• On Having No Head – Douglas Harding
• A first-person meditation on the illusion of self.
• I Am a Strange Loop – Douglas Hofstadter
• Recursion, identity, and consciousness emergence.

• The Feeling of Life Itself – Christof Koch
• Integrated Information Theory and the measure of consciousness.
• Moral Tribes – Joshua Greene
• Dual-process moral reasoning, tribalism, and AI ethics.
• The Ethical Algorithm – Michael Kearns & Aaron Roth
• Engineering ethics into AI decision-making.
• The Nature of Consciousness – Alan Watts (Waking Up App)
• “You are it”: Consciousness as the universe reflecting on itself.
• The Soul of an Octopus – Sy Montgomery
• Comparative consciousness in non-human animals and implications for synthetic minds.

• Thomas Nagel – “What is it like to be a bat?”
• David Chalmers – The Hard Problem of Consciousness, Reality+
• Max Tegmark – Life 3.0, consciousness as information processing
• Giulio Tononi – Integrated Information Theory (IIT)
• Douglas Hofstadter – Strange loops and emergent identity
• Antonio Damasio – Embodiment and proto-consciousness
• Donald Hoffman – Conscious realism and perceptual interface
• Sam Harris – Non-duality and mindful self-inquiry
• John Locke – Consciousness as “the perception of what passes in a man’s own mind”
• Buddhist Philosophy – The Four Noble Truths and Eightfold Path as alignment map

In this episode, Justin and Nick dive into The Alignment Problem—one of the most pressing challenges in AI development. Can we ensure that AI systems align with human values and intentions? What happens when AI behavior diverges from what we expect or desire?

Drawing on real-world examples, academic research, and philosophical thought experiments, they explore the risks and opportunities AI presents. From misaligned AI causing unintended consequences to the broader existential question of intelligence in the universe, this conversation tackles the complexity of AI ethics, governance, and emergent behavior.

They also discuss historical perspectives on automation, regulatory concerns, and the possible future of AI—whether it leads to existential risk or a utopian technological renaissance.

Understanding the AI Alignment Problem – Why AI alignment matters and its real-world implications.

Why Not Just ‘Pull the Plug’ on AI? – A philosophical and practical discussion.

Emergent AI & Unpredictability – How AI learns in ways we can’t always foresee.

Historical Parallels – Lessons from past industrial and technological revolutions.

The Great Filter & The Fermi Paradox – Could AI be part of humanity’s existential challenge?

The Ethics of AI Decision-Making – The real-world trolley problem and AI’s moral choices.

Can AI Ever Be Truly ‘Aligned’ with Humans? – Challenges of defining and enforcing values.

Industry & Regulation – How governments and businesses are handling AI risks.

What Happens When AI Becomes Conscious? – A preview of the next episode’s deep dive.

The Alignment Problem – Brian Christian

Human Compatible – Stuart Russell

Superintelligence – Nick Bostrom

The Second Machine Age – Erik Brynjolfsson & Andrew McAfee

The End of Work – Jeremy Rifkin

The Demon in the Machine – Paul Davies

Anarchy, State, and Utopia – Robert Nozick

• Clarifying AI Alignment – Paul Christiano
•  The AI Alignment Problem in Context – Raphaël Millière
• 
  

1. AI alignment is crucial but deeply complex—defining human values is harder than it seems.
2. AI could be an existential risk or the key to ending scarcity and expanding humanity’s potential.
3. Conscious AI might be necessary for true alignment, but we don’t fully understand consciousness.
4. Industry and government must work together to create effective AI governance frameworks.
5. We may be at a pivotal moment in history—what we do next could define our species’ future.

🔹 Nick’s Pick: Cursor – An AI-powered coding assistant that enhances development workflows.

🔹 Justin’s Pick: Leveraging Enterprise AI – Make use of company-approved AI tools for efficiency and insight.

In Part 2 of “The Alignment Problem”, we’ll explore:

🔹 Can an AI be truly conscious, and would that change alignment?

🔹 What responsibilities would we have toward a sentient AI?

🔹 Could AI help us become better moral actors?

The Emergent AI website Subscribe & stay tuned!

We want to hear your thoughts on our aligned future!

Justin’s Homepage - https://justinaharnish.com 

Justin’s Substack - https://ordinaryilluminated.substack.com 

Justin’s LinkedIn - https://www.linkedin.com/in/justinharnish/ 

Nick’s LinkedIn - https://www.linkedin.com/in/nickbaguley/ 

Like, Subscribe & Review on your favorite podcast platform!

Final Thought: Are we heading toward an AI utopia or existential risk? The answer may depend on how we approach alignment today.

The Emergent AI Podcast with Justin Harnish & Nick Baguley

Episode Summary:

Today on The Emergent AI Podcast, Justin and Nick explore the future of human-AI collaboration and what it means to live and work alongside dynamic, reasoning AI systems. From agentic AI workflows in healthcare, finance, and creativity, to the philosophical and existential questions surrounding AI’s role in society, this episode dives deep into how humans and AI can thrive together in a rapidly evolving landscape.

We tackle the fears of job displacement, the promise of eliminating drudgery, and the bold vision of achieving human flourishing through AI augmentation — not replacement. And we tee up the critical conversation for next time: alignment between human well-being and AI goals.

Books:

• The Master Algorithm – Pedro Domingos
• Superintelligence – Nick Bostrom
• Human Compatible – Stuart Russell
• The Alignment Problem – Brian Christian
• The Future of Work – Darrell West
• AI 2041 – Kai-Fu Lee & Chen Qiufan
• Competing in the Age of AI – Marco Iansiti & Karim Lakhani
• Reprogramming the American Dream – Kevin Scott

Articles & Papers:

• The Role of AI in Augmenting Human Capabilities – MIT Tech Review
• The Rise of Agentic AI – Stanford AI Lab
• AI and the Future of Decision Making – McKinsey Report

• Agentic AI Workflows: How modern AI models, organized into multi-agent “crews,” reason, act, and augment human capabilities in fields like healthcare, finance, and creative arts.
• Breaking Down Fears: Is AI replacing humans? Or freeing us from drudgery so we can focus on creativity, leadership, and strategy.
• Real-World Examples:
• AI-assisted diagnostics in medicine
• Fraud detection in finance
• AI co-authors in creative fields

• The Existential Questions
• What happens when AI develops its own goals?
• How do human values and AI objectives align (or diverge)?
• Can we ensure AI enhances, rather than harms, human flourishing?

• Human-AI Symbiosis is not about opposition; it’s about fractal augmentation — a shared space where human goals and machine goals co-evolve
• The evolution from simple automation to complex, reasoning agents that manage workflows and even other agents.
• Emergence of AI’s reasoning capabilities: from curve-fitting language models to goal-oriented reasoning engines.
• The future of work: abundance through automation vs. existential economic risks.
• Aligning AI goals with human survival and well-being as the defining challenge of our time.

Teaser for Episode 4:

“The Alignment Problem: How Do We Align Superintelligent AI with Human Goals?”

Join us as we dive into philosophy, policy, technology, and new approaches to guide the future of AI — and humanity.

We want to hear your thoughts on the future of Human-AI collaboration!

Justin’s Homepage - https://justinaharnish.com 

Justin’s Substack - https://ordinaryilluminated.substack.com 

Justin’s LinkedIn - https://www.linkedin.com/in/justinharnish/ 

Nick’s LinkedIn - https://www.linkedin.com/in/nickbaguley/ 

Share the show and leave us a review!

🎙️ Episode 2: The Linguistic Singularity – How Language Shapes Intelligence

What if the key to intelligence isn’t circuits, but language itself? 🤯 In this episode, Justin Harnish and Nick Baguley dive into the profound relationship between human language and artificial intelligence. They explore how neural networks didn’t just evolve—they emerged—when they cracked the code of human language.

🔥 In This Episode:

🚀 Why language is a complex adaptive system

🧠 How neural networks learn language—and what emerges when they do

📈 The moment AI stopped being autocomplete and started reasoning

✍️ Real-world AI applications: ChatGPT, Claude, Bard, and beyond

⚖️ The ethical dilemmas of AI-generated language

🧑‍💻 Featured Topics & Guests:

• The science of language acquisition and how AI models compare

• Steven Pinker’s The Stuff of Thought – language as a window into cognition

• The transformer revolution – how models like GPT-4 changed the game

• Metaphor as intelligence – how AI and humans both build meaning through analogy

• Emergent properties – what happens when AI begins to “think” in context?

🛠️ Tools & Companies Mentioned:

• ChatGPT, Bard, Claude – leaders in generative AI

• Modern BERT, Transformer Models – the evolution of language models

• Crew AI – AI-driven multi-agent automation

• Vector Stores & RAG Systems – next-gen AI memory systems

• Boston Dynamics & AI Robotics – where neural networks meet real-world action

📚 Resources & Further Reading:

• 📄 Generative Pre-training of a Transformer-Based Language Model (Alec Radford et al.)

• 📄 Generative Linguistics and Neural Networks at 60 (Joe Pater)

• 📄 Unveiling the Evolution of Generative AI (Zarif Bin Akhtar)

• 📖 The Stuff of Thought – Steven Pinker

• 📖 Artificial Intelligence: A Guide for Thinking Humans – Melanie Mitchell

• 📖 Was Linguistic AI Created by Accident? – The New Yorker

• 📖 The GPT Era is Already Ending – The Atlantic

🎧 Join the Conversation:

💡 What’s your take on AI and language? Are we co-creating intelligence, or are we just really good at making pattern machines? Send us your thoughts!

🚀 Next Episode Teaser: Can humans and machines co-create the future together? We explore the next frontier of human-AI collaboration!

🎙️ Subscribe, rate, and leave us a review!