Machine Learning Street Talk (MLST)

Will Williams is CTO of Speechmatics in Cambridge. In this sponsored episode - he shares deep technical insights into modern speech recognition technology and system architecture. The episode covers several key technical areas:

* Speechmatics' hybrid approach to ASR, which focusses on unsupervised learning methods, achieving comparable results with 100x less data than fully supervised approaches. Williams explains why this is more efficient and generalizable than end-to-end models like Whisper.

* Their production architecture implementing multiple operating points for different latency-accuracy trade-offs, with careful latency padding (up to 1.8 seconds) to ensure consistent user experience. The system uses lattice-based decoding with language model integration for improved accuracy.

* The challenges and solutions in real-time ASR, including their approach to diarization (speaker identification), handling cross-talk, and implicit source separation. Williams explains why these problems remain difficult even with modern deep learning approaches.

* Their testing and deployment infrastructure, including the use of mirrored environments for catching edge cases in production, and their strategy of maintaining global models rather than allowing customer-specific fine-tuning.

* Technical evolution in ASR, from early days of custom CUDA kernels and manual memory management to modern frameworks, with Williams offering interesting critiques of current PyTorch memory management approaches and arguing for more efficient direct memory allocation in production systems.

Get coding with their API! This is their URL:

https://www.speechmatics.com/

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

MLST is sponsored by Tufa Labs:

Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.

Interested? Apply for an ML research position: [email protected]

TOC

1. ASR Core Technology & Real-time Architecture

[00:00:00] 1.1 ASR and Diarization Fundamentals

[00:05:25] 1.2 Real-time Conversational AI Architecture

[00:09:21] 1.3 Neural Network Streaming Implementation

[00:12:49] 1.4 Multi-modal System Integration

2. Production System Optimization

[00:29:38] 2.1 Production Deployment and Testing Infrastructure

[00:35:40] 2.2 Model Architecture and Deployment Strategy

[00:37:12] 2.3 Latency-Accuracy Trade-offs

[00:39:15] 2.4 Language Model Integration

[00:40:32] 2.5 Lattice-based Decoding Architecture

3. Performance Evaluation & Ethical Considerations

[00:44:00] 3.1 ASR Performance Metrics and Capabilities

[00:46:35] 3.2 AI Regulation and Evaluation Methods

[00:51:09] 3.3 Benchmark and Testing Challenges

[00:54:30] 3.4 Real-world Implementation Metrics

[01:00:51] 3.5 Ethics and Privacy Considerations

4. ASR Technical Evolution

[01:09:00] 4.1 WER Calculation and Evaluation Methodologies

[01:10:21] 4.2 Supervised vs Self-Supervised Learning Approaches

[01:21:02] 4.3 Temporal Learning and Feature Processing

[01:24:45] 4.4 Feature Engineering to Automated ML

5. Enterprise Implementation & Scale

[01:27:55] 5.1 Future AI Systems and Adaptation

[01:31:52] 5.2 Technical Foundations and History

[01:34:53] 5.3 Infrastructure and Team Scaling

[01:38:05] 5.4 Research and Talent Strategy

[01:41:11] 5.5 Engineering Practice Evolution

Shownotes:

https://www.dropbox.com/scl/fi/d94b1jcgph9o8au8shdym/Speechmatics.pdf?rlkey=bi55wvktzomzx0y5sic6jz99y&st=6qwofv8t&dl=0

Dr. Sanjeev Namjoshi, a machine learning engineer who recently submitted a book on Active Inference to MIT Press, discusses the theoretical foundations and practical applications of Active Inference, the Free Energy Principle (FEP), and Bayesian mechanics. He explains how these frameworks describe how biological and artificial systems maintain stability by minimizing uncertainty about their environment.

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

MLST is sponsored by Tufa Labs:

Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.

Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.

Interested? Apply for an ML research position: [email protected]

Namjoshi traces the evolution of these fields from early 2000s neuroscience research to current developments, highlighting how Active Inference provides a unified framework for perception and action through variational free energy minimization. He contrasts this with traditional machine learning approaches, emphasizing Active Inference's natural capacity for exploration and curiosity through epistemic value.

He sees Active Inference as being at a similar stage to deep learning in the early 2000s - poised for significant breakthroughs but requiring better tools and wider adoption. While acknowledging current computational challenges, he emphasizes Active Inference's potential advantages over reinforcement learning, particularly its principled approach to exploration and planning.

Dr. Sanjeev Namjoshi

https://snamjoshi.github.io/

TOC:

1. Theoretical Foundations: AI Agency and Sentience

[00:00:00] 1.1 Intro

[00:02:45] 1.2 Free Energy Principle and Active Inference Theory

[00:11:16] 1.3 Emergence and Self-Organization in Complex Systems

[00:19:11] 1.4 Agency and Representation in AI Systems

[00:29:59] 1.5 Bayesian Mechanics and Systems Modeling

2. Technical Framework: Active Inference and Free Energy

[00:38:37] 2.1 Generative Processes and Agent-Environment Modeling

[00:42:27] 2.2 Markov Blankets and System Boundaries

[00:44:30] 2.3 Bayesian Inference and Prior Distributions

[00:52:41] 2.4 Variational Free Energy Minimization Framework

[00:55:07] 2.5 VFE Optimization Techniques: Generalized Filtering vs DEM

3. Implementation and Optimization Methods

[00:58:25] 3.1 Information Theory and Free Energy Concepts

[01:05:25] 3.2 Surprise Minimization and Action in Active Inference

[01:15:58] 3.3 Evolution of Active Inference Models: Continuous to Discrete Approaches

[01:26:00] 3.4 Uncertainty Reduction and Control Systems in Active Inference

4. Safety and Regulatory Frameworks

[01:32:40] 4.1 Historical Evolution of Risk Management and Predictive Systems

[01:36:12] 4.2 Agency and Reality: Philosophical Perspectives on Models

[01:39:20] 4.3 Limitations of Symbolic AI and Current System Design

[01:46:40] 4.4 AI Safety Regulation and Corporate Governance

5. Socioeconomic Integration and Modeling

[01:52:55] 5.1 Economic Policy and Public Sentiment Modeling

[01:55:21] 5.2 Free Energy Principle: Libertarian vs Collectivist Perspectives

[01:58:53] 5.3 Regulation of Complex Socio-Technical Systems

[02:03:04] 5.4 Evolution and Current State of Active Inference Research

6. Future Directions and Applications

[02:14:26] 6.1 Active Inference Applications and Future Development

[02:22:58] 6.2 Cultural Learning and Active Inference

[02:29:19] 6.3 Hierarchical Relationship Between FEP, Active Inference, and Bayesian Mechanics

[02:33:22] 6.4 Historical Evolution of Free Energy Principle

[02:38:52] 6.5 Active Inference vs Traditional Machine Learning Approaches

Transcript and shownotes with refs and URLs:

https://www.dropbox.com/scl/fi/qj22a660cob1795ej0gbw/SanjeevShow.pdf?rlkey=w323r3e8zfsnve22caayzb17k&st=el1fdgfr&dl=0

Dr. Joscha Bach discusses advanced AI, consciousness, and cognitive modeling. He presents consciousness as a virtual property emerging from self-organizing software patterns, challenging panpsychism and materialism. Bach introduces "Cyberanima," reinterpreting animism through information processing, viewing spirits as self-organizing software agents.

He addresses limitations of current large language models and advocates for smaller, more efficient AI models capable of reasoning from first principles. Bach describes his work with Liquid AI on novel neural network architectures for improved expressiveness and efficiency.

The interview covers AI's societal implications, including regulation challenges and impact on innovation. Bach argues for balancing oversight with technological progress, warning against overly restrictive regulations.

Throughout, Bach frames consciousness, intelligence, and agency as emergent properties of complex information processing systems, proposing a computational framework for cognitive phenomena and reality.

SPONSOR MESSAGE:

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

TOC

[00:00:00] 1.1 Consciousness and Intelligence in AI Development

[00:07:44] 1.2 Agency, Intelligence, and Their Relationship to Physical Reality

[00:13:36] 1.3 Virtual Patterns and Causal Structures in Consciousness

[00:25:49] 1.4 Reinterpreting Concepts of God and Animism in Information Processing Terms

[00:32:50] 1.5 Animism and Evolution as Competition Between Software Agents

2. Self-Organizing Systems and Cognitive Models in AI

[00:37:59] 2.1 Consciousness as self-organizing software

[00:45:49] 2.2 Critique of panpsychism and alternative views on consciousness

[00:50:48] 2.3 Emergence of consciousness in complex systems

[00:52:50] 2.4 Neuronal motivation and the origins of consciousness

[00:56:47] 2.5 Coherence and Self-Organization in AI Systems

3. Advanced AI Architectures and Cognitive Processes

[00:57:50] 3.1 Second-Order Software and Complex Mental Processes

[01:01:05] 3.2 Collective Agency and Shared Values in AI

[01:05:40] 3.3 Limitations of Current AI Agents and LLMs

[01:06:40] 3.4 Liquid AI and Novel Neural Network Architectures

[01:10:06] 3.5 AI Model Efficiency and Future Directions

[01:19:00] 3.6 LLM Limitations and Internal State Representation

4. AI Regulation and Societal Impact

[01:31:23] 4.1 AI Regulation and Societal Impact

[01:49:50] 4.2 Open-Source AI and Industry Challenges

Refs in shownotes and MP3 metadata

Shownotes:

https://www.dropbox.com/scl/fi/g28dosz19bzcfs5imrvbu/JoschaInterview.pdf?rlkey=s3y18jy192ktz6ogd7qtvry3d&st=10z7q7w9&dl=0

Alessandro Palmarini is a post-baccalaureate researcher at the Santa Fe Institute working under the supervision of Melanie Mitchell. He completed his undergraduate degree in Artificial Intelligence and Computer Science at the University of Edinburgh. Palmarini's current research focuses on developing AI systems that can efficiently acquire new skills from limited data, inspired by François Chollet's work on measuring intelligence. His work builds upon the DreamCoder program synthesis system, introducing a novel approach called "dream decompiling" to improve library learning in inductive program synthesis. Palmarini is particularly interested in addressing the Abstraction and Reasoning Corpus (ARC) challenge, aiming to create AI systems that can perform abstract reasoning tasks more efficiently than current approaches. His research explores the balance between computational efficiency and data efficiency in AI learning processes.

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

TOC:

1. Intelligence Measurement in AI Systems

[00:00:00] 1.1 Defining Intelligence in AI Systems

[00:02:00] 1.2 Research at Santa Fe Institute

[00:04:35] 1.3 Impact of Gaming on AI Development

[00:05:10] 1.4 Comparing AI and Human Learning Efficiency

2. Efficient Skill Acquisition in AI

[00:06:40] 2.1 Intelligence as Skill Acquisition Efficiency

[00:08:25] 2.2 Limitations of Current AI Systems in Generalization

[00:09:45] 2.3 Human vs. AI Cognitive Processes

[00:10:40] 2.4 Measuring AI Intelligence: Chollet's ARC Challenge

3. Program Synthesis and ARC Challenge

[00:12:55] 3.1 Philosophical Foundations of Program Synthesis

[00:17:14] 3.2 Introduction to Program Induction and ARC Tasks

[00:18:49] 3.3 DreamCoder: Principles and Techniques

[00:27:55] 3.4 Trade-offs in Program Synthesis Search Strategies

[00:31:52] 3.5 Neural Networks and Bayesian Program Learning

4. Advanced Program Synthesis Techniques

[00:32:30] 4.1 DreamCoder and Dream Decompiling Approach

[00:39:00] 4.2 Beta Distribution and Caching in Program Synthesis

[00:45:10] 4.3 Performance and Limitations of Dream Decompiling

[00:47:45] 4.4 Alessandro's Approach to ARC Challenge

[00:51:12] 4.5 Conclusion and Future Discussions

Refs:

Full reflist on YT VD, Show Notes and MP3 metadata

Show Notes: https://www.dropbox.com/scl/fi/x50201tgqucj5ba2q4typ/Ale.pdf?rlkey=0ubvk7p5gtyx1gpownpdadim8&st=5pniu3nq&dl=0

François Chollet discusses the limitations of Large Language Models (LLMs) and proposes a new approach to advancing artificial intelligence. He argues that current AI systems excel at pattern recognition but struggle with logical reasoning and true generalization.

This was Chollet's keynote talk at AGI-24, filmed in high-quality. We will be releasing a full interview with him shortly. A teaser clip from that is played in the intro!

Chollet introduces the Abstraction and Reasoning Corpus (ARC) as a benchmark for measuring AI progress towards human-like intelligence. He explains the concept of abstraction in AI systems and proposes combining deep learning with program synthesis to overcome current limitations. Chollet suggests that breakthroughs in AI might come from outside major tech labs and encourages researchers to explore new ideas in the pursuit of artificial general intelligence.

TOC

1. LLM Limitations and Intelligence Concepts

[00:00:00] 1.1 LLM Limitations and Composition

[00:12:05] 1.2 Intelligence as Process vs. Skill

[00:17:15] 1.3 Generalization as Key to AI Progress

2. ARC-AGI Benchmark and LLM Performance

[00:19:59] 2.1 Introduction to ARC-AGI Benchmark

[00:20:05] 2.2 Introduction to ARC-AGI and the ARC Prize

[00:23:35] 2.3 Performance of LLMs and Humans on ARC-AGI

3. Abstraction in AI Systems

[00:26:10] 3.1 The Kaleidoscope Hypothesis and Abstraction Spectrum

[00:30:05] 3.2 LLM Capabilities and Limitations in Abstraction

[00:32:10] 3.3 Value-Centric vs Program-Centric Abstraction

[00:33:25] 3.4 Types of Abstraction in AI Systems

4. Advancing AI: Combining Deep Learning and Program Synthesis

[00:34:05] 4.1 Limitations of Transformers and Need for Program Synthesis

[00:36:45] 4.2 Combining Deep Learning and Program Synthesis

[00:39:59] 4.3 Applying Combined Approaches to ARC Tasks

[00:44:20] 4.4 State-of-the-Art Solutions for ARC

Shownotes (new!): https://www.dropbox.com/scl/fi/i7nsyoahuei6np95lbjxw/CholletKeynote.pdf?rlkey=t3502kbov5exsdxhderq70b9i&st=1ca91ewz&dl=0

[0:01:15] Abstraction and Reasoning Corpus (ARC): AI benchmark (François Chollet)

https://arxiv.org/abs/1911.01547

[0:05:30] Monty Hall problem: Probability puzzle (Steve Selvin)

https://www.tandfonline.com/doi/abs/10.1080/00031305.1975.10479121

[0:06:20] LLM training dynamics analysis (Tirumala et al.)

https://arxiv.org/abs/2205.10770

[0:10:20] Transformer limitations on compositionality (Dziri et al.)

https://arxiv.org/abs/2305.18654

[0:10:25] Reversal Curse in LLMs (Berglund et al.)

https://arxiv.org/abs/2309.12288

[0:19:25] Measure of intelligence using algorithmic information theory (François Chollet)

https://arxiv.org/abs/1911.01547

[0:20:10] ARC-AGI: GitHub repository (François Chollet)

https://github.com/fchollet/ARC-AGI

[0:22:15] ARC Prize: $1,000,000+ competition (François Chollet)

https://arcprize.org/

[0:33:30] System 1 and System 2 thinking (Daniel Kahneman)

https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555

[0:34:00] Core knowledge in infants (Elizabeth Spelke)

https://www.harvardlds.org/wp-content/uploads/2017/01/SpelkeKinzler07-1.pdf

[0:34:30] Embedding interpretive spaces in ML (Tennenholtz et al.)

https://arxiv.org/abs/2310.04475

[0:44:20] Hypothesis Search with LLMs for ARC (Wang et al.)

https://arxiv.org/abs/2309.05660

[0:44:50] Ryan Greenblatt's high score on ARC public leaderboard

https://arcprize.org/

Ivan Zhang, co-founder of Cohere, discusses the company's enterprise-focused AI solutions. He explains Cohere's early emphasis on embedding technology and training models for secure environments.

Zhang highlights their implementation of Retrieval-Augmented Generation in healthcare, significantly reducing doctor preparation time. He explores the shift from monolithic AI models to heterogeneous systems and the importance of improving various AI system components. Zhang shares insights on using synthetic data to teach models reasoning, the democratization of software development through AI, and how his gaming skills transfer to running an AI company.

He advises young developers to fully embrace AI technologies and offers perspectives on AI reliability, potential risks, and future model architectures.

https://cohere.com/

https://ivanzhang.ca/

https://x.com/1vnzh

TOC:

00:00:00 Intro

00:03:20 AI & Language Model Evolution

00:06:09 Future AI Apps & Development

00:09:29 Impact on Software Dev Practices

00:13:03 Philosophical & Societal Implications

00:16:30 Compute Efficiency & RAG

00:20:39 Adoption Challenges & Solutions

00:22:30 GPU Optimization & Kubernetes Limits

00:24:16 Cohere's Implementation Approach

00:28:13 Gaming's Professional Influence

00:34:45 Transformer Optimizations

00:36:45 Future Models & System-Level Focus

00:39:20 Inference-Time Computation & Reasoning

00:42:05 Capturing Human Thought in AI

00:43:15 Research, Hiring & Developer Advice

REFS:

00:02:31 Cohere, https://cohere.com/

00:02:40 The Transformer architecture, https://arxiv.org/abs/1706.03762

00:03:22 The Innovator's Dilemma, https://www.amazon.com/Innovators-Dilemma-Technologies-Management-Innovation/dp/1633691780

00:09:15 The actor model, https://en.wikipedia.org/wiki/Actor_model

00:14:35 John Searle's Chinese Room Argument, https://plato.stanford.edu/entries/chinese-room/

00:18:00 Retrieval-Augmented Generation, https://arxiv.org/abs/2005.11401

00:18:40 Retrieval-Augmented Generation, https://docs.cohere.com/v2/docs/retrieval-augmented-generation-rag

00:35:39 Let’s Verify Step by Step, https://arxiv.org/pdf/2305.20050

00:39:20 Adaptive Inference-Time Compute, https://arxiv.org/abs/2410.02725

00:43:20 Ryan Greenblatt ARC entry, https://redwoodresearch.substack.com/p/getting-50-sota-on-arc-agi-with-gpt

Disclaimer: This show is part of our Cohere partnership series

Prof. Tim Rocktäschel, AI researcher at UCL and Google DeepMind, talks about open-ended AI systems. These systems aim to keep learning and improving on their own, like evolution does in nature.

Ad: Are you a hardcore ML engineer who wants to work for Daniel Cahn at SlingshotAI building AI for mental health? Give him an email! - danielc@slingshot.xyz

TOC:

00:00:00 Introduction to Open-Ended AI and Key Concepts

00:01:37 Tim Rocktäschel's Background and Research Focus

00:06:25 Defining Open-Endedness in AI Systems

00:10:39 Subjective Nature of Interestingness and Learnability

00:16:22 Open-Endedness in Practice: Examples and Limitations

00:17:50 Assessing Novelty in Open-ended AI Systems

00:20:05 Adversarial Attacks and AI Robustness

00:24:05 Rainbow Teaming and LLM Safety

00:25:48 Open-ended Research Approaches in AI

00:29:05 Balancing Long-term Vision and Exploration in AI Research

00:37:25 LLMs in Program Synthesis and Open-Ended Learning

00:37:55 Transition from Human-Based to Novel AI Strategies

00:39:00 Expanding Context Windows and Prompt Evolution

00:40:17 AI Intelligibility and Human-AI Interfaces

00:46:04 Self-Improvement and Evolution in AI Systems

Show notes (New!) https://www.dropbox.com/scl/fi/5avpsyz8jbn4j1az7kevs/TimR.pdf?rlkey=pqjlcqbtm3undp4udtgfmie8n&st=x50u1d1m&dl=0

REFS:

00:01:47 - UCL DARK Lab (Rocktäschel) - AI research lab focusing on RL and open-ended learning - https://ucldark.com/

00:02:31 - GENIE (Bruce) - Generative interactive environment from unlabelled videos - https://arxiv.org/abs/2402.15391

00:02:42 - Promptbreeder (Fernando) - Self-referential LLM prompt evolution - https://arxiv.org/abs/2309.16797

00:03:05 - Picbreeder (Secretan) - Collaborative online image evolution - https://dl.acm.org/doi/10.1145/1357054.1357328

00:03:14 - Why Greatness Cannot Be Planned (Stanley) - Book on open-ended exploration - https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237

00:04:36 - NetHack Learning Environment (Küttler) - RL research in procedurally generated game - https://arxiv.org/abs/2006.13760

00:07:35 - Open-ended learning (Clune) - AI systems for continual learning and adaptation - https://arxiv.org/abs/1905.10985

00:07:35 - OMNI (Zhang) - LLMs modeling human interestingness for exploration - https://arxiv.org/abs/2306.01711

00:10:42 - Observer theory (Wolfram) - Computationally bounded observers in complex systems - https://writings.stephenwolfram.com/2023/12/observer-theory/

00:15:25 - Human-Timescale Adaptation (Rocktäschel) - RL agent adapting to novel 3D tasks - https://arxiv.org/abs/2301.07608

00:16:15 - Open-Endedness for AGI (Hughes) - Importance of open-ended learning for AGI - https://arxiv.org/abs/2406.04268

00:16:35 - POET algorithm (Wang) - Open-ended approach to generate and solve challenges - https://arxiv.org/abs/1901.01753

00:17:20 - AlphaGo (Silver) - AI mastering the game of Go - https://deepmind.google/technologies/alphago/

00:20:35 - Adversarial Go attacks (Dennis) - Exploiting weaknesses in Go AI systems - https://www.ifaamas.org/Proceedings/aamas2024/pdfs/p1630.pdf

00:22:00 - Levels of AGI (Morris) - Framework for categorizing AGI progress - https://arxiv.org/abs/2311.02462

00:24:30 - Rainbow Teaming (Samvelyan) - LLM-based adversarial prompt generation - https://arxiv.org/abs/2402.16822

00:25:50 - Why Greatness Cannot Be Planned (Stanley) - 'False compass' and 'stepping stone collection' concepts - https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237

00:27:45 - AI Debate (Khan) - Improving LLM truthfulness through debate - https://proceedings.mlr.press/v235/khan24a.html

00:29:40 - Gemini (Google DeepMind) - Advanced multimodal AI model - https://deepmind.google/technologies/gemini/

00:30:15 - How to Take Smart Notes (Ahrens) - Effective note-taking methodology - https://www.amazon.com/How-Take-Smart-Notes-Nonfiction/dp/1542866502

(truncated, see shownotes)

Ben Goertzel discusses AGI development, transhumanism, and the potential societal impacts of superintelligent AI. He predicts human-level AGI by 2029 and argues that the transition to superintelligence could happen within a few years after. Goertzel explores the challenges of AI regulation, the limitations of current language models, and the need for neuro-symbolic approaches in AGI research. He also addresses concerns about resource allocation and cultural perspectives on transhumanism.

TOC:

[00:00:00] AGI Timeline Predictions and Development Speed

[00:00:45] Limitations of Language Models in AGI Development

[00:02:18] Current State and Trends in AI Research and Development

[00:09:02] Emergent Reasoning Capabilities and Limitations of LLMs

[00:18:15] Neuro-Symbolic Approaches and the Future of AI Systems

[00:20:00] Evolutionary Algorithms and LLMs in Creative Tasks

[00:21:25] Symbolic vs. Sub-Symbolic Approaches in AI

[00:28:05] Language as Internal Thought and External Communication

[00:30:20] AGI Development and Goal-Directed Behavior

[00:35:51] Consciousness and AI: Expanding States of Experience

[00:48:50] AI Regulation: Challenges and Approaches

[00:55:35] Challenges in AI Regulation

[00:59:20] AI Alignment and Ethical Considerations

[01:09:15] AGI Development Timeline Predictions

[01:12:40] OpenCog Hyperon and AGI Progress

[01:17:48] Transhumanism and Resource Allocation Debate

[01:20:12] Cultural Perspectives on Transhumanism

[01:23:54] AGI and Post-Scarcity Society

[01:31:35] Challenges and Implications of AGI Development

New! PDF Show notes: https://www.dropbox.com/scl/fi/fyetzwgoaf70gpovyfc4x/BenGoertzel.pdf?rlkey=pze5dt9vgf01tf2wip32p5hk5&st=svbcofm3&dl=0

Refs:

00:00:15 Ray Kurzweil's AGI timeline prediction, Ray Kurzweil, https://en.wikipedia.org/wiki/Technological_singularity

00:01:45 Ben Goertzel: SingularityNET founder, Ben Goertzel, https://singularitynet.io/

00:02:35 AGI Conference series, AGI Conference Organizers, https://agi-conf.org/2024/

00:03:55 Ben Goertzel's contributions to AGI, Wikipedia contributors, https://en.wikipedia.org/wiki/Ben_Goertzel

00:11:05 Chain-of-Thought prompting, Subbarao Kambhampati, https://arxiv.org/abs/2405.04776

00:11:35 Algorithmic information content, Pieter Adriaans, https://plato.stanford.edu/entries/information-entropy/

00:12:10 Turing completeness in neural networks, Various contributors, https://plato.stanford.edu/entries/turing-machine/

00:16:15 AlphaGeometry: AI for geometry problems, Trieu, Li, et al., https://www.nature.com/articles/s41586-023-06747-5

00:18:25 Shane Legg and Ben Goertzel's collaboration, Shane Legg, https://en.wikipedia.org/wiki/Shane_Legg

00:20:00 Evolutionary algorithms in music generation, Yanxu Chen, https://arxiv.org/html/2409.03715v1

00:22:00 Peirce's theory of semiotics, Charles Sanders Peirce, https://plato.stanford.edu/entries/peirce-semiotics/

00:28:10 Chomsky's view on language, Noam Chomsky, https://chomsky.info/1983____/

00:34:05 Greg Egan's 'Diaspora', Greg Egan, https://www.amazon.co.uk/Diaspora-post-apocalyptic-thriller-perfect-MIRROR/dp/0575082097

00:40:35 'The Consciousness Explosion', Ben Goertzel & Gabriel Axel Montes, https://www.amazon.com/Consciousness-Explosion-Technological-Experiential-Singularity/dp/B0D8C7QYZD

00:41:55 Ray Kurzweil's books on singularity, Ray Kurzweil, https://www.amazon.com/Singularity-Near-Humans-Transcend-Biology/dp/0143037889

00:50:50 California AI regulation bills, California State Senate, https://sd18.senate.ca.gov/news/senate-unanimously-approves-senator-padillas-artificial-intelligence-package

00:56:40 Limitations of Compute Thresholds, Sara Hooker, https://arxiv.org/abs/2407.05694

00:56:55 'Taming Silicon Valley', Gary F. Marcus, https://www.penguinrandomhouse.com/books/768076/taming-silicon-valley-by-gary-f-marcus/

01:09:15 Kurzweil's AGI prediction update, Ray Kurzweil, https://www.theguardian.com/technology/article/2024/jun/29/ray-kurzweil-google-ai-the-singularity-is-nearer

AI expert Prof. Gary Marcus doesn't mince words about today's artificial intelligence. He argues that despite the buzz, chatbots like ChatGPT aren't as smart as they seem and could cause real problems if we're not careful.

Marcus is worried about tech companies putting profits before people. He thinks AI could make fake news and privacy issues even worse. He's also concerned that a few big tech companies have too much power. Looking ahead, Marcus believes the AI hype will die down as reality sets in. He wants to see AI developed in smarter, more responsible ways. His message to the public? We need to speak up and demand better AI before it's too late.

Buy Taming Silicon Valley:

https://amzn.to/3XTlC5s

Gary Marcus:

https://garymarcus.substack.com/

https://x.com/GaryMarcus

Interviewer:

Dr. Tim Scarfe

(Refs in top comment)

TOC

[00:00:00] AI Flaws, Improvements & Industry Critique

[00:16:29] AI Safety Theater & Image Generation Issues

[00:23:49] AI's Lack of World Models & Human-like Understanding

[00:31:09] LLMs: Superficial Intelligence vs. True Reasoning

[00:34:45] AI in Specialized Domains: Chess, Coding & Limitations

[00:42:10] AI-Generated Code: Capabilities & Human-AI Interaction

[00:48:10] AI Regulation: Industry Resistance & Oversight Challenges

[00:54:55] Copyright Issues in AI & Tech Business Models

[00:57:26] AI's Societal Impact: Risks, Misinformation & Ethics

[01:23:14] AI X-risk, Alignment & Moral Principles Implementation

[01:37:10] Persistent AI Flaws: System Limitations & Architecture Challenges

[01:44:33] AI Future: Surveillance Concerns, Economic Challenges & Neuro-Symbolic AI

YT version with refs: https://youtu.be/o9MfuUoGlSw

Prof. Mark Solms, a neuroscientist and psychoanalyst, discusses his groundbreaking work on consciousness, challenging conventional cortex-centric views and emphasizing the role of brainstem structures in generating consciousness and affect.

MLST is sponsored by Brave:

The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.

Key points discussed:

The limitations of vision-centric approaches to consciousness studies.

Evidence from decorticated animals and hydranencephalic children supporting the brainstem's role in consciousness.

The relationship between homeostasis, the free energy principle, and consciousness.

Critiques of behaviorism and modern theories of consciousness.

The importance of subjective experience in understanding brain function.

The discussion also explored broader topics:

The potential impact of affect-based theories on AI development.

The role of the SEEKING system in exploration and learning.

Connections between neuroscience, psychoanalysis, and philosophy of mind.

Challenges in studying consciousness and the limitations of current theories.

Mark Solms:

https://neuroscience.uct.ac.za/contacts/mark-solms

Show notes and transcript: https://www.dropbox.com/scl/fo/roipwmnlfmwk2e7kivzms/ACjZF-VIGC2-Suo30KcwVV0?rlkey=53y8v2cajfcgrf17p1h7v3suz&st=z8vu81hn&dl=0

TOC (*) are best bits

00:00:00 1. Intro: Challenging vision-centric approaches to consciousness *

00:02:20 2. Evidence from decorticated animals and hydranencephalic children *

00:07:40 3. Emotional responses in hydranencephalic children

00:10:40 4. Brainstem stimulation and affective states

00:15:00 5. Brainstem's role in generating affective consciousness *

00:21:50 6. Dual-aspect monism and the mind-brain relationship

00:29:37 7. Information, affect, and the hard problem of consciousness *

00:37:25 8. Wheeler's participatory universe and Chalmers' theories

00:48:51 9. Homeostasis, free energy principle, and consciousness *

00:59:25 10. Affect, voluntary behavior, and decision-making

01:05:45 11. Psychoactive substances, REM sleep, and consciousness research

01:12:14 12. Critiquing behaviorism and modern consciousness theories *

01:24:25 13. The SEEKING system and exploration in neuroscience

Refs:

1. Mark Solms' book "The Hidden Spring" [00:20:34] (MUST READ!)

https://amzn.to/3XyETb3

2. Karl Friston's free energy principle [00:03:50]

https://www.nature.com/articles/nrn2787

3. Hydranencephaly condition [00:07:10]

https://en.wikipedia.org/wiki/Hydranencephaly

4. Periaqueductal gray (PAG) [00:08:57]

https://en.wikipedia.org/wiki/Periaqueductal_gray

5. Positron Emission Tomography (PET) [00:13:52]

https://en.wikipedia.org/wiki/Positron_emission_tomography

6. Paul MacLean's triune brain theory [00:03:30]

https://en.wikipedia.org/wiki/Triune_brain

7. Baruch Spinoza's philosophy of mind [00:23:48]

https://plato.stanford.edu/entries/spinoza-epistemology-mind

8. Claude Shannon's "A Mathematical Theory of Communication" [00:32:15]

https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf

9. Francis Crick's "The Astonishing Hypothesis" [00:39:57]

https://en.wikipedia.org/wiki/The_Astonishing_Hypothesis

10. Frank Jackson's Knowledge Argument [00:40:54]

https://plato.stanford.edu/entries/qualia-knowledge/

11. Mesolimbic dopamine system [01:11:51]

https://en.wikipedia.org/wiki/Mesolimbic_pathway

12. Jaak Panksepp's SEEKING system [01:25:23]

https://en.wikipedia.org/wiki/Jaak_Panksepp#Affective_neuroscience