Co-hosts Calum and David dig deep into aspects of David's recent new book "The Singularity Principles". Calum (CC) says he is, in part, unconvinced. David  (DW) agrees that the projects he recommends are hard, but suggests some practical ways forward.

0.25 The technological singularity may be nearer than we think
1.10 Confusions about the singularity
1.35 “Taking back control of the singularity”
2.40 The “Singularity Shadow”: over-confident predictions which repulse people
3.30 The over-confidence includes predictions of timescale…
4.00 … and outcomes
4.45 The Singularity as the Rapture of the Nerds?
5.20 The Singularity is not a religion …
5.40 .. although if positive, it will confer almost godlike powers
6.35 Much discussion of the Singularity is dystopian, but there could be enormous benefits, including…
7.15 Digital twins for cells and whole bodies, and super longevity
7.30 A new enlightenment
7.50 Nuclear fusion
8.10 Humanity’s superpower is intelligence
8.30 Amplifying our intelligence should increase our power
9.50 DW’s timeline: 50% chance of AGI by 2050, 10% by 2030
10.10 The timeline is contingent on human actions
10.40 Even if AGI isn’t coming until 2070, we should be working on AI alignment today
11.10 AI Impact’s survey of all contributors to NeurIPS
11.35 Median view: 50% chance of AGI in 2059, and many were pessimistic
12.15 This discussion can’t be left to AI researchers
12.40 A bad beta version might be our last invention
13.00 A few hundred people are now working on AI alignment, and tens of thousands on advancing AI
13.35 The growth of the AI research population is still faster
13.40 CC: Three routes to a positive outcome
13.55 1. Luck. The world turns out to be configured in our favour
14.30 2. Mathematical approaches to AI alignment succeed
14.45 We either align AIs forever, or manage to control them. This is very hard
14.55 3. We merge with the superintelligent machines
15.40 Uploading is a huge engineering challenge
15.55 Philosophical issues raised by uploading: is the self retained?
16.10 DW: routes 2 and 3 are too binary. A fourth route is solving morality
18.15 Individual humans will be augmented, indeed we already are
18.55 But augmented humans won’t necessarily be benign
19.30 DW: We have to solve beneficence
20.00 CC: We can’t hope to solve our moral debates before AGI arrives
20.20 In which case we are relying on route 1 – luck
20.30 DW: Progress in philosophy *is* possible, and must be accelerated
21.15 The Universal Declaration of Human Rights shows that generalised moral principles can be agreed
22.25 CC: That sounds impossible. The UDHR is very broad and often ignored
23.05 Solving morality is even harder than the MIRI project, and reinforces the idea that route 3 is our best hope
23.50 It’s not unreasonable to hope that wisdom correlates with intelligence
24.00 DW: We can proceed step by step, starting with progress on facial recognition, autonomous weapons, and such intermediate questions
25.10 CC: We are so far from solving moral questions. Americans can’t even agree if a coup against their democracy was a bad thing
25.40 DW: We have to make progress, and quickly. AI might help us.
26.50 The essence of transhumanism is that we can use technology to improve ourselves
27.20 CC: If you had a magic wand, your first wish should probably be to make all humans see each other as members of the same tribe
27.50 Is AI ethics a helpful term?
28.05 AI ethics is a growing profession, but if problems are ethical then people who disagree with you are bad, not just wrong
28.55 AI ethics makes debates about AI harder to resolve, and more angry
29.15 AI researchers are understandably offended by finger-wagging, self-proclaimed AI ethicists who may not understand what they are talking about

How likely is it that, by 2030, someone will build artificial general intelligence (AGI)?

Ross Nordby is an AI researcher who has shortened his AGI timelines: he has changed his mind about when AGI might be expected to exist. He recently published an article on the LessWrong community discussion site, giving his argument in favour of shortening these timelines. He now identifies 2030 as the date by which it is 50% likely that AGI will exist. In this episode, we ask Ross questions about his argument, and consider some of the implications that arise.

Article by Ross: https://www.lesswrong.com/posts/K4urTDkBbtNuLivJx/why-i-think-strong-general-ai-is-coming-soon

Effective Altruism Long-Term Future Fund: https://funds.effectivealtruism.org/funds/far-future

MIRI (Machine Intelligence Research Institution): https://intelligence.org/

00.57 Ross’ background: real-time graphics, mostly in video games
02.10 Increased familiarity with AI made him reconsider his AGI timeline
02.37 He submitted a grant request to the Effective Altruism Long-Term Future Fund to move into AI safety work
03.50 What Ross was researching: can we make an AI intrinsically interpretable?
04.25 The AGI Ross is interested in is defined by capability, regardless of consciousness or sentience
04.55 An AI that is itself "goalless" might be put to uses with destructive side-effects
06.10 The leading AI research groups are still DeepMind and OpenAI
06.43 Other groups, like Anthropic, are more interested in alignment
07.22 If you can align an AI to any goal at all, that is progress: it indicates you have some control
08.00 Is this not all abstract and theoretical - a distraction from more pressing problems?
08.30 There are other serious problems, like pandemics and global warming, but we have to solve them all
08.45 Globally, only around 300 people are focused on AI alignment: not enough
10.05 AGI might well be less than three decades away
10.50 AlphaGo surprised the community, which was expecting Go to be winnable 10-15 years later
11.10 Then AlphaGo was surpassed by systems like AlphaZero and MuZero, which were actually simpler, and more flexible
11.20 AlphaTensor frames matrix multiplication as a game, and becomes superhuman at it
11.40 In 2018, the Transformer paper was published, but no-one forecast GPT-3’s capabilities
12.00 This year, Minerva (similar to GPT-3) got 50% correct on the math dataset: high school competition math problems
13.16 Illustrators now feel threatened by systems like Dall-E, Stable Diffusion, etc
13.30 The conclusion is that intelligence is easier to simulate than we thought
13.40 But these systems also do stupid things. They are brittle
18.00 But we could use transformers more intelligently
19.20 They turn out to be able to write code, and to explain jokes, and do maths reasoning
21:10 Google's Gopher AI
22.05 Machines don’t yet have internal models of the world, which we call common sense
24.00 But an early version of GPT-3 demonstrated the ability to model a human thought process alongside a machine’s
27.15 Ross’ current timeline is 50% probability of AGI by 2030, and 90+% by 2050
27:35 Counterarguments?
29.35 So what is to be done?
30.55 If convinced that AGI is coming soon, most lay people would probably demand that all AI research stops immediately. Which isn’t possible
31.40 Maybe publicity would be good in order to generate resources for AI alignment. And to avoid a backlash against secrecy
33.55 It would be great if more billionaires opened their wallets, but actually there are funds available for people who want to work on the problem
34.20 People who can help would not have to take a pay cut to work on AI alignment

Audio engineering by Alexander Chace

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Why do human brains consume much less power than artificial neural networks? Simon Thorpe, Research Director of CNRS, explains his view that the key to artificial general intelligence is a "terabrain" that copies from human brains the sparse-firing networks with spiking neurons.

00.11 Recapping "the AI paradox"
00.28 The nervousness of CTOs regarding AI
00.43 Introducing Simon
01.43 45 years since Oxford, working out how the brain does amazing things
02.45 Brain visual perception as feed-forward vs. feedback
03.40 The ideas behind the system that performed so well in the 2012 ImageNet challenge
04.20 The role of prompts to alter perception
05.30 Drawbacks of human perceptual expectations
06.05 The video of a gorilla on the basketball court
06.50 Conjuring tricks and distractions
07.10 Energy consumption: human neurons vs. artificial neurons
07.26 The standard model would need 500 petaflops
08.40 Exaflop computing has just arrived
08.50 30 MW vs. 20 W (less than a lightbulb)
09.34 Companies working on low-power computing systems
09.48 Power requirements for edge computing
10.10 The need for 86,000 neuromorphic chips?
10.25 Dense activation of neurons vs. sparse activation
10.58 Real brains are event driven
11.16 Real neurons send spikes not floating point numbers
11.55 SpikeNET by Arnaud Delorme
12.50 Why are sparse networks studied so little?
14.40 A recent debate with Yann LeCun of Facebook and Bill Dally of Nvidia
15.40 One spike can contain many bits of information
16.24 Revisiting an experiment with eels from 1927 (Lord Edgar Adrian)
17.06 Biology just needs one spike
17.50 Chips moved from floating point to fixed point
19.25 Other mentions of sparse systems - MoE (Mixture of Experts)
19.50 Sparse systems are easier to interpret
20.30 Advocacy for "grandmother cells"
21.23 Chicks that imprinted on yellow boots
22.35 A semantic web in the 1960s
22.50 The Mozart cell
23.02 An expert system implemented in a neural network with spiking neurons
23.14 Power consumption reduced by a factor of one million
23.40 Experimental progress
23.53 Dedicated silicon: Spikenet Technology, acquired by BrainChip
24.18 The Terabrain Project, using standard off-the-shelf hardware
24.40 Impressive recent simulations on GPUs and on a MacBook Pro
26.26 A homegrown learning rule
26.44 Experiments with "frozen noise"
27.28 Anticipating emulating an entire human brain on a Mac Studio M1 Ultra
28.25 The likely impact of these ideas
29.00 This software will be given away
29.17 Anticipating "local learning" without the results being sent to Big Tech
30.40 GPT-3 could run on your phone next year
31.12 Our interview next year might be, not with Simon, but with his Terabrain
31.22 Our phones know us better than our spouses do

Simon's academic page: https://cerco.cnrs.fr/page-perso-simon-thorpe/

Simon's personal blog: https://simonthorpesideas.blogspot.com/

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

This episode features Daniel Hulme, founder of Satalia and chief AI officer at WPP. What is AI good at today? And how can organisations increase the likelihood of deploying AI successfully?

02.55 What is AI good at today?
03.25 Deep learning isn’t yet being widely used in companies. Executives are wary of self-adapting systems
04.15 Six categories of AI deployment today
04.20 1. Automation. Using “if … then …” statements
04.50 2. Generative AI, like Dall-E
05.15 3. Humanisation, like DeepFake technology and natural language models
05.40 4. Machine learning to extract insights from data – finding correlations that humans could not
06.05 5. Complex decision making, aka operations research, or optimisation. “Companies don’t have ML problems, they have decision problems”
06.25 6. Augmenting humans physically or cognitively
06.50 Aren’t the tech giants using true AI systems in their operations?
07.15 A/B testing is a simple form of adaptation. Google A/B tested the colours of their logo
08 .00 Complex adaptive systems with many moving parts are much riskier. If they go wrong, huge damage can occur
08.30 CTOs demand consistency from operational systems, and can’t tolerate the mistakes that are essential to learning
09.25 Can’t the mistakes be made in simulated environments?
10.20 Elon Musk says simulating the world is not how to develop self-driving cars
10.45 Companies undergoing digital transformations are building ERPs, which are “glorified databases”
11.20 The idea is to develop digital twins, which enable them to ask “what if…” questions
11.30 The coming confluence of three digital twins: workflow, workforce, and administrative processes
12.18 Why don’t supermarkets offer digital twins to their customers? They’re coming
14.55 People often think that creating a data lake and adding a system like Tableau on top is deploying AI
15.15 Even if you give humans better insights they often don’t make better decisions
15.20 Data scientists are not equipped to address opportunities in all 6 of the categories listed earlier
15.40 Companies should start by identifying and then prioritising the frictions in their organisations
16.10 Some companies are taking on “tech debt” which they will have to unwind in five years
16.25 Why aren’t large process industry companies boasting about massive revenue improvements or cost savings?
17.00 To make those decisions you need the right data, and top optimisation skills. That’s unusual
17.55 Companies ask for “quick wins” but that is an oxymoron
18.10 We do see project ROIs of 200%, but most projects fail due to under-investment, or mis-understandings
19.00 Don’t start by just collecting data. The example of a low-cost airline which collected data about everything except rivals’ pricing
20.15 Humans usually do know where the signals are
22.25 Some of Daniel’s favourite AI projects
23.00 Tesco’s last-mile delivery system, which saves 20m delivery miles a year
24.00 Solving PwC’s consultant allocation problem radically improved many lives
25.10 In the next decade there will be a move away from pure ML towards ML+ optimisation
26.35 How these systems have been applied to Satalia
28.10 Daniel has thought a lot about how AI can enable companies to be very adaptable, and allocate decisions well
29.00 Satalia staff used to make recommendations for their own salaries, and their colleagues would make AI-weighted votes
29.30 The goal is to scale this approach not just across WPP, but across the planet
30.35 Heads of HR in WPP operating companies love the idea

Daniel's entry on Wikipedia: https://en.wikipedia.org/wiki/Daniel_J._Hulme

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Calum and David reflect on their involvement in two recent conferences, one in Riyadh, and one in Dublin. Each conference highlighted a potential disruption in a major industry: a country with large ambitions in the AI space, and a new foundation in the longevity space.

00.00 A tale of two cities, two conferences, two industries
00.44 First, the 2nd Saudi Global AI Conference
01.03 Vision 2030
01.11 Saudi has always been a coalition between the fundamentalist Wahhabis and the Royal Family
01.38 The King chooses reform in the wake of 9/11
02.07 Mohamed bin Salman appointed Crown Prince, who embarks on reform
02.28 The partial liberation of women, and the fundamentalists side-lined
03.10 The “Sheikhdown” in 2017
03.49 The Khashoggi affair and the Yemen war lead to Saudi being shunned
04.26 The West is missing what’s going on in Saudi
05.00 Lifting the Saudi economy’s reliance on petrochemicals
05.27 AI is central to Vision 2030
06.00 Can Saudi become one of the world’s top 10 or 15 AI countries?
06.20 The AI duopoly between the US and China is so strong, this isn’t as hard as you might think
06.55 Saudi’s advantages
07.22 Saudi’s disadvantages
07.54 The goal is not implausible
08.10 The short-term goals of the conference. A forum for discussions, deals, and trying to open the world’s eyes
09.45 Saudi is arguably on the way to becoming another Dubai. Continuation and success are not inevitable, but it is encouraging
11.00 Fastest-growth country in the G20, with an oil bonanza
11.25 The proposed brand-new city of Neom with The Line, a futuristic environment
13.07 The second conference: the Longevity Summit in Dublin
13.48 A new foundation announced
14.05 Reports updating on progress in longevity research around the world
14.20 A dozen were new and surprising. Four examples…
14.50 1. Bats. A speaker from Dublin discussed why they live so long – 40 years – and what we can learn from that
15.55 2. Parabiosis on steroids. Linking the blood flow of two animals suggests there are aging elements in our blood which can be removed
17.50 3. Using AI to develop drugs. Companies like Exscientia and Insilico. Cortex Discovery is a smaller, perhaps more nimble player
19.40 4. Hevolution, a new longevity fund backed with up to $1bn of Saudi money per year for 20 years
22.05 As Aubrey de Grey has long said, we need engineering as much as research
22.40 Aubrey thinks aging should be tackled by undoing cell damage rather than changing the human metabolism
24.00 Three phases of his career. Methuselah. SENS. New foundation
25.00 Let’s avoid cancer, heart disease and dementias by continually reversing aging damage
26.00 He is always itchy to explore new areas. This led to a power struggle within SENS, which he lost
27.00 What should previous SENS donors do now?
27.15 The rich crypto investors who have provided large amounts to SENS are backing the new foundation
28.30 One of the new foundation’s investment areas will be parabiosis
28.55 Cryonics will be another investment area
29.15 Lobbying legislators will be another
29.50 Robust Mouse Rejuvenation will be the initial priority
30.50 Pets may be the animal models whose rejuvenation breaks humanity’s “trance of death”
31.05 David has been appointed a director the new foundation
31.50 The other directors
33.05 An exciting future

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

The conference websites: https://globalaisummit.org/ and https://longevitysummitdublin.com/

For more about the podcast hosts, see https://calumchace.com/ and https://dw2blog.com/

This episode continues our discussion with AI researcher Aleksa Gordić from DeepMind on understanding today’s most advanced AI systems.

00.07 This episode builds on Episode 5
01.05 We start with GANs – Generative Adversarial Networks
01.33 Solving the problem of stability, with higher resolution
03.24 GANs are notoriously hard to train. They suffer from mode collapse
03.45 Worse, the model might not learn anything, and the result is pure noise
03.55 DC GANs introduced convolutional layers to stabilise them and enable higher resolution
04.37 The technique of outpainting
05.55 Generating text as well as images, and producing stories
06.14 AI Dungeon
06.28 From GANs to Diffusion models
06.48 DDPM (De-noising diffusion probabilistic models) does for diffusion models what DC GANs did for GANs
07.20 They are more stable, and don’t suffer from mode collapse
07.30 They do have downsides. They are much more computation intensive
08.24 What does the word diffusion mean in this context?
08.40 It’s adopted from physics. It peels noise away from the image
09.17 Isn’t that rewinding entropy?
09.45 One application is making a photo taken in 1830 look like one taken yesterday
09.58 Semantic Segmentation Masks convert bands of flat colour into realistic images of sky, earth, sea, etc
10.35 Bounding boxes generate objects of a specified class from tiny inputs
11.00 The images are not taken from previously seen images on the internet, but invented from scratch
11.40 The model saw a lot of images during training, but during the creation process it does not refer back to them
12.40 Failures are eliminated by amendments, as always with models like this
12.55 Scott Alexander blogged about models producing images with wrong relationships, and how this was fixed within 3 months
13.30 The failure modes get harder to find as the obvious ones are eliminated
13.45 Even with 175 billion parameters, GPT-3 struggled to handle three digits in computation
15.18 Are you often surprised by what the models do next?
15.50 The research community is like a hive mind, and you never know where the next idea will come from
16.40 Often the next thing comes from a couple of students at a university
16.58 How Ian Goodfellow created the first GAN
17.35 Are the older tribes described by Pedro Domingos (analogisers, evolutionists, Bayesians…) now obsolete?
18.15 We should cultivate different approaches because you never know where they might lead
19.15 Symbolic AI (aka Good Old Fashioned AI, or GOFAI) is still alive and kicking
19.40 AlphaGo combined deep learning and GOFAI
21.00 Doug Lennart is still persevering with Cyc, a purely GOFAI approach
21.30 GOFAI models had no learning element. They can’t go beyond the humans whose expertise they encapsulate
22.25 The now-famous move 37 in AlphaGo’s game two against Lee Sedol in 2016
23.40 Moravec’s paradox. Easy things are hard, and hard things are easy
24.20 The combination of deep learning and symbolic AI has been long urged, and in fact is already happening
24.40 Will models always demand more and more compute?
25.10 The human brain has far more compute power than even our biggest systems today
25.45 Sparse, or MoE (Mixture of Experts) systems are quite efficient
26.00 We need more compute, better algorithms, and more efficiency
26.55 Dedicated AI chips will help a lot with efficiency
26.25 Cerebros claims that GPT-3 could be trained on a single chip
27.50 Models can increasingly be trained for general purposes and then tweaked for particular tasks
28.30 Some of the big new models are open access
29.00 What else should people learn about with regard to advanced AI?
29.20 Neural Radiance Fields (NERF) models
30.40 Flamingo and Gato
31.15 We have mostly discussed research in these episodes, rather than engineering

Welcome to episode 5 of the London Futurist podcast, with your co-hosts David Wood and Calum Chace.

We’re attempting something rather ambitious in episodes 5 and 6. We try to explain how today’s cutting edge artificial intelligence systems work, using language familiar to lay people, rather than people with maths or computer science degrees.

Understanding how Transformers and Generative Adversarial Networks (GANs) work means getting to grips with concepts like matrix transformations, vectors, and landscapes with 500 dimensions.

This is challenging stuff, but do persevere. These AI systems are already having a profound impact, and that impact will only grow. Even at the level of pure self-interest, it is often said that in the short term, AIs won’t take all the jobs, but people who understand AI will take the best jobs.

We are extremely fortunate to have as our guide for these episodes a brilliant AI researcher at DeepMind, Aleksa Gordić.

Note that Aleksa is speaking in personal capacity and is not representing DeepMind.

Aleksa's YouTube channel is https://www.youtube.com/c/TheAIEpiphany

00.03 An ambitious couple of episodes
01.22 Introducing Aleksa, a double rising star
02.15 Keeping it simple
02.50 Aleksa's current research, and previous work on Microsoft's HoloLens
03.40 Self-taught in AI. Not representing DeepMind
04.20 The narrative of the Big Bang in 2012, when Machine Learning started to work in AI.
05.15 What machine learning is
05.45 AlexNet. Bigger data sets and more powerful computers
06.40 Deep learning a subset of machine learning, and a re-branding of artificial neural networks
07.27 2017 and the arrival of Transformers
07.40 Attention is All You Need
08.16 Before this there were LSTMs, Long Short-Term Memories
08.40 Why Transformers beat LSTMs
09.58 Tokenisation. Splitting text into smaller units and mapping them onto higher dimension networks
10.30 3D space is defined by three numbers
10.55 Humans cannot envisage multi-dimensional spaces with hundreds of dimensions, but it's OK to imagine them as 3D spaces
11.55 Some dimensions of the word "princess"
12.30 Black boxes
13.05 People are trying to understand how machines handle the dimensions
13.50 "Man is to king as woman is to queen." Using mathematical operators on this kind of relationship
14.35 Not everything is explainable
14.45 Machines discover the relationships themselves
15.15 Supervised and self-supervised learning. Rewarding or penalising the machine for predicting labels
16.25 Vectors are best viewed as arrows in 3D space, although that is over-simplifying
17.20 For instance the relationship between "queen" and "woman" is a vector
17.50 Self-supervised systems do their own labelling
18.30 The labels and relationships have probability distributions
19.20 For instance, a princess is far more likely to wear a slipper than a dog
19.35 Large numbers of parameters
19.40 BERT, the original Transformer, had a hundred million or so parameters
20.04 Now it's in the hundreds of billions, or even trillions
20.24 A parameter is analogous to a synapse in the human brain
21.19 Synapses can have different weights
22.10 The more parameters, the lower the loss
22.35 Not just text, but images too, because images can also be represented as tokens
23.00 In late 2020 Google released the first vision Transformer
23.29 Dall-E and Midjourney are diffusion models, which have replaced GANs
24.15 What are GANs, or Generative Adversarial Networks?
24.45 Two types of model: Generators and Discriminators. The first tries to fool the second
26.20 Simple text can produce photorealistic images
27.10 Aleksa's YouTube videos are available at "The AI Epiphany"
27.40 Close

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

In this episode, co-hosts Calum Chace and David Wood explore a number of recent developments in AI - developments that are rapidly changing what counts as "state of the art" in AI.

00.05: Short recap of previous episodes
00.20: A couple of Geoff Hinton stories
02.27: Today's subject: the state of AI today
02.53: Search
03.35: Games
03.58: Translation
04.33: Maps
05.33: Making the world understandable. Increasingly
07.00: Transformers. Attention is all you need
08.00: Masked language models
08.18: GPT-2 and GPT-3
08.54: Parameters and synapses
10.15: Foundation models produce much of the content on the internet
10.40: Data is even more important than size
11.45: Brittleness and transfer learning
13.15: Do machines understand?
14.05: Human understanding and stochastic parrots
15.27: Chatbots
16.22: Tay embarrasses Microsoft
16.53: Blenderbot
17.19: Far from AGI. LaMDA and Blaise Lemoine
18.26: The value of anthropomorphising
19.53: Automation
20.25: Robotic Process Automation (RPA)
20.55: Drug discovery
21.45: New antibiotics. Discovering Halicin
23.50: AI drug discovery as practiced by Insilico, Exscientia and others
25.33: Eroom's Law
26.34: AlphaFold. How 200m proteins fold
28.30: Towards a complete model of the cell
29.19: Analysis
30.04: Air traffic controllers use only 10% of the data available to them
30.36: Transfer learning can mitigate the escalating demand for compute power
31.18: Next up: the short-term future of AI

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

For more about the podcast hosts, see https://calumchace.com/ and https://dw2blog.com/

In this episode, co-hosts Calum Chace and David Wood continue their review of progress in AI, taking up the story at the 2012 "Big Bang".

00.05: Introduction: exponential impact, big bangs, jolts, and jerks
00.45: What enabled the Big Bang
01.25: Moore's Law
02.05: Moore's Law has always evolved since its inception in 1965
03.08: Intel's tick tock becomes tic tac toe
03.49: GPUs - Graphic Processing Units
04.29: TPUs - Tensor Processing Units
04.46: Moore's Law is not dead or dying
05.10: 3D chips
05.32: Memristors
05.54: Neuromorphic chips
06.48: Quantum computing
08.18: The astonishing effect of exponential growth
09.08: We have seen this effect in computing already. The cost of an iPhone in the 1950s.
09.42: Exponential growth can't continue forever, but Moore's Law hasn't reached any theoretical limits
10.33: Reasons why Moore's Law might end: too small, too expensive, not worthwhile
11.20: Counter-arguments
12.01: "Plenty more room at the bottom"
12.56: Software and algorithms can help keep Moore's Law going
14.15: Using AI to improve chip design
14.40: Data is critical
15.00: ImageNet, Fei Fei Lee, Amazon Turk
16.10: AIs labelling data
16.35: The Big Bang
17.00: Jürgen Schmidhuber challenges the narrative
17.41: The Big Bang enabled AI to make money
18.24: 2015 and the Great Robot Freak-Out
18.43: Progress in many domains, especially natural language processing
19.44: Machine Learning and Deep Learning
20.25: Boiling the ocean vs the scientific method's hypothesis-driven approach
21.15: Deep Learning: levels
21.57: How Deep Learning systems recognise faces
22.48: Supervised, Unsupervised, and Reinforcement Learning
24.00: Variants, including Deep Reinforcement Learning and Self-Supervised Learning
24.30: Yann LeCun's camera metaphor for Deep Learning
26.05: Lack of transparency is a concern
27.45: Explainable AI. Is it achievable?
29.00: Other AI problems
29.17: Has another Big Bang taken place? Large Language Models like GPT-3
30.08: Few-shot learning and transfer learning
30.40: Escaping Uncanny Valley
31.50: Gato and partially general AI

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

For more about the podcast hosts, see https://calumchace.com/ and https://dw2blog.com/

AI is a subject that we will all benefit from understanding better. In this episode, co-hosts Calum Chace and David Wood review progress in AI from the Greeks to the 2012 "Big Bang".

00.05: A prediction
01.09: AI is likely to cause two singularities in this pivotal century - a jobless economy, and superintelligence
02.22: Counterpoint: it may require AGI to displace most people from the workforce. So only one singularity?
03.27: Jobs are nowhere near all that matters in humans
04.11: Are the "Three Cs jobs" safe? Those involving Creativity, Compassion, and Commonsense? Probably not.
05.15: 2012, the Big Bang in AI
05.48: AI now makes money. Google and Facebook ate Rupert Murdoch's lunch
06.30: AI might make the difference between military success and military failure. So there's a geopolitical race as well as a commercial race
07.18: Defining AI.
09.03: Intelligence vs Consciousness
10.15: Does the Turing Test test for Intelligence or Consciousness?
12.30: Can customer service agents pass the Turing Test?
13.07: Attributing consciousness by brain architecture or by behaviour
15.13: Creativity. Move 37 in game two of AlphaGo vs Lee Sedol, and Hassabis' three buckets of creativity
17.13: Music and art produced by AI as examples
19.05: History: Start with the Greeks, Hephaestus (Vulcan to the Romans) built automata, and Aristotle speculated about technological unemployment
19.58: AI has featured in science fiction from the beginning, eg Mary Shelley's Frankenstein, Samuel Butler's Erewhon, E.M. Forster's "The Machine Stops"
20.55: Post-WW2 developments. Conference in Paris in 1951 on "Computing machines and human thought". Norbert Weiner and cybernetics
22.48: The Dartmouth Conference
23.55: Perceptrons - very simple models of the human brain
25.13: Perceptrons debunked by Minsky and Papert, so Symbolic AI takes over
25.49: This debunking was a mistake. More data and better hardware overcomes the hurdles
27.20: Two AI winters, when research funding dries up 
28.07: David was taught maths at Cambridge by James Lighthill, author of the report which helped cause the first AI winter
28.58: The Japanese 5th generation computing project under-delivered in the 1980s. But it prompted an AI revival, and its ambitions have been realised by more recent advances
30.45: No more AI winters?

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

For more about the podcast hosts, see https://calumchace.com/ and https://dw2blog.com/