In the 4th century BC, the Greek philosopher Plato theorised that humans do not perceive the world as it really is. All we can see is shadows on a wall.

In 2003, the Swedish philosopher Nick Bostrom published a paper which formalised an argument to prove Plato was right. The paper argued that one of the following three statements is true:
1. We will go extinct fairly soon
2. Advanced civilisations don’t produce simulations containing entities which think they are naturally-occurring sentient intelligences. (This could be because it is impossible.)
3. We are in a simulation.

The reason for this is that if it is possible, and civilisations can become advanced without exploding, then there will be vast numbers of simulations, and it is vanishingly unlikely that any randomly selected civilisation (like us) is a naturally-occurring one.

Some people find this argument pretty convincing. As we will hear later, some of us have added twists to the argument. But some people go even further, and speculate about how we might bust out of the simulation.

One such person is our friend and our guest in this episode, Roman Yampolskiy, Professor of Computer Science at the University of Louisville.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Further reading:

"How to Hack the Simulation" by Roman Yampolskiy: https://www.researchgate.net/publication/364811408_How_to_Hack_the_Simulation

"The Simulation Argument" by Nick Bostrom: https://www.simulation-argument.com/

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

This episode discusses progress at Insilico Medicine, the AI drug development company founded by our guest, longevity pioneer Alex Zhavoronkov.

1.20 In Feb 2022, Insilico got an IPF drug into phase 1 clinical trials: a first for a wholly AI-developed drug
1.50  Insilico is now well-funded; its software is widely used in the pharma industry
2.30 How drug development works. First you create a hypothesis about what causes a disease
4.00 Pandaomics is Insilico’s software to generate hypotheses. It combines 20+ AI models, and huge public data repositories
6.00 This first phase is usually done in academia. It usually costs $ billions to develop a hypothesis. 95% of them fail
6.50 The second phase is developing a molecule which might treat the disease
7.15 This is the job of Insilico’s Chemistry 42 platform
7.30 The classical approach is to test thousands of molecules to see if they bind to the target protein
7.50 AI, by contrast, is able to "imagine" a novel molecule which might bind to it
8.00 You then test 10-15 molecules which have the desired characteristics
8.20 This is done with a variety of genetic algorithms, Generative Adversarial Networks (GANs), and some Transformer networks
8.35 Insilico has a “zoo” of 40 validated models
10.40 Given the ten-fold improvement, why hasn’t the whole drug industry adopted this process?
10.50 They do all have AI groups and they are trying to change, but they are huge companies, and it takes time
11.50 Is it better to invent new molecules, or re-purpose old drugs, which are already known to be safe in humans?
13.00 You can’t gain IP with re-purposed drugs: either somebody else “owns” them, or they are already generic
15.00 The IPF drug was identified during aging research, using aging clocks, and a deep neural net trained on longitudinal data
17.10 The third phase is where Insilico’s other platform, InClinico, comes into play
17.35 InClinico predicts the results of phase 2 (clinical efficacy) trials
18.15 InClinico is trained on massive data sets about previous trials
19.40 InClinico is actually Insilico’s oldest system. Its value has only been ascertained now that some drugs have made it all the way through the pipeline
22.05 A major pharma company asked Insilico to predict the outcome of ten of its trials
22.30 Nine of these ten trials were predicted correctly
23.00 But the company decided that adopting this methodology would be too much of an upheaval; it was unwilling to rely on outsiders so heavily
24.15 Hedge funds and banks have no such qualms
24.25 Insilico is doing pilots for their investments in biotech startups
26.30 Alex is from Latvia originally, studied in Canada, started his career in the US, but Insilico was established in Hong Kong. Why?
27.00 Chinese CROs, Contract Research Organisations, enable you to do research without having your own wetlab 
28.00 Like Apple, Insilico designs in the US and does operations in China. You can also do clinical studies there
28.45 They needed their own people inside those CROs, so had to be co-located
29.10 Hong Kong still has gre

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

Real Talk About Marketing

An Acxiom podcast where we discuss marketing made better, bringing you real...

Listen on: Apple Podcasts   Spotify

Co-hosts Calum and David dig deep into aspects of David's recent new book "The Singularity Principles". Calum (CC) says he is, in part, unconvinced. David  (DW) agrees that the projects he recommends are hard, but suggests some practical ways forward.

0.25 The technological singularity may be nearer than we think
1.10 Confusions about the singularity
1.35 “Taking back control of the singularity”
2.40 The “Singularity Shadow”: over-confident predictions which repulse people
3.30 The over-confidence includes predictions of timescale…
4.00 … and outcomes
4.45 The Singularity as the Rapture of the Nerds?
5.20 The Singularity is not a religion …
5.40 .. although if positive, it will confer almost godlike powers
6.35 Much discussion of the Singularity is dystopian, but there could be enormous benefits, including…
7.15 Digital twins for cells and whole bodies, and super longevity
7.30 A new enlightenment
7.50 Nuclear fusion
8.10 Humanity’s superpower is intelligence
8.30 Amplifying our intelligence should increase our power
9.50 DW’s timeline: 50% chance of AGI by 2050, 10% by 2030
10.10 The timeline is contingent on human actions
10.40 Even if AGI isn’t coming until 2070, we should be working on AI alignment today
11.10 AI Impact’s survey of all contributors to NeurIPS
11.35 Median view: 50% chance of AGI in 2059, and many were pessimistic
12.15 This discussion can’t be left to AI researchers
12.40 A bad beta version might be our last invention
13.00 A few hundred people are now working on AI alignment, and tens of thousands on advancing AI
13.35 The growth of the AI research population is still faster
13.40 CC: Three routes to a positive outcome
13.55 1. Luck. The world turns out to be configured in our favour
14.30 2. Mathematical approaches to AI alignment succeed
14.45 We either align AIs forever, or manage to control them. This is very hard
14.55 3. We merge with the superintelligent machines
15.40 Uploading is a huge engineering challenge
15.55 Philosophical issues raised by uploading: is the self retained?
16.10 DW: routes 2 and 3 are too binary. A fourth route is solving morality
18.15 Individual humans will be augmented, indeed we already are
18.55 But augmented humans won’t necessarily be benign
19.30 DW: We have to solve beneficence
20.00 CC: We can’t hope to solve our moral debates before AGI arrives
20.20 In which case we are relying on route 1 – luck
20.30 DW: Progress in philosophy *is* possible, and must be accelerated
21.15 The Universal Declaration of Human Rights shows that generalised moral principles can be agreed
22.25 CC: That sounds impossible. The UDHR is very broad and often ignored
23.05 Solving morality is even harder than the MIRI project, and reinforces the idea that route 3 is our best hope
23.50 It’s not unreasonable to hope that wisdom correlates with intelligence
24.00 DW: We can proceed step by step, starting with progress on facial recognition, autonomous weapons, and such intermediate questions
25.10 CC: We are so far from solving moral questions. Americans can’t even agree if a coup against their democracy was a bad thing
25.40 DW: We have to make progress, and quickly. AI might help us.
26.50 The essence of transhumanism is that we can use technology to improve ourselves
27.20 CC: If you had a magic wand, your first wish should probably be to make all humans see each other as members of the same tribe
27.50 Is

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

How likely is it that, by 2030, someone will build artificial general intelligence (AGI)?

Ross Nordby is an AI researcher who has shortened his AGI timelines: he has changed his mind about when AGI might be expected to exist. He recently published an article on the LessWrong community discussion site, giving his argument in favour of shortening these timelines. He now identifies 2030 as the date by which it is 50% likely that AGI will exist. In this episode, we ask Ross questions about his argument, and consider some of the implications that arise.

Article by Ross: https://www.lesswrong.com/posts/K4urTDkBbtNuLivJx/why-i-think-strong-general-ai-is-coming-soon

Effective Altruism Long-Term Future Fund: https://funds.effectivealtruism.org/funds/far-future

MIRI (Machine Intelligence Research Institution): https://intelligence.org/

00.57 Ross’ background: real-time graphics, mostly in video games
02.10 Increased familiarity with AI made him reconsider his AGI timeline
02.37 He submitted a grant request to the Effective Altruism Long-Term Future Fund to move into AI safety work
03.50 What Ross was researching: can we make an AI intrinsically interpretable?
04.25 The AGI Ross is interested in is defined by capability, regardless of consciousness or sentience
04.55 An AI that is itself "goalless" might be put to uses with destructive side-effects
06.10 The leading AI research groups are still DeepMind and OpenAI
06.43 Other groups, like Anthropic, are more interested in alignment
07.22 If you can align an AI to any goal at all, that is progress: it indicates you have some control
08.00 Is this not all abstract and theoretical - a distraction from more pressing problems?
08.30 There are other serious problems, like pandemics and global warming, but we have to solve them all
08.45 Globally, only around 300 people are focused on AI alignment: not enough
10.05 AGI might well be less than three decades away
10.50 AlphaGo surprised the community, which was expecting Go to be winnable 10-15 years later
11.10 Then AlphaGo was surpassed by systems like AlphaZero and MuZero, which were actually simpler, and more flexible
11.20 AlphaTensor frames matrix multiplication as a game, and becomes superhuman at it
11.40 In 2018, the Transformer paper was published, but no-one forecast GPT-3’s capabilities
12.00 This year, Minerva (similar to GPT-3) got 50% correct on the math dataset: high school competition math problems
13.16 Illustrators now feel threatened by systems like Dall-E, Stable Diffusion, etc
13.30 The conclusion is that intelligence is easier to simulate than we thought
13.40 But these systems also do stupid things. They are brittle
18.00 But we could use transformers more intelligently
19.20 They turn out to be able to write code, and to explain jokes, and do maths reasoning
21:10 Google's Gopher AI
22.05 Machines don’t yet have internal models of the world, which we call common sense
24.00 But an early version of GPT-3 demonstrated the ability to model a human thought process alongside a machine’s
27.15 Ross’ current timeline is 50% probability of AGI by 2030, and 90+% by 2050
27:35 Counterarguments?
29.35 So what is to be done?
30.55 If convinced that AGI is coming soon, most lay people would probably demand that all AI research stops immediately. Which isn’t possible
31.40 Maybe publicity would be good in order to generate resources for AI alignment. And to avoid a backlash against

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

Why do human brains consume much less power than artificial neural networks? Simon Thorpe, Research Director of CNRS, explains his view that the key to artificial general intelligence is a "terabrain" that copies from human brains the sparse-firing networks with spiking neurons.

00.11 Recapping "the AI paradox"
00.28 The nervousness of CTOs regarding AI
00.43 Introducing Simon
01.43 45 years since Oxford, working out how the brain does amazing things
02.45 Brain visual perception as feed-forward vs. feedback
03.40 The ideas behind the system that performed so well in the 2012 ImageNet challenge
04.20 The role of prompts to alter perception
05.30 Drawbacks of human perceptual expectations
06.05 The video of a gorilla on the basketball court
06.50 Conjuring tricks and distractions
07.10 Energy consumption: human neurons vs. artificial neurons
07.26 The standard model would need 500 petaflops
08.40 Exaflop computing has just arrived
08.50 30 MW vs. 20 W (less than a lightbulb)
09.34 Companies working on low-power computing systems
09.48 Power requirements for edge computing
10.10 The need for 86,000 neuromorphic chips?
10.25 Dense activation of neurons vs. sparse activation
10.58 Real brains are event driven
11.16 Real neurons send spikes not floating point numbers
11.55 SpikeNET by Arnaud Delorme
12.50 Why are sparse networks studied so little?
14.40 A recent debate with Yann LeCun of Facebook and Bill Dally of Nvidia
15.40 One spike can contain many bits of information
16.24 Revisiting an experiment with eels from 1927 (Lord Edgar Adrian)
17.06 Biology just needs one spike
17.50 Chips moved from floating point to fixed point
19.25 Other mentions of sparse systems - MoE (Mixture of Experts)
19.50 Sparse systems are easier to interpret
20.30 Advocacy for "grandmother cells"
21.23 Chicks that imprinted on yellow boots
22.35 A semantic web in the 1960s
22.50 The Mozart cell
23.02 An expert system implemented in a neural network with spiking neurons
23.14 Power consumption reduced by a factor of one million
23.40 Experimental progress
23.53 Dedicated silicon: Spikenet Technology, acquired by BrainChip
24.18 The Terabrain Project, using standard off-the-shelf hardware
24.40 Impressive recent simulations on GPUs and on a MacBook Pro
26.26 A homegrown learning rule
26.44 Experiments with "frozen noise"
27.28 Anticipating emulating an entire human brain on a Mac Studio M1 Ultra
28.25 The likely impact of these ideas
29.00 This software will be given away
29.17 Anticipating "local learning" without the results being sent to Big Tech
30.40 GPT-3 could run on your phone next year
31.12 Our interview next year might be, not with Simon, but with his Terabrain
31.22 Our phones know us better than our spouses do

Simon's academic page: https://cerco.cnrs.fr/page-perso-simon-thorpe/

Simon's personal blog: https://simonthorpesideas.blogspot.com/

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

This episode features Daniel Hulme, founder of Satalia and chief AI officer at WPP. What is AI good at today? And how can organisations increase the likelihood of deploying AI successfully?

02.55 What is AI good at today?
03.25 Deep learning isn’t yet being widely used in companies. Executives are wary of self-adapting systems
04.15 Six categories of AI deployment today
04.20 1. Automation. Using “if … then …” statements
04.50 2. Generative AI, like Dall-E
05.15 3. Humanisation, like DeepFake technology and natural language models
05.40 4. Machine learning to extract insights from data – finding correlations that humans could not
06.05 5. Complex decision making, aka operations research, or optimisation. “Companies don’t have ML problems, they have decision problems”
06.25 6. Augmenting humans physically or cognitively
06.50 Aren’t the tech giants using true AI systems in their operations?
07.15 A/B testing is a simple form of adaptation. Google A/B tested the colours of their logo
08 .00 Complex adaptive systems with many moving parts are much riskier. If they go wrong, huge damage can occur
08.30 CTOs demand consistency from operational systems, and can’t tolerate the mistakes that are essential to learning
09.25 Can’t the mistakes be made in simulated environments?
10.20 Elon Musk says simulating the world is not how to develop self-driving cars
10.45 Companies undergoing digital transformations are building ERPs, which are “glorified databases”
11.20 The idea is to develop digital twins, which enable them to ask “what if…” questions
11.30 The coming confluence of three digital twins: workflow, workforce, and administrative processes
12.18 Why don’t supermarkets offer digital twins to their customers? They’re coming
14.55 People often think that creating a data lake and adding a system like Tableau on top is deploying AI
15.15 Even if you give humans better insights they often don’t make better decisions
15.20 Data scientists are not equipped to address opportunities in all 6 of the categories listed earlier
15.40 Companies should start by identifying and then prioritising the frictions in their organisations
16.10 Some companies are taking on “tech debt” which they will have to unwind in five years
16.25 Why aren’t large process industry companies boasting about massive revenue improvements or cost savings?
17.00 To make those decisions you need the right data, and top optimisation skills. That’s unusual
17.55 Companies ask for “quick wins” but that is an oxymoron
18.10 We do see project ROIs of 200%, but most projects fail due to under-investment, or mis-understandings
19.00 Don’t start by just collecting data. The example of a low-cost airline which collected data about everything except rivals’ pricing
20.15 Humans usually do know where the signals are
22.25 Some of Daniel’s favourite AI projects
23.00 Tesco’s last-mile delivery system, which saves 20m delivery miles a year
24.00 Solving PwC’s consultant allocation problem radically improved many lives
25.10 In the next decade there will be a move away from pure ML towards ML+ optimisation
26.35 How these systems have been applied to Satalia
28.10 Daniel has thought a lot about how AI can enable companies to be very adaptable, and allocate decisions well
29.00 Satalia staff used to make recommendations for their own salaries, and their colleagues would make AI-weighted votes
29.30 The goal is to scale this approach not just

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

Calum and David reflect on their involvement in two recent conferences, one in Riyadh, and one in Dublin. Each conference highlighted a potential disruption in a major industry: a country with large ambitions in the AI space, and a new foundation in the longevity space.

00.00 A tale of two cities, two conferences, two industries
00.44 First, the 2nd Saudi Global AI Conference
01.03 Vision 2030
01.11 Saudi has always been a coalition between the fundamentalist Wahhabis and the Royal Family
01.38 The King chooses reform in the wake of 9/11
02.07 Mohamed bin Salman appointed Crown Prince, who embarks on reform
02.28 The partial liberation of women, and the fundamentalists side-lined
03.10 The “Sheikhdown” in 2017
03.49 The Khashoggi affair and the Yemen war lead to Saudi being shunned
04.26 The West is missing what’s going on in Saudi
05.00 Lifting the Saudi economy’s reliance on petrochemicals
05.27 AI is central to Vision 2030
06.00 Can Saudi become one of the world’s top 10 or 15 AI countries?
06.20 The AI duopoly between the US and China is so strong, this isn’t as hard as you might think
06.55 Saudi’s advantages
07.22 Saudi’s disadvantages
07.54 The goal is not implausible
08.10 The short-term goals of the conference. A forum for discussions, deals, and trying to open the world’s eyes
09.45 Saudi is arguably on the way to becoming another Dubai. Continuation and success are not inevitable, but it is encouraging
11.00 Fastest-growth country in the G20, with an oil bonanza
11.25 The proposed brand-new city of Neom with The Line, a futuristic environment
13.07 The second conference: the Longevity Summit in Dublin
13.48 A new foundation announced
14.05 Reports updating on progress in longevity research around the world
14.20 A dozen were new and surprising. Four examples…
14.50 1. Bats. A speaker from Dublin discussed why they live so long – 40 years – and what we can learn from that
15.55 2. Parabiosis on steroids. Linking the blood flow of two animals suggests there are aging elements in our blood which can be removed
17.50 3. Using AI to develop drugs. Companies like Exscientia and Insilico. Cortex Discovery is a smaller, perhaps more nimble player
19.40 4. Hevolution, a new longevity fund backed with up to $1bn of Saudi money per year for 20 years
22.05 As Aubrey de Grey has long said, we need engineering as much as research
22.40 Aubrey thinks aging should be tackled by undoing cell damage rather than changing the human metabolism
24.00 Three phases of his career. Methuselah. SENS. New foundation
25.00 Let’s avoid cancer, heart disease and dementias by continually reversing aging damage
26.00 He is always itchy to explore new areas. This led to a power struggle within SENS, which he lost
27.00 What should previous SENS donors do now?
27.15 The rich crypto investors who have provided large amounts to SENS are backing the new foundation
28.30 One of the new foundation’s investment areas will be parabiosis
28.55 Cryonics will be another investment area
29.15 Lobbying legislators will be another
29.50 Robust Mouse Rejuvenation will be the initial priority
30.50 Pets may be the animal models whose rejuvenation breaks humanity’s “trance of death”
31.05 David has been appointed a director the new foundation
31.50 The other directors
33.05 An exciting future

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

This episode continues our discussion with AI researcher Aleksa Gordić from DeepMind on understanding today’s most advanced AI systems.

00.07 This episode builds on Episode 5
01.05 We start with GANs – Generative Adversarial Networks
01.33 Solving the problem of stability, with higher resolution
03.24 GANs are notoriously hard to train. They suffer from mode collapse
03.45 Worse, the model might not learn anything, and the result is pure noise
03.55 DC GANs introduced convolutional layers to stabilise them and enable higher resolution
04.37 The technique of outpainting
05.55 Generating text as well as images, and producing stories
06.14 AI Dungeon
06.28 From GANs to Diffusion models
06.48 DDPM (De-noising diffusion probabilistic models) does for diffusion models what DC GANs did for GANs
07.20 They are more stable, and don’t suffer from mode collapse
07.30 They do have downsides. They are much more computation intensive
08.24 What does the word diffusion mean in this context?
08.40 It’s adopted from physics. It peels noise away from the image
09.17 Isn’t that rewinding entropy?
09.45 One application is making a photo taken in 1830 look like one taken yesterday
09.58 Semantic Segmentation Masks convert bands of flat colour into realistic images of sky, earth, sea, etc
10.35 Bounding boxes generate objects of a specified class from tiny inputs
11.00 The images are not taken from previously seen images on the internet, but invented from scratch
11.40 The model saw a lot of images during training, but during the creation process it does not refer back to them
12.40 Failures are eliminated by amendments, as always with models like this
12.55 Scott Alexander blogged about models producing images with wrong relationships, and how this was fixed within 3 months
13.30 The failure modes get harder to find as the obvious ones are eliminated
13.45 Even with 175 billion parameters, GPT-3 struggled to handle three digits in computation
15.18 Are you often surprised by what the models do next?
15.50 The research community is like a hive mind, and you never know where the next idea will come from
16.40 Often the next thing comes from a couple of students at a university
16.58 How Ian Goodfellow created the first GAN
17.35 Are the older tribes described by Pedro Domingos (analogisers, evolutionists, Bayesians…) now obsolete?
18.15 We should cultivate different approaches because you never know where they might lead
19.15 Symbolic AI (aka Good Old Fashioned AI, or GOFAI) is still alive and kicking
19.40 AlphaGo combined deep learning and GOFAI
21.00 Doug Lennart is still persevering with Cyc, a purely GOFAI approach
21.30 GOFAI models had no learning element. They can’t go beyond the humans whose expertise they encapsulate
22.25 The now-famous move 37 in AlphaGo’s game two against Lee Sedol in 2016
23.40 Moravec’s paradox. Easy things are hard, and hard things are easy
24.20 The combination of deep learning and symbolic AI has been long urged, and in fact is already happening
24.40 Will models always demand more and more compute?
25.10 The human brain has far more compute power than even our biggest systems today
25.45 Sparse, or MoE (Mixture of Experts) systems are quite efficient
26.00 We need more compute, better algorithms, and more efficiency
26.55 Dedicated AI chips will help a lot with efficiency
26.25 Cerebros claims that GPT-3 could be trained on a single chip
27

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

Welcome to episode 5 of the London Futurist podcast, with your co-hosts David Wood and Calum Chace.

We’re attempting something rather ambitious in episodes 5 and 6. We try to explain how today’s cutting edge artificial intelligence systems work, using language familiar to lay people, rather than people with maths or computer science degrees.

Understanding how Transformers and Generative Adversarial Networks (GANs) work means getting to grips with concepts like matrix transformations, vectors, and landscapes with 500 dimensions.

This is challenging stuff, but do persevere. These AI systems are already having a profound impact, and that impact will only grow. Even at the level of pure self-interest, it is often said that in the short term, AIs won’t take all the jobs, but people who understand AI will take the best jobs.

We are extremely fortunate to have as our guide for these episodes a brilliant AI researcher at DeepMind, Aleksa Gordić.

Note that Aleksa is speaking in personal capacity and is not representing DeepMind.

Aleksa's YouTube channel is https://www.youtube.com/c/TheAIEpiphany

00.03 An ambitious couple of episodes
01.22 Introducing Aleksa, a double rising star
02.15 Keeping it simple
02.50 Aleksa's current research, and previous work on Microsoft's HoloLens
03.40 Self-taught in AI. Not representing DeepMind
04.20 The narrative of the Big Bang in 2012, when Machine Learning started to work in AI.
05.15 What machine learning is
05.45 AlexNet. Bigger data sets and more powerful computers
06.40 Deep learning a subset of machine learning, and a re-branding of artificial neural networks
07.27 2017 and the arrival of Transformers
07.40 Attention is All You Need
08.16 Before this there were LSTMs, Long Short-Term Memories
08.40 Why Transformers beat LSTMs
09.58 Tokenisation. Splitting text into smaller units and mapping them onto higher dimension networks
10.30 3D space is defined by three numbers
10.55 Humans cannot envisage multi-dimensional spaces with hundreds of dimensions, but it's OK to imagine them as 3D spaces
11.55 Some dimensions of the word "princess"
12.30 Black boxes
13.05 People are trying to understand how machines handle the dimensions
13.50 "Man is to king as woman is to queen." Using mathematical operators on this kind of relationship
14.35 Not everything is explainable
14.45 Machines discover the relationships themselves
15.15 Supervised and self-supervised learning. Rewarding or penalising the machine for predicting labels
16.25 Vectors are best viewed as arrows in 3D space, although that is over-simplifying
17.20 For instance the relationship between "queen" and "woman" is a vector
17.50 Self-supervised systems do their own labelling
18.30 The labels and relationships have probability distributions
19.20 For instance, a princess is far more likely to wear a slipper than a dog
19.35 Large numbers of parameters
19.40 BERT, the original Transformer, had a hundred million or so parameters
20.04 Now it's in the hundreds of billions, or even trillions
20.24 A parameter is analogous to a synapse in the human brain
21.19 Synapses can have different weights
22.10 The more parameters, the lower the loss
22.35 Not just text, but images too, because images can also be represented as tokens
23.00 In late 2020 Google released the first vision Transformer
23.29 Dall-E and Midjourney are diffusion models, which have replaced GANs
24

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify

In this episode, co-hosts Calum Chace and David Wood explore a number of recent developments in AI - developments that are rapidly changing what counts as "state of the art" in AI.

00.05: Short recap of previous episodes
00.20: A couple of Geoff Hinton stories
02.27: Today's subject: the state of AI today
02.53: Search
03.35: Games
03.58: Translation
04.33: Maps
05.33: Making the world understandable. Increasingly
07.00: Transformers. Attention is all you need
08.00: Masked language models
08.18: GPT-2 and GPT-3
08.54: Parameters and synapses
10.15: Foundation models produce much of the content on the internet
10.40: Data is even more important than size
11.45: Brittleness and transfer learning
13.15: Do machines understand?
14.05: Human understanding and stochastic parrots
15.27: Chatbots
16.22: Tay embarrasses Microsoft
16.53: Blenderbot
17.19: Far from AGI. LaMDA and Blaise Lemoine
18.26: The value of anthropomorphising
19.53: Automation
20.25: Robotic Process Automation (RPA)
20.55: Drug discovery
21.45: New antibiotics. Discovering Halicin
23.50: AI drug discovery as practiced by Insilico, Exscientia and others
25.33: Eroom's Law
26.34: AlphaFold. How 200m proteins fold
28.30: Towards a complete model of the cell
29.19: Analysis
30.04: Air traffic controllers use only 10% of the data available to them
30.36: Transfer learning can mitigate the escalating demand for compute power
31.18: Next up: the short-term future of AI

Audio engineering by Alexander Chace.

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

For more about the podcast hosts, see https://calumchace.com/ and https://dw2blog.com/

Promoguy Talk Pills
Agency in Amsterdam dives into topics like Tech, AI, digital marketing, and more drama...

Listen on: Apple Podcasts   Spotify