The Technium

The semantic web was an effort to mark up webpages with facts about their content. The dream was to enable computers, to understand the world by reading these webpages. We discuss why the effort failed, how it relates to symbolic AI/knowledge graphs, and how the vision could still be realized today.

Links/Resources:

• Introducing the Knowledge Graph
• https://twobithistory.org/2018/05/27/semantic-web.html
• Metacrap: Putting the torch to seven straw-men of the meta-utopia
• Whatever happened to the Semantic Web? https://news.ycombinator.com/item?id=16806657
• http://ai.stanford.edu/blog/introduction-to-knowledge-graphs/
• Aggregation Theory: https://stratechery.com/concept/aggregation-theory/

Chapters:

0:00 Intros

2:12 What is the Semantic Web?

3:59 The Early 2000's

8:32 The Idealism of the Semantic Web

11:30 How it worked

18:19 Roots in Symbolic AI

22:53 The OG Web 3.0

25:28 Why Did the Semantic Web Fail?

35:46 A Misalignment with the Medium

41:10 Crypto and Semantic Web Collide

43:25 Deep Learning and the Semantic Web Collide

47:09 Vestiges of the Semantic Web

49:44 Path to Intelligent Agents

55:03 Incentive structures

1:02:49 Token Curated Registries and the Semantic Web

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

TLA+ is a language for formally modeling and verifying the correctness of programs, especially concurrent and distributed ones. We discuss how it could be used to improve the practice of software engineering and what it would take for formal methods to gain more popularity.

Links/Resources:

• TLA+ Homepage https://lamport.azurewebsites.net/tla/tla.html
• https://learntla.com/introduction/
• https://lamport.azurewebsites.net/video/videos.html
• https://lamport.azurewebsites.net/tla/tutorial/contents.html
• https://lamport.azurewebsites.net/tla/industrial-use.html
• https://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-services-uses-formal-methods/fulltext

Chapters:

0:00 Intros

2:14 What is TLA+?

5:10 Bugs in Concurrent and Distributed Programming

12:58 What's it like to use?

19:09 Global State Machine

20:34 Testimonials of Industrial Use Cases

24:56 Why is it not more popular?

27:32 How do we currently design our programs?

37:16 Everyone is a distributed systems programmer now

41:29 Thick client app management

49:47 Can haz use for Security?

53:08 TLA+ is for the design, not implementation

59:35 TLA+ all the things

1:02:55 Formal methods in Crypto?

1:08:19 Intrigued with Formal Methods

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Transformers are a building block of Machine Learning systems that have seen great success recently at subsuming all other techniques.

We discuss at a high level, its attention mechanism and its multimodal properties, and the types of applications this can be put to use now and in the future.

Links/Resources:

• Introductions to transformers
  • https://www.machinecurve.com/index.php/2020/12/28/introduction-to-transformers-in-machine-learning/
  • https://deeplobe.ai/machine-learning-for-transformers-explained-with-language-translation/
  • https://www.youtube.com/watch?v=yGTUuEx3GkA
• Attention is All you Need https://arxiv.org/pdf/1706.03762.pdf
• Attention: https://distill.pub/2016/augmented-rnns/
• Transformers replacing CNNs https://becominghuman.ai/transformers-in-vision-e2e87b739feb
• AI models consolidating https://twitter.com/karpathy/status/1468370605229547522?ref_src=twsrc^tfw
• GTP implementation https://github.com/karpathy/minGPT/blob/master/mingpt/model.py
• Google introduces new arch to reduce cost of transformers https://analyticsindiamag.com/google-introduces-new-architecture-to-reduce-cost-of-transformers/
• LaMDA by Google https://gpt3demo.com/apps/lamda-google

Chapters:

0:00 Intros
1:57 What are Transformers?
4:59 How does it work at a high level?
9:27 Self Attention Mechanism
14:03 Input structure agnostic
16:59 Stack it high, pump it with data
25:28 More MultiModal Learning
27:22 The Narrow Waist
34:34 Transformers for Compilation
40:24 Specialized Hardware
43:52 Multimodal Applications
47:19 Generating Media as a Self-sustaining Entity
52:42 The Jobs this Destroys
58:02 Two machines need to talk to each other
1:04:42 A Young Lady's Primer
1:11:19 Try them out!

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

CORBA is an acronym that stands for common object request, broker architecture.  It provides interoperability among distributed objects regardless of whether they're remote or local, regardless of whether they're written in different languages or in different locations of the network. 

We discuss its influence on distributed systems, and where we are today with these large scale systems

Links/Resources:

Chapters:

0:37 Intros
2:20 What is CORBA?
4:08 90's Throwback
7:25 The Idea of Distributed Objects
9:55 How does it work?
13:16 The Promise of Interoperability
15:35 Influence on Computing Today
18:52 Why did CORBA fail?
24:39 Lots of People Missed the Web
30:09 Distributed Computing
36:06 The REST model of Distributed
40:35 RPC lives on today
47:13 Hard to get this right
52:08 Negotiating Interfaces from Scratch

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Unison is a programming language for distributed systems by leveraging immutable and content addressable code. 

We discuss the many implications of a language whose code is content addressable, from fast rename, easy caching, to feature flagging.

Links/Resources:

- Unison Lang https://www.unisonweb.org/
- Unison at StrangeLoop https://www.youtube.com/watch?v=gCWtkvDQ2ZI
- Unison and its Scala-based Runtime https://www.youtube.com/watch?v=knqlWboqf_U
- What color is your function? https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/
- Python Environment XKCD https://xkcd.com/1987/
  

Chapters:

0:48 Intros
2:32 What is Unison?
6:46 Advantages of Content Addressable Code
15:39 Realizing the Scale before it's Too Late
22:03 Controlled Refactoring Experience
25:54 Abilities or Algebraic Effects
31:08 Programming the Whole Network
34:57 Facilitating the Culture around Coding
50:37 A Library Ecosystem
1:00:50 Reproducibility and Old Computers
1:07:55 Hot Patch Your Server
1:10:37 The Road to Popularity

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Smalltalk was one of the first object oriented programming languages and a software authoring environment that was highly influential in the history of computing.

We discuss its designer Alan Kay, his vision of personal computing, and the lasting influence of Smalltalk despite its limited commercial adoption.

Links/Resources:

• Actor Model - https://www.brianstorti.com/the-actor-model/
  • https://dspace.mit.edu/handle/1721.1/6272
• Smalltalk Demo - https://www.youtube.com/watch?v=uknEhXyZgsg
• Smalltalk Build your own Debugger - https://youtu.be/NqKyHEJe9_w?t=355
• Early History of Smalltalk - http://worrydream.com/EarlyHistoryOfSmalltalk/
  • “A remarkable number of artists, scientists, philosophers are quite dull outside of their specialty (and one suspects within it as well). The first siren's song we need to be wary of is the one that promises a connection between an interesting pursuit and interesting thoughts. The music is not in the piano, and it is possible to graduate Juilliard without finding or feeling it.”
• Why was smalltalk not a commercial success? 
  • http://www.wirfs-brock.com/allen/posts/914
  • https://gbracha.blogspot.com/2020/05/bits-of-history-words-of-advice.html
• Lively and Caffine 
  • https://lively-kernel.org/

Chapters:

0:00 Intros

1:57 The Retro Future

2:34 Origin and Motivation of Smalltalk

7:20 Object Orientation is about message passing

11:51 Meta-circular inspiration

14:31 Smalltalk is an entire environment

18:59 Designed with the GUI in mind

24:35 The Music isn't in the Piano

27:21 Taking a bet on the future

31:42 Surprising things about Smalltalk

34:27 Learning to Program By Reading "Literature”

39:56 Did Smalltalk Achieve It's Goals?

44:43 The Failure of Enterprise Smalltalk

52:30 Even Good Ideas Have to Survive the Marketplace

55:23 Big Ideas Meet Reality

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Dynamicland is a communal computer where the computer is the room. It's a new dynamic, medium to think about systems with other people. 

We discuss taking computation out of the realm of abstraction and putting it in the realm of the physical world, it, and how it breaks down barriers to entry.

Chapters:
0:00 Intros
2:10 What is Dynamicland?
11:17 The Communal Aspect of Computing
18:30 Computing other than Symbol Manipulation
31:29 The Concrete System
38:43 Realtalk is Declarative
43:39 Leverage the Real World for Compute
46:21 Squint beyond the Art Project
49:07 Having your thoughts out in the open
53:55 The Libraries of the Future
1:01:25 Interactive decision making 
1:06:59 Throwaway Computing
1:19:46 Dynamicland for Games
1:23:08 A wish for the future

Links/Resources:
- Dynamicland website: https://dynamicland.org/
- Research Agenda: http://worrydream.com/cdg/ResearchAgenda-v0.19-poster.pdf
- Humane representation of thought: https://vimeo.com/115154289
- Seeing Spaces: http://worrydream.com/SeeingSpaces/
- Embodiment and Visualization: https://vimeo.com/114252897
- Notes about the experience
    - https://dynamicland.org/research-notes/social-dynamics-of-programming-together/
    - https://omar.website/posts/notes-from-dynamicland-geokit/
    - http://vitor.io/on-dynamicland**](http://vitor.io/on-dynamicland)
    - https://tashian.com/articles/dynamicland/
    - https://www.phenomenalworld.org/analysis/the-next-big-thing-is-a-room/
    - https://futureofcoding.org/notes/bret-victor/dynamicland.html

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. 

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

Non-Fungible Tokens (NFTs) are unique digital collectibles that are represented on a blockchain in the form of a token. NFTs make it possible to own digital media assets in the same way that you can own a digital currency asset, like Bitcoin.

We discuss various applications of NFTs outside of art, and its potential as a building block for future applications.

Links/Resources:

• Girl with House on Fire Meme. https://knowyourmeme.com/memes/disaster-girl
• Ethereum Name Service https://ens.domains/
• Party Bid to Fractionalize NFTs https://www.partybid.app/
• Flash Loans https://coincodecap.com/flash-loan
• Aavegotchi https://aavegotchi.com/
• Uniswap https://uniswap.org/
• 3LAU and NFTs https://decrypt.co/59788/musician-3lau-sells-album-in-nft-form-for-11-million
• Demand Curves https://corporatefinanceinstitute.com/resources/knowledge/economics/demand-curve/
• Davie Bowie on the relationship of media and the internet https://www.youtube.com/watch?v=FiK7s_0tGsg&t=636s
• Skeuomorph https://en.wikipedia.org/wiki/Skeuomorph
• ERC-998 Composable standard https://github.com/ethereum/EIPs/blob/master/EIPS/eip-998.md
• Neolastics https://neolastics.com/
• Latent Space https://stats.stackexchange.com/questions/442352/what-is-a-latent-space
• Jack Dorsey Tweet for $2.9 million https://v.cent.co/tweet/20
• Schelling Point https://medium.com/@willemvandenbergh_85885/on-schelling-points-network-effects-and-lindy-inherent-properties-of-communication-c4eb69b55c60
• Invoice Refactoring https://en.wikipedia.org/wiki/Factoring_(finance)
• Accruing Value in a Value Chain https://stratechery.com/2015/netflix-and-the-conservation-of-attractive-profits/
• The Technium NFT! https://opensea.io/assets/matic/0x2953399124f0cbb46d2cbacd8a89cf0599974963/73071695664951725756103887812533794393262849524101052862455651661423314993153

Chapters:

0:14 Intros
2:00 What are Non-fungible Tokens?
7:21 Ethereum Name Service domains are NFTs
20:05 "Non-fungible" only notable in the digital world
22:54 NFTs are Digital Native Objects
26:21 Fractionalizing NFTs
31:46 NFTs as Collateral
36:02 NFT as Social Proof
42:15 NFT as meme-ified finance
44:36 Permissionless Composability of NFTs
51:15 NFTs for Community Building
55:41 The Curator Economy
1:00:29 Creator Feedback Loop with Fans
1:07:15 Permissionless Remixing
1:10:28 We're still making Skeuomorphs
1:13:43 Content Agnostic and Responsive
1:20:37 Owning the Latent Space
1:24:09 Financialization of Memes
1:30:50 NFTs as boring assets
1:42:27 NFTs Scaled Up and Down
1:49:50 Yet Another Building Block

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

IPFS is a distributed storage network. The content is accessible through peers located anywhere in the world that might relay information store it or both, and IPFS finds data by its content address rather than its locations. 

We discuss the main principles behind IPFS, the current use cases, and how it changes the basic unit economics of some businesses, as well as its interplanetary future.

Chapters:

00:00 Intros
02:25 What is IPFS?
13:58 Three Principles of IPFS
15:01 Content-addressable URIs
18:44 Content Linking in a DAG
21:27 Distributed Hash Table for Discovery
22:48 Pinning Content
30:58 Censorship-resistance
36:38 Used for NFTs
39:04 Used for Video and Music Streaming
42:56 Use for Package Manager
49:14 Use for Machine Learning
54:38 Interplanetary Linked Data
56:37 A Key Building Block
59:22 As a Public Good
01:06:36 Developers Tools on top of IPFS
01:10:07 Shifting Operational Burden
01:18:15 The Interplanetary Future

Links/Resources:

• Content Addressing https://simpleaswater.com/ipfs-cids/
• Linked Data https://ontola.io/what-is-linked-data/
• Distributed Hash Tables https://www.cs.cmu.edu/~dga/15-744/S07/lectures/16-dht.pdf
• Napster, Kaaza, Gnutella https://www.slideshare.net/uschmidt/peertopeer-systems/20-Comparison_Napster_Gnutella_KaZaAType_of
• Juan Benet of IPFS https://research.protocol.ai/authors/juan-benet/
• ProtoSchool https://proto.school/
• Marc Andressen’s Blog Archive https://pmarchive.com/
• Left Pad Debacle https://www.davidhaney.io/npm-left-pad-have-we-forgotten-how-to-program/
• NPM as a private company https://www.youtube.com/watch?v=MO8hZlgK5zc&t=46s
• Transfer Learning https://builtin.com/data-science/transfer-learning
• Deno Programming language https://deno.land/
• IPLD https://ipld.io/
• Jack Dorsey Regrets Shutting down API https://www.revyuh.com/news/software/developers/twitters-founder-admits-that-shutting-down-the-api-was-worst-thing-we-did-it-affected-users-and-developers/
• Datomic https://www.datomic.com/

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

End-user programming is a vision of computing that allows people who are not professional software developers to program their own computers.

We discuss some of our favorite end-user programming environments, the qualities that make an end-user programming environment successful and what the world would look like if everyone could program.

Chapters:
0:00: Intros
3:15: What is end-user programming?
6:28: Hypercard: An Early Pioneer
15:22: Quality #1 : Embodiment
17:52: Quality #2: Liveness
19:51: Quality #3: In-Place Toolchain
23:10: Elitism: Programmers vs. Script Kiddies
26:37: Yahoo Pipes: A Web 2.0 Programming Environment
32:25: What makes an environment successful?
40:28: Twitter, Hashtags, and "Situated Software”
47:36: Is programming just inherently hard?
1:04:26: Modern EUP Environments
1:10:39: The World If Everyone Could Program
1:15:01: Programming in the Metaverse

Links/Resources:
[0] https://www.inkandswitch.com/end-user-programming/]
[1] https://www.robinsloan.com/notes/home-cooked-app/
[2] https://www.johndcook.com/blog/2011/07/21/software-exoskeletons/
[3] https://www.gwern.net/docs/technology/2004-03-30-shirky-situatedsoftware.html

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. 

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

[WEBSITE](https://technium.transistor.fm/)
[SPOTIFY](https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7)