The Technium

LLMs for software development can go way beyond Github Copilot. In this episode, we talk about how these models could change the workflow for existing developers, but more importantly how they could change the very idea of what software is.

Links/Resources:

https://simonwillison.net/2023/Mar/11/llama/

https://github.com/RootbeerComputer/backend-GPT

https://lukebechtel.com/blog/gpt4-generating-code

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. 

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Dependent types are a more expressive type system in programming languages used to catch a larger class of errors at compile time. What are would be typically assertions at runtime can now be caught at compile time.

Show notes:

• Proposition as Types
• Parse, Don’t Validation
• “Scala vs Idris: Dependent types, now and in the future”

Resources:

• http://www.e-pig.org/downloads/ydtm.pdf
• https://gist.github.com/Hirrolot/27e6b02a051df333811a23b97c375196
• Proof Theory Impressionism: Blurring the Curry-Howard Line
• Type Systems - The Good, Bad and Ugly
• Dependent types for practical use
• Idris: Practical Dependent Types with Practical Examples
• Making Illegal States unrepresentable
• Can types replace validation
  https://www.cs.ox.ac.uk/ralf.hinze/WG2.8/26/slides/xavier.pdf

Activity Pub is a decentralized social networking protocol. It allows different web applications to interact so that their users can share information, even if the websites or applications are running different software. 

00:00 S04E07 Activitypub
02:13 What is Activity Pub?
15:02 Interoperability
33:00 A New Kind of Social
43:53 Providing Distribution
53:03 The Use Cases
01:07:41 Ideology
01:14:25 The Wild West
01:24:07 Deconstructed Content
01:34:52 Taking a Position

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. 

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Links/Resources:

• https://tinysubversions.com/notes/reading-activitypub/
• https://flak.tedunangst.com/post/ActivityPub-as-it-has-been-understood
• https://web.archive.org/web/20201124231343/https://cjslep.com/c/blog/an-activitypub-philosophy
• https://overengineer.dev/blog/2018/02/01/activitypub-one-protocol-to-rule-them-all.html
• https://overengineer.dev/blog/2019/01/13/activitypub-final-thoughts-one-year-later.html
• https://www.jeremydormitzer.com/blog/what-is-activitypub.html
• https://tinysubversions.com/notes/decentralized-social-networks/
• https://kyefox.com/post/707900440336039936/activitypub-could-be-the-future
• https://ariadne.space/2019/01/07/activitypub-the-worse-is-better-approach-to-federated-social-networking/
• https://activitypub.rocks/
• https://overengineer.dev/blog/2019/01/13/activitypub-final-thoughts-one-year-later.html
• https://socialhub.activitypub.rocks/t/introduction-to-activitypub/508
• https://github.com/joyeusenoelle/GuideToMastodon
• https://victoria.dev/blog/why-pixelfed-wont-save-us-from-instagram/
• https://web.archive.org/web/20190713233100/https://blog.dereferenced.org/federation-what-flows-where-and-why
• https://web.archive.org/web/20190108060531/https://blog.dereferenced.org/activitypub-the-worse-is-better-approach-to-federated-social-networking
• https://www.eugenewei.com/blog/2021/9/29/and-you-will-know-us-by-the-company-we-keep
• https://twitter.com/AIMindFlow

APL is an array based programming language developed by Kenneth Iverson in the 1960s. Its central data type is the multi-dimensional array and hence it's very useful for workloads involving a lot of matrix math. APL predominantly uses symbols and it leverages consistent composability and execution rules to enable it as a notation as a tool for thought. 

Links/Resources:
* https://mathspp.com/blog/why-apl-is-a-language-worth-knowing
* https://www.quora.com/What-made-APL-programming-so-revolutionary?share=1
* https://tryapl.org/
* A Personal View of APL
* Notation as a Tool for Thought
* Heaviside Operator Calculus
* Conway’s Game of Life in APL
* Maxwell’s Equations: From 20 to 4
* Alan Kay’s answer to “What made APL so revolutionary?”
* Testimonies
    * https://news.ycombinator.com/item?id=27463149
    * https://news.ycombinator.com/item?id=27464005
    * https://news.ycombinator.com/item?id=27465512
* https://github.com/razetime/ngn-k-tutorial/blob/main/01-intro.md
00 - Objectives and Prerequisites | "Learn APL with Neural Networks"
https://dl.acm.org/doi/pdf/10.1145/3315454.3329960

Chapters:
[00:00:00] Intros
[00:03:36] What is APL?
[00:05:42] Yes, it uses weird symbols
[00:06:51] A language should change your thinking
[00:13:08] Notation as a tool of thought
[00:20:33] Connections to functional programming
[00:28:23] What it feels like to program in APL
[00:33:10] APL and IBM = big bucks
[00:35:20] The echos of APL in modern data science tools
[00:43:52] APL for big data pipelines
[00:48:56] APL for Machine learning
[00:54:59] APL for JSON manipulation
[01:03:18] APL as a spreadsheet language
[01:16:11] Superhuman programming: APL and LLMs
[01:26:59] Making APL more explainable
[01:38:18] Outro

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

io_uring is a new asynchronous, API to enable fast and efficient system calls in the Linux kernel.  It promises faster user-land programs that heavily use file system IO on Linux, for all applications, not just databases.  

00:00 S04 E05 io_uring
02:55 What is io_uring?
05:59 What's wrong with existing async interfaces?
17:25 Copying is slow
19:05 Two ring buffers
28:07 What would the effect for the end user be?
34:50 Different operating modes
37:27 C10K for Filesystem IO
43:04 Speed is simplicity
47:07 Deconstructed  databases
01:02:12 Effect systems
01:06:26 System functionality in programming language
01:10:55 OS as Actor Model
01:16:08 The Kernel is Userland

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. 

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: [https://technium.transistor.fm/](https://technium.transistor.fm/)
SPOTIFY: [https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7](https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7)
APPLE PODCASTS: [https://podcasts.apple.com/us/podcast/the-technium/id1608747545](https://podcasts.apple.com/us/podcast/the-technium/id1608747545)

Xanadu was the first hypertext project founded in 1960 by Ted Nelson. It aims to facilitate a type of media called hypermedia, which is non-sequential writing in which the reader can choose their own path through an electronic document.

Links/Resources:
http://www.rheingold.com/texts/tft/14.html
https://en.wikipedia.org/wiki/Project_Xanadu
https://mimix.io/en/blog/xanadu
https://sentido-labs.com/en/library/201904240732/Xanadu%20Hypertext%20Documents.html
https://www.notion.so/blog/ted-nelson
https://www.wired.com/1995/06/xanadu/
https://xanadu.com.au/ararat
https://www.youtube.com/watch?v=-_-5cGEU9S0
https://www.youtube.com/watch?v=hMKy52Intac
https://www.youtube.com/watch?v=1gPM3GqjMR4
https://www.youtube.com/watch?v=hGKbRcvIZT8
https://www.youtube.com/watch?v=qyzgoeeloJA
https://www.youtube.com/watch?v=_xYwgJW7T8o
https://jasoncrawford.org/the-lessons-of-xanadu
https://blockprotocol.org/
https://github.com/subconsciousnetwork/noosphere/blob/main/design/explainer.md
https://maggieappleton.com/xanadu-patterns
https://aaronzlewis.com/blog/2019/05/01/spreading-threading/
https://www.zombo.com/
https://stratechery.com/concept/aggregation-theory/
https://maggieappleton.com/tools-for-thought
https://cdixon.org/2015/01/31/come-for-the-tool-stay-for-the-network

Chapters:
[00:00:00] Intros
[00:03:22] What is Xanadu?
[00:15:49] Transclusion and Bidirectionality
[00:26:32] Versioning
[00:29:47] Vision divorced from implementation
[00:35:13] Baked in Payments
[00:46:15] Hypermedia as Envisioned
[00:56:20] Tiktok as Hypermedia
[01:01:52] Alternative business model for the web
[01:16:19] Failure to Launch
[01:26:15] Linearization as a forge
[01:31:51] Success of Xanadu's Vision
[01:37:04] Passing the torch

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

ChatGPT is a language model by OpenAI which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

00:00 S04E03 ChatGPT Part 2 
02:30 Expanding Humanity's Limits 
11:33 Tool-assisted speedrun of Science 
16:18 As a Civic Moderator 
24:32 Social Upheaval 
28:40 Platform Risk 
49:51 Billion Dollar Solo Founder 
58:35 Running out of Data 
01:03:22 In other industries 
01:24:52 Limitations 
01:31:50 Build something that gets better as AI gets better

Links:

https://chat.openai.com/

https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulator#Simulators

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

https://xyproblem.info/

https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1

===== About “The Technium” ===== 
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world. Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials ===== 
WEBSITE:

https://technium.transistor.fm/

SPOTIFY:

https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7

APPLE PODCASTS:

https://podcasts.apple.com/us/podcast/the-technium/id1608747545

ChatGPT is a language model by OpenAI which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.

00:00 S04E03 ChatGPT Part 1
02:58 What is ChatGPT? 
06:17 Lineage of ChatGPT 
10:01 General Reasoning from Training on Code
25:13 Reinforcement Learning from Human Feedback
29:37 Dumb compute, stacked high
42:00 Talking to an Alien Intelligence
45:14 What's it good at and bad at?
56:05 The XY problem
01:03:36 GPT in Education
01:13:26 Eating High Status Jobs, Fulfilling Latent Demand

Links:
- https://chat.openai.com/
- https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulator#Simulators
- http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- https://xyproblem.info/

===== About “The Technium” =====
The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.
Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/ 
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7 
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

Nix is a functional package manager. It creates completely reproducible builds and package management, including support for multiple versions of packages side-by-size with no issues.

Chapters:

[00:00:00] Intro

[00:02:41] What is Nix?

[00:05:01] The Problem with Package Management

[00:09:41] How Nix Works

[00:15:08] Reproducible Environment = Less Bit Rot?

[00:17:46] Nix vs. Docker

[00:21:32] Adoption at Growing Startups

[00:26:05] Use Case: Consistency Across Environments

[00:28:04] Use Case: Provisioning New Dev Machines

[00:32:24] Use Case: Portable Editor Configs

[00:35:05] Why Hasn't This Taken Off?

[00:40:27] Leaky Abstractions

[00:44:17] Security and other enterprise value props

[00:52:31] Improving the documentation

[00:56:34] Nix for software archival/archaeology

[01:02:21] Reducing DevOps/SRE burden

[01:07:51] Integrating Nix directly into programming languages

[01:09:57] Someone should build the Heroku for Nix

[01:25:17] How to take a position on Nix

Links/Resources:

• https://edolstra.github.io/pubs/phd-thesis.pdf
• https://www.iohannes.us/en/commentary/nix-critique/
• https://nixos.org/guides/how-nix-works.html
• https://nix.dev/
• https://blog.replit.com/powered-by-nix
• https://blog.replit.com/nix-vs-docker
• https://blog.wesleyac.com/posts/the-curse-of-nixos
• https://bicompact.space/blog/2020/06/20/nix-os-thoughts/
• https://shopify.engineering/what-is-nix
• https://news.ycombinator.com/item?id=28241661
• https://nixcloud.io/tour
• https://www.youtube.com/playlist?list=PLRGI9KQ3_HP_OFRG6R-p4iFgMSK1t5BHs

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====

WEBSITE: https://technium.transistor.fm/ 

SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7 

APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545

'Visual Programming' refers a style of programming that allows the user to specify a programs in a two-(or more)-dimensional fashion. Visual programming environments represent the data, control flow, or program state in a graphical way, allowing them to be directly manipulated. It has been a hot area of research from the very beginning of personal computing, to today.

This week we will cover a few major visual programming environments, why visual programming has remained compelling over the decades, and whether there is untapped potential for VP today.

Chapters:

[00:00:00] Intros

[00:03:50] What is Visual Programming?

[00:05:42] Origins

[00:14:34] Block-based Visual Programming

[00:20:26] Wire and Dataflow-based Visual Programming

[00:31:51] An Umbrella Term

[00:36:31] Conceptual History

[00:48:23] The Duality of Direct Manipulation

[00:58:40] Direct Manipulation of Running State

[01:11:25] Programming by Example

[01:21:17] Fill in the Details for Me

[01:28:49] Strengths of Visual Programming

[01:43:36] Leveraging the Visual Cortex

[01:50:58] Second Order Effects

Links/Resources:

• SketchPad demo: https://www.youtube.com/watch?v=2Cq8S3jzJiQ
• Pygmilion 
  • Paper: http://worrydream.com/refs/Smith%20-%20Pygmalion.pdf
  • Demo:: https://youtu.be/xNW8wUpbqQM?t=319
• Grail
  • Demo: https://www.youtube.com/watch?v=2Cq8S3jzJiQ
• Hypercard
  • Demo: https://www.youtube.com/watch?v=2Cq8S3jzJiQ
• Viewpoint https://scottkim.com/2020/06/07/viewpoint/
• Scratch
  • https://www.bryanbraun.com/2022/07/16/scratch-is-a-big-deal/
• Labview (imperative control flow): https://www.ni.com/en-us/shop/labview.html
• Unreal Engine Blueprint (functional)
  • https://docs.unrealengine.com/5.0/en-US/blueprints-visual-scripting-in-unreal-engine/
  • https://blueprintsfromhell.tumblr.com/
• Max/MSP for musicians
  • https://cycling74.com/products/max
• Others
  • https://cables.gl/
  • https://nodes.io/
    
    

===== About “The Technium” =====

The Technium is a weekly podcast discussing the edge of technology and what we can build with it. Each week, Sri and Wil introduce a big idea in the future of computing and extrapolate the effect it will have on the world.

Follow us for new videos every week on web3, cryptocurrency, programming languages, machine learning, artificial intelligence, and more!

===== Socials =====
WEBSITE: https://technium.transistor.fm/
SPOTIFY: https://open.spotify.com/show/1ljTFMgTeRQJ69KRWAkBy7
APPLE PODCASTS: https://podcasts.apple.com/us/podcast/the-technium/id1608747545