The Swyx Mixtape

From the Huberman Lab podcast: https://www.listennotes.com/podcasts/huberman-lab/dr-chris-palmer-diet-JsmfBwoKN6M/ 2h 23mins in

Listen to the Levels podcast: https://www.listennotes.com/podcasts/levels-a-whole-new/142-the-truth-about-why-keto-upKlt7fGAJY/

Subscribe to Changelog++: https://changelog.com/podcast/519/discuss

Featuring

• Shawn Wang – Twitter, GitHub, Website
• Adam Stacoviak – Mastodon, Twitter, GitHub, LinkedIn, Website
• Jerod Santo – Mastodon, Twitter, GitHub, LinkedIn

Notes and Links

• AI Notes
• Why “Prompt Engineering” and “Generative AI” are overhyped
• Multiverse, not Metaverse
• The Particle/Wave Duality Theory of Knowledge
• OpenRAIL: Towards open and responsible AI licensing frameworks
• Open-ish from Luis Villa
• ChatGPT for Google
• The Myth of The Infrastructure Phase

ChatGPT examples in the wild

• Debugging code
• TypeScript answer is wrong
• Fix code and explain fix
• dynamic programming
• Translating/refactoring Wasplang DSL
• AWS IAM policies
• Code that combines multiple cloud services
• Solving a code problem
• Explain computer networks homework
• Rewriting code from elixir to PHP
• Turning ChatGPT into an interpreter for a custom language, and then generating code and executing it, and solving Advent of Code correctly
• Including getting #1 place
• “I haven’t done a single google search or consulted any external documentation to do it and I was able to progress faster than I have ever did before when learning a new thing.”
• Build holy grail website and followup with framework, copy, repsonsiveness

For ++ subscribers

• Getting Senpai To Notice You
• Moving to Obsidian as a Public Second Brain

Transcript

**Jerod Santo:** Alright, well we have Sean Wang here again. Swyx, welcome back to the show.

**Shawn Wang:** Thanks for having me back on. I have lost count of how many times, but I need to track my annual appearance on the Changelog.

**Adam Stacoviak:** Is that twice this year on this show, and then once on JS Party at least, right?

**Shawn Wang:** Something like that, yeah. I don't know, it's a dream come true, because, I changed careers into tech listening to the Changelog, so every time I'm asked on, I'm always super-grateful. So yeah, here to chat about all the hottest, latest things, right?

**Adam Stacoviak:** Yeah.

**Jerod Santo:** That's right, there's so much going on right now. It seems like things just exploded this fall. So we had Stable Diffusion back in late August; it really blew up at the end of August. And then in September is when we had Simon Willison on the show to talk about Stable Diffusion breaking the internet. You've been tracking this stuff really closely. You even have a Substack, and you've got Obsidian notes out there in the wild, and then of course, you're learning in public, so whenever Swyx is learning something, we're all kind of learning along with you... Which is why we brought you back on. I actually included your Stable Diffusion 2.0 summary stuff in our Changelog News episode a couple of weeks back, and a really interesting part of that post that you have, that I didn't talk about much, but I touched on and I want you to expand upon here is this idea of prompt engineering, not as a cool thing, but really as a product smell. And when I first saw it, I was like, "No, man, it's cool." And then I read your explainer and I'm like, "No, he's right. This is kind of a smell."

**Adam Stacoviak:** "Dang it, he's right again."

**Jerod Santo:** Yeah. We just learned about prompt engineering back in September, with Simon, and talking about casting spells and all this, and now it's like, well, you think it's overhyped. I'll stop prompting you, and I'll just let you engineer an answer.

**Jerod Santo:** Well, so I don't know if you know, but the Substack itself got its start because I listened to the Simon episode, and I was like, "No, no, no. Spellcasting is not the way to view this thing. It's not something we glorify." And that's why I wrote "Multiverse, not Metaverse", because the argument was that prompting is -- you can view prompting as a window into a different universe, with a different seed, and every seed is a different universe. And funny enough, there's a finite number of seeds, because basically, Stable Diffusion has a 512x512 space that determines the total number of seeds.

So yeah, prompt engineering \[unintelligible 00:04:23.23\] is not my opinion. I'm just reporting on what the AI thought leaders are already saying, and I just happen to agree with it, which is that it's very, very brittle. The most interesting finding in the academic arena about prompt engineering is that default GPT-3, they ran it against some benchmarks and it came up with like a score of 17 out of 100. So that's a pretty low benchmark of like just some logical, deductive reasoning type intelligence tests. But then you add the prompt "Let's think step by step" to it, and that increases the score from 17 to 83... Which is extremely -- like, that sounds great. Like I said, it's a magic spell that I can just kind of throw onto any problems and make it think better... But if you think about it a little bit more, like, would you actually use this in a real work environment, if you said the wrong thing and it suddenly deteriorates in quality - that's not good, and that's not something that you want to have in any stable, robust product; you want robustness, you want natural language understanding, to understand what you want, not to react to random artifacts and keywords that you give.

Since then, we actually now know why "Let's think step by step" is a magic keyword, by the way, because -- and this is part of transformer architecture, which is that the neural network has a very limited working memory, and if you ask a question that requires too many steps to calculate the end result, it do...

Listen to the Modern Mann: https://www.modernmann.co.uk/new/fog 20 mins in

• Send My Love  https://www.youtube.com/watch?v=q2qAsUxiogw
• Shape of You https://www.youtube.com/watch?v=07fQy1WS1WM

From the Kelsey Hightower Distinguished Gentleman Twitter Space.

https://twitter.com/i/spaces/1dRJZMpoMArGB 2hrs 43mins in

From the Kelsey Hightower Distinguished Gentleman Twitter Space.

https://twitter.com/i/spaces/1dRJZMpoMArGB

From the Kelsey Hightower Distinguished Gentleman Twitter Space.

https://twitter.com/i/spaces/1dRJZMpoMArGB

28mins into https://twitter.com/i/spaces/1dRJZMpoMArGB

My Fave New Podcasts of 2022: https://www.swyx.io/fave-podcasts-2022/

Selected listener survey responses

if you missed the survey in July, you can still fill it in here: https://forms.gle/gmVYak98azGi28it8

What other podcasts do Swyx Mixtape listeners enjoy?

• Conan O‘Brian needs a friend; alphalist cto podcast;remote ruby
• ShopTalk Show, WTF with Marc Maron, Hardcore History
• Corecursive, Working Code, Soft Skills Engineering
• Software unscripted, your undivided attention, startups for the rest of us
• I like software sessions by Jeremy Jung and I have listened to quite a bit of Zero Knowledge podcast which is crypto related.
• Switched on Pop, Stratechery, Radio Derb
• cppcast, hanselmines, go time
• Swyx Mixtape, Ezra Klein Show, Acquired

What are your favorite episode(s) of the Mixtape so far? 

• Star Wars Lofi because I loved it. Career+Luck as it is a great tooic
• Behind the scenes of the business world. I’m not interested enough to listen to full podcasts about business, but I really appreciate the small nuggets of insight.
• All of them not about cryptocurrency
• The levels one (because I’m a diabetic dev)
• I started extensively listening starting in June. I really enjoye the episodes covering snowflake databricks. You introduced the idea of betting on technology to me and I trust your judgement. So the podcast gives me a list of interesting tech I should at least know about without too much effort on my end.
• This one! I like it! And I can’t remember some of the very good earlier ones. I don’t listen to that many podcasts. I dunno how you do it! Kudos
• latest deep work series was great, I love to work or at least think about and how to improve. the learning in public thing was what made you follow for quite a while now/ the episode about the drowned singer/actor from glee touched me and I researched the topic and learned how to hopefully act correctly if I ever got in rip current
• One about Xbox losing all it's market share after the Xbox One launch (I love tech war stories--this field moves so fast. I find I know a lot about the preset but so little about the past)