Latent Space

Legora can’t read a single trace of what its agent does for a lawyer. The data belongs to the law firm’s clients. How do you build and test an agent when you can’t see its work?

Jacob Lauritzen is the CTO of Legora, one of the fastest-growing companies in legal tech history. The team started in a Stockholm basement with ChatGPT, Swedish law, and a search tool. Today, a lawyer can use Legora to draft and edit Word documents, review 10,000 contracts in a single table, or run due diligence. The agent also coordinates work between lawyers, clients, and other agents. When Legora made its own acquisitions, the team used it for diligence.

What we talked about

• Why Legora resisted adding a model picker for years, and why it finally added one

• Jacob’s estimate that models already do about 60% of legal work well, and why the rest takes a smarter model

• How three engineers and three lawyers build test environments modeled on customers’ work, much like the “worlds” used in reinforcement learning

• A citation problem where the model got it right 70% of the time, then 99% after one release. Legora didn’t have to change anything.

• Why Legora runs everything on Azure and why it doesn’t use ElevenLabs

• Tabular Review, the million-cell spreadsheet the industry copied down to the name

• How intake works, how agents create lists, and how a list can start another agent

• What legal engineers and forward-deployed engineers each do, and who gets to write code

• The 300 Spartans slide Jacob used to argue that 20 engineers would be enough forever. Now there are 100, and he still feels short-handed.

• Why Jacob finds Fable and Opus too chatty for drafting and likes Astra’s concise writing

• Why drafting a long contract is a search problem

• Whether model labs will cut off legal tools the way they cut off coding tools

• What happens to law firm partnerships, the billable hour, and jobs for junior lawyers

Legora is hiring engineers in New York, Stockholm, and London.

Chapters

0:00 Cold open

0:40 Meet Jacob Lauritzen, CTO of Legora

1:05 Starting out as underdogs in a Stockholm basement

1:54 How Legora grew from ChatGPT with Swedish law into a legal agent

4:12 The switch from assistant to agent

5:23 Which models Legora uses and why it added a model picker

7:48 Are today’s models good enough for lawyers?

8:20 What agents couldn’t do nine months ago

9:18 Testing the agent in simulated worlds and a 10,000-file data room

10:07 The evals team and Legora’s 100 legal engineers

11:03 When evals didn’t seem worth it and one release took citations from 70% to 99%

12:18 What legal practice could look like in five to ten years

14:18 Will the law firm partnership model survive?

16:40 What happens to paralegals and junior lawyers?

18:00 The billable hour and why earlier legal tech fell short

19:39 Where the money goes now

22:44 How Legora used its own product for acquisition diligence

23:59 How Swedish and American law firms differ

26:12 Product metrics when you can’t read the traces

27:41 Paraphrasing traces and narrowing failures down to 10 or 20 types

28:42 Why data privacy led Legora to run everything on Azure

30:04 Open-weight models and model arbitrage

30:55 Transcription models and why Legora doesn’t use ElevenLabs

33:33 The agent, Tabular Review, projects, and the Word add-in

34:24 How a lawyer uses Tabular Review

35:51 Why a spreadsheet works and what you learn from the first 10 rows

37:53 How the legal intake bot works in Slack

39:11 How agents create lists and lists start agents

40:15 Building products on those same pieces and acquiring Wexler

42:37 What forward-deployed engineers do at Legora

43:52 How legal engineers and forward-deployed engineers onboard a big client

46:43 What’s next for the forward-deployed team over the coming year

47:24 Why Legora resisted building APIs

49:26 Can the agent be the API?

50:12 The 300 Spartans slide and the plan to stay at 20 engineers

51:32 The bets nobody asked Legora to make

53:05 Post-training open-weight models for legal research

54:05 Search, retrieval, and late interaction

54:51 What makes a good legal agent harness and why drafting depends on search

56:25 Which model writes the best draft?

57:03 Hallucinations and Legora’s citation system

57:41 Do lawyers check the citations?

58:19 MapReduce or one big context window for a million-token contract?

1:00:16 Getting anonymized examples and the idea of buying a law firm for its data

1:01:53 Why Jacob advises against default embeddings

1:02:44 Will lawyers work in Legora or through Claude?

1:03:26 Will the model labs cut Legora off?

1:04:24 Law schools and scholarships for computer science majors

1:05:28 Hiring in New York, Stockholm, and London

1:06:09 Why most work eventually becomes legal work

Guest — Jacob Lauritzen, CTO of Legora

https://legora.com

Co-host — swyx, Latent Space

Host — Basil Chatha, Forward Deployed

https://forwardeployed.com

At AI Engineer New York this month, we sat down with Aman Singh, the lead AI Engineer at Arca, an “AI-native” company providing wealth management services.

We discovered that while agents do most of the background work in the firm, there are guardrails in place to ensure that human advisors are making the final decisions. The goal is for AI to guide the process; or as Singh nicely put it, “agents prompt people more than people prompt agents.”

Arca recently raised $48.5 Million in a round led by General Catalyst, with participation from Index Ventures and Venrock.

One of the features of using AI in financial services, explained Singh, is that it’s typically long-horizon work — although not long-running. In his AIE presentation, Singh noted that agent tasks at Arca can take up to ten days. “It's very stop-start, there's many actors in between, many decision points, many handoffs,” he said.

For this reason, Arca has developed an internal memory system.

“So we have this persistence for every client,” Singh said in our podcast discussion. “What has happened so far? What are we tracking for next? What needs to happen immediately in the next couple of days [and] what are we tracking for the long term? Which individual is responsible for which action? All of this is being tracked by our memory systems that our agents contribute to.”

Arca uses models from frontier labs like Anthropic and OpenAI, with Claude its primary model provider. The company has built its own harness, which automatically selects the best model to use for a particular task, based on the company’s internal benchmarks.

Owning the harness, rather than using a managed agent harness, also means that Arca has the flexibility to change model providers if it needs to.

Safety is obviously key for financial services. Singh noted that although agents do much of the work, an agent’s role is limited to analysis and proposing solutions. Trading and any other money movement is handled by a human advisor.

“All of our safety restrictions are fully deterministic,” he said. “And they take the decision making off the agent's hands.”

In other words, there are certain tools and tasks the agent can execute by itself — but not ones that involve risk to the client. Arca also runs verification checks on known AI security issues, such as prompt injections.

Arca’s agent isn’t itself sandboxed, but it does use sandboxes for computation. Singh explained they’ve gone even further, with extra security measures for handling sensitive data.

“We've built systems to make it safe for code inside of that sandbox to access our data,” he said. “So we have this thing which translates the code it [the agent] wrote into a strict format that we expect, throw away the code, and you install guardrails on that format before it reaches your back end. So we allow the agents to use code to query our data, which means that they can aggregate it, they can munge it, they can work with it as they see fit to produce [things] like a financial plan.”

Another interesting process Arca has implemented is using feedback from human advisors, along with traces from the agents, to improve the harness. A “meta-agent” is used to analyze all this data and suggest changes to the harness.

• 00:00 — Introducing Arca’s AI-native wealth management firm

• 01:40 — How AI supports human financial advisors

• 02:40 — Choosing models with internal benchmarks

• 04:01 — Building a custom harness and routing tasks

• 05:25 — Handling a $100,000 tax bill: Agents and human approvals

• 07:20 — Agent memory across long-running tasks

• 09:22 — Safety guardrails and prompt injection defenses

• 11:02 — Compliance and auditability by default

• 12:14 — “Agents prompt people more than people prompt agents”

• 13:46 — Why Arca owns its agent infrastructure

• 15:05 — AI-native firms versus firms using AI tools

• 16:09 — Sandboxes and financial plan verification

• 18:40 — Decision models and checking financial facts

• 19:51 — Why trade execution stays with human experts

• 20:32 — Turning staff feedback into evals and improvements

• 22:22 — Turning operational knowledge into agent context

• 23:41 — Client examples: Hidden fees and proactive service

I first met Paul Copplestone and Ant Wilson in Singapore, shortly after they finished YC as the hottest startup of YC S20. Six years later, we finally got them on the podcast! This is their first podcast together, after taking an open-source, fully remote company from zero to a $10 billion valuation.

The title is a question I wanted to understand for myself: why do agents love Supabase? There are plenty of managed databases. Postgres hosting is competitive. Being familiar to a model is useful distribution, but it doesn't explain all the work needed to turn a generated application into something that runs. But when I ran my frontier AEO tracker, it was an almost complete sweep of first choice for app backends (and in contention for managed db) for every frontier model from Astra to Fable to Muse.

At Select, Supabase grouped its announcements into Build, Operate and Scale.

The things that stood out to me were surprisingly practical: start a local backend without Docker, keep the desired database schema in Git, and run compute beside the database. Each removes a piece of work that developers have been doing for years. Agents make that friction harder to ignore.

Watch the full conversation, including my opening commentary →

My favorite developer-tool announcements make me think: of course I want this today. The new alpha runtime for running Supabase locally without Docker is one of those. I use Colima rather than Docker Desktop, and even then, removing another dependency from the path to a working database is welcome. Local setup is a small part of a human developer's week; an agent that repeatedly creates and tests environments encounters it over and over.

Declarative schemas are the bigger idea. The familiar SQL workflow accumulates a history: create the schema, then migrate, migrate, migrate. A declarative representation gives the agent a description of the state you want, in files it can inspect and edit alongside the application.

In the commentary, I compare this to a Terraform moment for the database. Put the desired state in Git and the agent can read it, change it, fork it and work against it as part of the repository. Humans get the same advantage: the backend becomes easier to understand without reconstructing its entire history.

That is the part I like most about the agent-friendly direction. These are improvements I wanted as a developer already. Making them useful to agents gives the team another reason to finish the job.

I think Supabase Compute, now in private alpha, may be undersold. An application needs somewhere to do its work as well as somewhere to store the result. Once you have a database, you still need a place to run the processes that chunk documents, generate embeddings or do other longer-running work.

Paul and Ant discuss running containers close to the database. That closes a gap between getting an application started and operating it: fewer separate services to configure, connect and explain to the agent. Edge functions remain useful, but they don't cover every workload.

The operating side matters too. Logs, checks, connections and context help an agent understand what it is changing. A model that can issue SQL still needs to know what the database contains, how the application uses it, and what happened when something failed. Supabase's Operate announcements are worth reading alongside the new ways to build.

The founders say Supabase is launching about five million databases a month. They are thinking about what happens if agents push that toward billions of applications. At the same time, existing customers are growing into much larger workloads. Serving more small databases and supporting a customer's increasingly large database are different engineering problems.

That helps explain the breadth of the Scale announcements: OrioleDB, Multigres and the Turso acquisition. The conversation gets into the work behind the new “prototype to petabyte” positioning, including Postgres reliability and the choices involved in scaling up and scaling out.

One observation from Paul and Ant is particularly interesting for anyone building developer tools: as AI-driven usage grew, they say activation and conversion improved. Their explanation is that an agent can carry a user from signup to a working setup in one flow. The person doesn't have to figure out every configuration step before seeing the application work.

We also talk about the memes and community that helped Supabase grow, open-source adoption and enterprise sales, and whether agents will converge on the same application patterns. But I wanted this episode to capture the founders' chemistry as well. It is a chance to hear Paul and Ant think through the product together—and to see how much work sits behind a backend that an agent can make look easy.

• Supabase Compute — run workloads alongside the database.

• Multigres — the team’s approach to highly available Postgres, currently in private alpha.

• Supabase acquires Turso — the database architecture behind its scale-out ambitions.

• Supabase Select — the event discussed in this conversation.

0:00 Welcome and why agents love Supabase
5:23 Paul Copplestone and Ant Wilson
6:39 Compute beside the database
8:05 OrioleDB and Multigres
12:50 Scaling up, scaling out, and Turso
17:53 Developer marketing and memes
23:15 The launch and the migration
31:03 Vectors, growth, and AI recommendations
41:43 Product-led sales and enterprise growth
49:38 Live migrations and analytics
58:43 Agent-ready local development
1:06:43 Community and company workflows

Katelyn Lesse and Angela Jiang join Latent Space to explain the engineering behind the Claude Platform: launching Opus 5.5, separating the model’s “brain” from the agent’s “hands,” scaling APIs, and building reliable managed agents.

A new introduction covers Anthropic’s S-1, the improvement and investment loops, and why the platform matters.

Watch the full episode on YouTube

Show notes

Timestamps

0:00 Anthropic’s S-1 and this interview
1:06 Revenue per employee and efficiency
3:13 The improvement and investment loops
4:04 Why Claude Platform matters
6:25 Training incidents and production safeguards
7:30 Opus 5.5 launch and pricing
9:25 Distillation and preserved thinking
14:35 Product and engineering at Anthropic
18:05 Scaling the Claude Platform
19:25 Model access and safety decisions
26:25 Trust grants and product roadmaps
30:25 How Anthropic launches models
35:55 Benchmarks and model personalization
42:10 Claude Managed Agents
44:25 Memory, dreaming, and agent harnesses
48:25 Evaluating agents
53:25 CPU constraints and scaling
56:05 Rebuilding the Messages API in Rust
1:03:25 The brain and the hands
1:05:25 Protecting credentials and sandboxes
1:08:25 Training incidents versus production
1:10:55 Pacing and launching frontier models
1:12:25 Safety classifiers and model routing
1:19:25 Emerging agent workloads
1:20:55 Fighting AI slop
1:27:25 Management, writing, and working styles
1:33:25 Building Anthropic’s DevRel team
1:40:25 Developer relations and product voices
1:43:25 What to try with Opus 5.5

Warm welcome to Basil Chatha, our newest cohost from the Forward Deployed side of the AI Engineering world! We’ll be ramping up a new show with him and couldn’t pass up an interview with Modal’s CEO Eric on the occasion of their monster Series D!

Two years ago, we spoke with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, predicting load, or paying for idle resources. CTO Akshat then joined us to talk about Modal’s infra primitives and stack to offer those primitives and agent experience.

A lot has changed in the past 5 months. Since their $5B series C, Modal now reports it crossed $1 billion in annualized run rate, raised a $750 million Series D at a $16B valuation, and has spun up nearly three billion sandboxes. In this conversation, Erik joins Vibhu to explain the infrastructure and economics behind that growth—and why demand for compute rarely looks like the contracts used to buy it.

Watch the full conversation on YouTube →

A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle. Real workloads behave differently: they grow, spike, and fall throughout the day. Buy enough for the peak and you pay for idle hardware; buy less and you risk missing demand.

Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it. The smoother the aggregate demand, the more useful that pool can be. It’s a useful companion to our conversation with SF Compute’s Evan Conrad: Evan focuses on making capacity tradeable; Erik focuses on making it usable in smaller, elastic slices.

There is a financial risk on the other side. If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply. Erik says he doesn’t expect an imminent collapse, but thinks the possibility matters when building the business. Jump to the GPU price discussion →

Modal launched sandboxes after Erik and cofounder Akshat Bubna noticed that language models could generate code—and that someone would need a place to run it. Early adoption was modest enough that the team discussed shutting the product down. Then coding agents, background agents, and reinforcement learning created new demand. Erik says sandboxes have grown to roughly a third of Modal’s revenue, a result he didn’t predict at launch.

That growth also changes what infrastructure has to do. An agent may create an environment, run code, inspect the result, and try again many times. The latency of starting and restoring those environments becomes part of the experience. In our recent interview with Akshat, we explored this shift from developer experience to agent experience; here Erik adds the product and business story. Jump to the sandbox discussion →

Elasticity only works if capacity can arrive quickly enough to matter. Modal’s serverless GPU write-up describes the machinery: a buffer of available GPUs, a custom filesystem, process checkpointing, and GPU memory snapshots. Those pieces reduce the work needed to bring another model replica online. It is a concrete example of how infrastructure decisions show up in an application’s responsiveness and cost.

We also discuss Erik’s path from Spotify’s recommendation system to Modal, how the company buys compute, alternatives to NVIDIA, forward-deployed engineers, and why code review is optional at Modal. Jump to engineering culture →

• Modal — the AI infrastructure platform Erik founded.

• Erik’s writing — his essays on infrastructure, markets, and building companies.

• Modal’s Series D announcement — the company’s latest reported growth and sandbox milestones.

• How Modal achieved truly serverless GPUs — the startup and scaling techniques behind elastic inference.

• Modal’s resource solver — how it selects capacity across changing prices and availability.

• Scaling to one million concurrent sandboxes — an engineering account of sandbox scaling.

• Our first interview with Erik — the original serverless infrastructure thesis.

• Our interview with Akshat Bubna — infrastructure and developer experience for agents.

• The Programmable Data Center — Evan Conrad on GPU trading and physical settlement.

• 0:00 What Modal does

• 1:25 From Spotify to building Modal

• 4:19 Founding before the generative AI boom

• 7:43 First customers and fundraising

• 10:00 Building for inference and autoscaling

• 13:21 Where Modal gets GPU capacity

• 15:39 Renting vs. owning infrastructure

• 19:45 The risk of GPU prices falling

• 24:18 Is AI demand infinite?

• 26:12 Alternatives to NVIDIA

• 28:25 What Modal can learn from neoclouds

• 30:26 GPUs as an asset class

• 31:55 Modal’s products: functions and sandboxes

• 36:03 The case for smaller training workloads

• 41:21 How agent sandboxes are changing

• 46:48 Anthropic and sandbox security

• 49:29 Using agents to test security

• 52:51 Developer experience in the age of agents

• 57:45 What Modal’s forward-deployed engineers do

• 59:55 Modal’s approach to product management

• 1:03:29 Optional code reviews and engineering culture

• 1:05:56 Cloud, on-prem and bring-your-own-cloud

• 1:09:25 Capacity, fundraising and growth

• 1:13:07 Scaling the team

• 1:17:28 Erik’s tweets: ephemeral sandboxes

• 1:18:31 The problem with rectangular GPU contracts

• 1:21:20 Inference providers vs. model builders

• 1:23:25 Where models improve next

Automatically transcribed from the revised edit; names and wording may contain errors.

0:00 Thanks for coming on, Eric Fern-Hartson from Modal. Could you explain the business a little bit and maybe how you got here? Yeah. So we build, the easiest way to put it is we build AI infrastructure. So we offer a compute platform and a storage platform that enables a lot of customers like Dacagon, and Suno, and Runway, and Cognition, and many others to essentially build AI applications. So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs and other things. So we have basically built a software SDK that enables customers to move really fast and build this. We manage all the capacity underneath, make it easier to deploy and scale and run these things in production. So you were on Light and Space like two years ago, a lot has happened since then. How big were you back then, like whatever numbers you can share and comparatively, how big are you now? I actually don’t remember where we were, but it’s probably like at least 50X, there’s revenue and users and users. Yeah. I use it actually for a side project that I have. Nice. Yeah. It’s like, yeah, turn 2D images into 3D models. Yeah. So yeah, just hosting a couple of models here. What do you think? Any feedback? I mean, I love it. Nice. Good. I mean, yeah, that’s why I use it over any competitors or anything. Good. So before you started Modal, you were at Spotify and also like better.com, I think.

1:30 That’s right. Yeah. Yeah. What were you doing at Spotify? I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system. The first version I joined after school in 2008, 2009 maybe, and spent seven years at Spotify. And most of that time was building basically from scratch and music recommendation system and built up a small team and we launched a bunch of features like related artists and Discover Weekly and a bunch of other stuff. So a lot of, a lot of matrix factorization, a lot of compute intensive stuff. And arguably like a lot of what sort of later became the genesis of Modal was just building AI applications and realizing like the infrastructure fucking sucks. Like, there’s got to be a better way to do that. So a lot of sort of, you know, the rationale for Modal was like kind of building a tool I always wanted to have throughout my years at Spotify. Yeah. So Modal was more of an offshoot of the stuff you learn at Spotify, not much from Better.com. Better.com was a little bit of a side tour. It was kind of a roller coaster, a long story about the company. We talked about it separately. But I was a CTO for five years of better and grew to a tech team from, I think, you know, two to three, four hundred. So I learned a lot about management and building teams and hiring and also learned a lot about developer productivity. I thought a lot about tools and things like that. Did a lot more like full stack stuff. But yeah, I kind of came back to my, was close to my heart, which is infrastructure and AI and machine learning after that. What was Better.com doing again? Mortgages. Okay. Yeah. How did you get into that?

3:00 I mean, we don’t have to spend too much time on that. Yeah, no, it’s fine. Like this guy, Zach Frankel, who I didn’t know at that time, ended up recruiting me to Better as a CTO and the company was 10 people. And, you know, for whatever reason, you know, the founder of Vishal, very interesting guy, very, very smart, a little crazy in mostly positive ways. We got very well along. So we, yeah, we started building it in 2015 and I spent five years growing that company and then, you know, eventually turned out that doing mortgages is pretty hard and, you know, interest rates went up very quickly after the pandemic and that cost a lot of issues at the company. But by then I had already left. That was like 2020, 2021 when you left? Yeah, something like that. Yeah. So, OK, you left before you had the idea for a model or you left and then you started? It’s hard to say. I had a lot of random ideas when I left Better and I, I actually wasn’t sure if I wanted to start a company. I was kind of just, you know, during the pandemic, I sort of almost took like a sabbatical at two kids. I mean, I still have kids, but I mean, at that time there were there were I think two and four and so I decided to hang out and took some time off and was kind of working like part-time, like hacking on certain things and wasn’t sure maybe I should do some consulting and I started building stuff. And I just got like super pumped about this idea that eventually became model. But actually both, you know, model was like, I think the 30 I built that I got, you know, that so there were like a few attempts before then that went nowhere.

4:30 But my model was like the first idea was like, this is kind of cool. Like I should do this. This is like, this could be a really cool platform. Were the other ideas like also in a similar vein like developer tools? The first thing was like some sort of web scraping thing, which went nowhere. But actually, you know, in hindsight, maybe could have gone somewhere. And was this before the chat GBT launch? Yeah. Yeah. 2021. Oh, OK. So chat GBT was late 2022, right? Then I started building, so maybe actually to rewind a little bit context, like when I worked at Spotify, I released an open source framework, Kaluigi, which is like workflow orchestration. So sort of the same categories, airflow and pre-effect index or sort of dated categories to some extent, like people don’t talk about as much. But for a while, it was like, you know, got a lot of attention. Anyway, I never turned it into business. But a couple of years later, when in 2021, when I was thinking about business ideas or things to work on, I kind of came back to workflow orchestration. And I started hacking on a bunch of ideas of, you know, running code in the, you know, like basic orchestrating code running in the cloud. And what I realized was like, you know, at the end of the day, like core puzzle pieces, not just like orchestrating the computer, it’s like executing the computer. And I was like, that’s kind of a bigger problem actually to solve. So in a way, I actually like, you know, left it on the shelf and then said, like, actually, I’m going to like go to this other thing over here and I’ll go back. I’ll get back to the workflow scheduling a little bit. But, but like, you know, I never got back to long story short and ended up building modal, which is like the focus is more like, let’s just figure out a way to make it easy to run code in the computer, in the cloud. And the focus was initially like fairly general purpose.

6:00 And it actually in a way like it’s always, we’ve kind of gone full circle more recently, but, but we focused on so many different applications. It’s kind of the idea was like, let’s just make it super easy to run code in the, in the cloud. Could be GPU code, AI, machine learning. It could also be like data stuff. And focused a lot on, you know, how do we scale it up? How do we make the developer experience awesome? Realized super early that like in order to do that, we’d have to go very deep in a stack and built on file system, built on container runtime and did a lot of, you know, solved a lot of hard engineering challenges. And then you mentioned chat, CPT, that was actually not quite the killer app. But like, if you remember, like that fall, another thing came out around the same time, which was stable diffusion and stable diffusion turned out to be a killer app for our, for what we’re building at that time. We were only four or five people at that time. So we started seeing a lot of people coming to us and wanting to run a stable diffusion model. And then we were real, that, that, that made us realize like maybe gen AI inference is actually a really good use case for us. And how are you finding it? I’m just like compulsive tweeter. I was even more like now, like I’m too distracted by building a company, but I always like tweeted a lot. I was blogged a lot. So I think I had a little bit of a following. And so, you know, in the early days, I like a lot of what drove the growth was just like me just compulsively just like tweeting about model and like what I was building and getting people excited and writing long blog posts about where the cloud is going and all these things. And I call it a bit like founder led marketing. I always like feel like you kind of have to lean into like what you’re good at

7:30 and like what I’m good at is like, I guess like just shit posting and being terminally online. And so, you know, like work, it’s still like, you know, I think a lot of our growth is we’re just kind of good at social media and just like being, you know, online a lot. Nice. How long until you had your first customer? It took a while. Also, did you raise like early on? We didn’t raise for a while. I actually self funded it for a while, even with up until the first few employees, which in a way was like kind of nice because it may be really like frugal about money and like wanting to see like ROI. I wasn’t taking out a salary for a while for a long time. Actually, at some point it became a problem because I needed the money, but we didn’t raise for a while. And until early 2022, we raised the seed rounds. So at that point I’d already been working on it for about a year. Akshah joins the CTO and co-founder in about five years ago, about now August, maybe 2021. But then it took until like basically like late 2022 for us to see any meaningful revenue. I guess that’s why I’m proud of you. Yeah. And stable diffusion then in our case was the main thing that’s sort of, so we went from like, you know, $100 a month in like, I think like November, 2022. And the next month we did like $2,000 a month. And then like January, 2023, we did like 30,000 all of a sudden. And, you know, so finally I felt this like massive, just like anxiety thing off my chest. I’m like, maybe this is actually like we’re onto something. It feels like we’re onto something. And then 2023 we grew very fast.

9:00 And I guess we’ve been growing very fast since then basically. That 30 K a month moment, was that just like a couple of big customers or was it like very diverse? It’s just one. And I don’t know if I ever talked about like who that was and if they even know it, but I think they suspect it. It was a company in New York called Mage. They’re doing like a stable diffusion company. So at some point, you know, in the first, I think first quarter of 2023, I think there were like 80, 90% of our revenue. But then after that, you know, it started diversifying a little bit. And late 2023, we got soon as a big customer. They were a big chunk of our revenue for a while too. But then, you know, it’s over time, like we actually have really good revenue diversification now. Yeah. For the stable diffusion stuff, I remember using it for like comfy UI. I’m assuming a lot of people were using comfy UI, like running comfy UI workflows on it. Some people were, but I think the people that were using model, they tended to be more like low level stuff. Like they did like complicated stuff. What they like with models, like it’s always been pretty hackable. So they deployed their own custom workflows with adapters and custom inference and a lot of optimizations and all the stuff we had done in order to get the developer experience to be really good, turn out to be really good for also running production workloads. Cause auto scaling turns out to be actually a very similar problem. Like you want to scale up containers up and down very fast. So we just happened to be like very well positioned to capture a lot of those workloads in the early days. Did tools for auto scaling like not exist? Like why did you have to build those yourself?

10:30 In general, like my thesis is like, you know, every new sort of era of like, you know, technology kind of runs up against limitations of tools. You know, whether that’s you look back, you know, like the web or the cloud or, you know, mobile or PCs or, you know, AI or whatever. So I think, you know, it’s often the case that tools exist, but, but they’re often not like particularly good. And often you kind of need to rethink a lot of those tools in order to reap the benefits of, you know, and really turbo charge and start building for this new world. And you saw this with cloud, you know, we have to invent things like Kubernetes and Docker and things like that. And in a similar way, you know, like with, with gen AI, you could build these things on the sort of prior generation of tools. Like you can, you can build these things on top of Kubernetes and Docker and in these things, but, but, but those technologies are built for a very different type of workload. They’re built for a very sort of steady work, steady back end workloads. They have an even, you know, steady number of requests, like running a pretty low utilization because CPUs are normally pretty cheap. And so people don’t really worry so much about cost. And, but now we’re entering this new world and you have, you know, very expensive GPUs you want to manage capacity and utilization. You have very large models that you need to load over network into CPU into GPU. And, and it turns out like a lot of these tools, like, you know, like they were just not built for that. So we felt that, you know, we had to rebuild a lot of that, you know, building our own, I mentioned the file system, but we also spent a lot of time over the last few years, like building, you know, some really cool stuff around like CPU

12:00 snapshotting, GPU snapshotting and audit, we built on autoscaler and scheduler and, and, and in all kinds of capacity management tools. And we kind of feel like you have to do that because like you can’t build, you know, for this new generation of software on top of, you can’t build like a wrapper on top of the old stuff in order to do the new stuff. You kind of need to start over a little bit from scratch. So I guess like the auto scaling, like did you, did you see any like weird behaviors on the platform? Like was there anything that like broke while you guys were scaling? I mean, capacity is always from time to time been, been a challenge. Like how do you get like, you know, hundreds and thousands and tens of thousands of GPUs? So that’s, that’s always challenging. We had to go multi region very early. So I think that was like kind of a fateful decision in hindsight, like doing that and kind of just biting the bullet and building it. Like we always thought of like our capacity management as like one big pull. Like it’s like one big pull over the entire world. Like we don’t think of it as like separate clusters running in each region. And so that was like one of the like very consequential decisions we made early to kind of think about it that way. So I don’t know if there’s like been like, you know, specific issues. I mean, we’ve obviously had bugs and, you know, outages and I hate, you know, as much as I, you know, would love for a model to be a hundred percent uptime, you know, and I think recently we’ve basically hit four or five nines, but throughout the history of model, of course, we’ve had, you know, some run up against scaling issues. Yeah. The GPU market and well, actually before that. So you guys use AWS like GCP, like all these cloud providers underneath.

13:30 Like do you also buy any infrastructure? No, let’s talk a bit about like cloud capacity. Because I think it’s actually changed a lot. Like in the first few years we did, we got almost all the capacity pretty much from, from the hyperscalers spot and on demand capacity, mostly spot capacity, running in all the different regions, you know, using initially AWS, but also GCP and overtime measure. We started renting some capacity from Oracle pretty early because we wanted the eight tens and eight 100s and we got really good rates on those. And over time we added a couple more clouds, like doing reservations, but, but like still up until like actually end of last year, like most of our capacity was either spot or one year reservations. Spot means you’re just like renting it right there and then. Exactly. Yeah. And it’s also preemptible, which we had to deal with. But, but it sort of works in Agria because you can sort of just, you know, aggregate capacity from many, you know, hundreds of regions basically. So that, that, that model worked really well until we started running up again. I mean, first of all, like modal started growing like very fast as we needed more capacity, but fundamental also like the cloud market started changing a lot. So in the last year, basically now in order to get the capacity we need, we had to work with a much larger set of clouds. We actually work with almost 25 clouds at this point. And in many cases, in order to get the capacity we want, like we, we have to go into, you know, three plus year deals and you have to pay a meaningful amount of money upfront. And you’re locking in pricing on a three year term, right?

15:00 So, you know, like, and, and, and so we have to spend the entire, you know, last six to nine months, like, you know, kind of rethinking how we get capacity. We have to build a whole procurement like desk, like we have like four guys working full time basically, all they’re doing is like procuring capacity, calling Neoclouds, you know, doing deals, vetting the capacity. We’ve had to build a whole engineering team, basically 10 people just like managing like bare metal capacity from all over the world and dealing with network issues and bad drives and GPUs falling off the bus and, and all these things. We don’t own capacity. We rent all the capacity so far, um, you know, on, on, on typically this three year deals. And that’s like a big decision. Like, should we buy, should, should we, you know, or should we keep renting? I’m actually like not like mega dogmatic about it. I think renting is nice because there’s a certain amount of capital efficiency to it. If you’re buying, you know, like, I mean, first of all, there’s not this like operational complexity of it. Like you have to, you know, deal with vendors and suppliers and, and, you know, power and, and, and cables and all kinds of stuff. But there’s also, you know, financial complexity to it. Like, how do you fund it? Like, should you raise equity, you know, and, and you can raise a billion dollars and go buy hardware for it. That’s a very dilutive way to do it. Um, you can raise debt, but you know, if you raise that, you know, you typically want to have like locked in long-term revenue to, to, to pay the interest rates. That’s also not trivial. But you know, like we look at the economics, we look at, uh, the operational

16:30 complexity, we look at the risks. Uh, it’s a complex trade-off and, and we’re open to, to, to buying. We’ll, we’ll sort of, there’s other constructs too. Like we’re, it’s to get the capacity we need for, for 2027. It’s going to take a lot of work on, on, you know, going a lot of different avenues to, to lock it up. So do you have to hire a lot of like finance people to do all this stuff or not? Really? You know, I’ve had to spend a lot of time on this and we’ve had to, to, to spend a lot of time on this. And I do think over time, I mean, there, there is a lot of, you know, these are very expensive assets, uh, the hardware. And, you know, I think, I mean, it’s kind of staggering. You starting to see like AI investments starting to become like several percentage points of, of, of GDP of the U S and, you know, starting to see like interest rates in, in America, shooting up in the last few weeks. And I mean, I like, at this stage, like, you know, and you go and look at the Neoclouds, like these are very, very sophisticated financial engineering companies. And so yeah, we, we look at that and, you know, we look at all the options and, and it’s possible we might have to do similar things ourselves. I wouldn’t want to like, I would rather focus on being, you know, like a very lean software company, like doing the things we’re good at. I don’t think it’s good in general for software companies to own hard assets. You know, in the same way as like, should you own your office? Like I don’t think that’s necessarily the right solution for most companies. But of course, like in order to get the capacity you want, like, you know, everything is on the table. Yeah. And I guess the supply is going to be constrained for a while.

18:00 So you kind of have to do that. Like, do you, do you agree with that? Or do you think this is just like a moment in time? I don’t necessarily agree with that. I mean, like Nvidia is delivering, you know, several hundred thousand black walls, you know, a month, right? And, and I don’t want to say like the market is like 100% efficient, but like, you know, in general, like if you’re willing to pay a high price, you can get GPU capacity, right? Like if you’re willing to pay more than everyone else, and, and of course, Anthropic and OpenAI is buying up as much as they can. And, but fundamentally, like there’s constantly supply coming on the market. And so I don’t necessarily think it’s like a fully cornered resource. Prices are going up. And I do think it’s worth kind of reflecting on that. It’s like the normal state of hardware is like, it’s like out of like gravity. Like eventually, hardware prices always go down. So we’re a little bit in like a weird sort of period of time where like hardware prices are going up. And, you know, it’s hard to say, you know, how long this is going to last, you know, could be another six months, could be another 36 months. Like it’s really hard to know how long. But I fundamentally don’t know if like it’s true to say, like, you know, you need to buy capacity in order to me, that’s a little bit to sort of almost like the great toilet paper shortage of the pandemic or whatever. Right. Like, you know, at least this like, you know, run on the bank type situations. And, but that being said, like, you know, capacity is going to be tight for a while. And, you know, prices are going up. And, you know, luckily, I think, you know, it’s not even necessarily bad for us. If prices go up, because, because, you know, scarcity also means like we can

19:30 charge more from our customers. And, you know, and the fact that we have a lot of reservations locked in at a rate means like, to some extent, we’re effectively like long GPU prices already. So I don’t necessarily worry too much about prices going up. I do worry more about actually prices collapsing. I don’t think that’s going to happen anytime soon. But I think that would be a quite bad thing for a lot of the players in the market. Why would that be bad? I think it would be bad for if, for anyone who’s locked in a lot of supply. Yeah. Because they paid way too much and they paid too much, you know, so there’s a risk of buying at the top, right? And again, like, I don’t worry about that much. Like, I don’t think it’s going to be like, you know, it’s going to happen anytime soon. But like, you know, like I said, like hardware prices in the very long run, they tend to go down. And so it’s, it’s worth, you know, reflecting on that, right? Right. Like right now, I think, I think your, your capacity is constrained and, and that leads to sort of, you know, in economic terms, there’s a scarcity rent, right? Like who captured the scarcity rent? I think the primarily goes to two places, mostly Nvidia, but of course, but also to some extent, Neoclouds, like Neoclouds and in general, like, you know, anyone owning capacity has better economics. Like that’s undeniably true today with capacity being constrained. Long term, you know, in the very long run, I don’t think that’s necessarily entirely true, right? Like there’s always going to be a little bit of a premium you can get from owning GPUs, but is it worth the risk? You know, is it worth the risk of like holding this like very expensive

21:00 assets on your balance sheet? I’m not so sure, but you know, we’re going to see, like it’s going to take a few years to get through this. And, and, and in the long run, you know, we might own some, we might rent some, maybe a combination, maybe do some other crazy stuff, joint mentors, who knows? Maybe some complex SPDs, whatever. Um, everything is on the table. How do you compete with open AI and enthropping and buying GPUs capacity? I’m assuming they’d want to like buy up literally everything that’s available. They are. And, and I think what you’re seeing is, um, you know, a lot of that, the, the price is going up. Is it a reflection of that? That being said, you know, NVIDIA is shipping a lot of stuff. I think also like to some extent, like NVIDIA is like kind of prefer a little bit of a diversification. Like they don’t want like, you know, small amount of large buyers in much, in the same way as you see, you know, hyperscalers getting a little bit of a constrained allocation. Uh, I think, you know, also NVIDIA prefers to see, you know, a lot of the neoclouds and a lot of alternative clouds, like getting more allocation. And I think a little bit, what we see in the market is also this is more speculation, but I think we see, you know, smaller clouds that are a little bit more sort of risky to work with being more of an opportunity for us. Like where, you know, maybe NVIDIA, sorry, enthropping and opening AI, my feeling is like they prefer to buy from like the, the, the try and through, uh, part of it is also because they had to do a lot of training runs. So they need to have like very large blocks, but there’s plenty of capacity out there, you know, you can get 32 nodes in Armenia.

22:30 You can get 128 nodes in Indonesia. Like, you know, like we love that kind of shit. Like, you know, so we can go out there and like get a lot of, you know, smaller capacity all over the world. Pretty good prices. Is it pretty much just like if someone offers you like a certain number of GPUs, like that’s all you care about. Is there nothing like, oh, you care about like how reliable it is, you know, is it just some like janky warehouse that someone has these in? I mean, sometimes it’s kind of a janky warehouse. It feels like, you know, but, but like, we do have like a spec sheet, basically. Like when we talk to vendors, the first thing we do is we send them a spec sheet. We say, we need this number of IOPS. We need, you know, like the CPU needs to be at least, at least, you know, have, have this many, you know, whatever the pass mark scores or whatever it is. We need to see like, you know, minimum, you know, North, North, South, bandwidth, egress and ingress. So we do have a list of requirements. Often we try to do like an acceptance testing period where we get like one node and, and runs and tests over time, you know, like to some extent, it also involves a little bit of a leap of faith. You know, sometimes we sign very big commitments, you know, 10s or 100s millions of dollars and we start working with the vendor that we still don’t fully know. And, you know, over time, like, you know, we’ll see, you know, and, and, and I think over, like, in a little bit of, like, there’s a little bit of sort of just, you know, our own sort of, like, you know, how like semi analysis has like a cluster max, I kind of feel like we almost like have our own, like modal max score internally. Like, I know which vendors I like more than others, you know, like, and, and,

24:00 and we sort of prefer to do additional deals with the vendors that we’ve seen, you know, very good reliability, not just that also, like it’s not just reliability, it’s also, are they very responsive when, when we have, you know, support issues, do the fixed issues we ask for to turn around time. If things go down, you know, how quickly do they fix it? Like things like that. Do you agree that the demand for intelligence is basically infinite? And if so, like, why wouldn’t you just buy up as much as you, as much capacity as possible? And then, like, if you over buy, you can just like sell it to somebody else. I think it’s very easy to say when there’s like a scarce resource, like, let’s just buy as much as you can. But I think, you know, first of all, like, if we bought it today, we couldn’t utilize all of it. Like we could use utilize a lot of it, right? But like, But you can sell it off to somebody else who wants to use it, right? I think the challenge is like, if there was like something in the market that happens, there are a lot of correlations that happen at the same time. If, you know, prices going down and utilization going down and the market for reselling capacity goes, vanishes at the kind of the same time. I’m sitting here and like, I’m extremely aggressive, like buying capacity every day and like raising money to buy, you know, so overall, I’m, I’m very bullish, but like, I do want to point out and like, you know, like there are tail risks here. And when those tail risks happen, like it could be get very bad. Part of it is like, maybe I’m, you know, I’m just old and I’ve seen this in the past, like I actually had my first job in 1999. So I saw the dot com crisis and I’ve seen the great financial crisis and things like that. And, you know, like, I want to build a company that, that is around for like 20

25:30 to 30, 50 more years, right? Like, and so, you know, I think we can certainly go out and like buying an insane amount of capacity right now and trying to just corner the market. But, you know, do you know about the Hunt Brothers? It’s like a legendary story in trading. They try to corner the silver market. It fails spectacularly. You know, so it’s a cornering the market. I think in general, like it can be not, not a good idea. And, but, but we are in general very, very aggressive, but we’re ramping up, you know, like at an aggressive plan that basically matches our demand growth and matches where we want to try to get to at the end of next year in terms of run runway or run rate. Are you only buying up NVIDIA stuff or also like, you know, maybe a capacity from other types of companies like training them and we’re looking at it. I think the alternative accelerators are starting to get very interesting. It’s, you know, ironically, like every, every year in the last five, maybe even 10 years I’ve been saying, you know, like next year, it’s going to be the year of like non NVIDIA accelerators. And then I’m always wrong. But, but I do think 2027 might be the year of non NVIDIA accelerators. Finally, so we’ll see if I’m wrong this year too. But so I, you know, I am these are starting to get like really good scores on certain things. I think, you know, TP use are obviously used very, you know, widely internally at Google. I seem to work really well. I don’t know much about training. I don’t feel like I see quite as, but, you know, the team is very good. I know they’re working on a lot of the software stuff.

27:00 You know, hardware is hard in a certain way. Like software is like easier. And that makes me feel like, you know, if we just fix the sort of software layer for these accelerators, like, I think there’s going to be a lot of demand for them. And I think fundamentally, you know, with NVIDIA being such a bottleneck, like, we’ll probably see, you know, more alternative accelerators. That being said, you know, there’s bottlenecks further down the pipe. I mean, you know, there’s like TSMC and ASML, maybe like a deeper bottleneck to all of this anyway. But, you know, long term, I think monopolies don’t always last. And so I think you’d be healthy for the market to have more alternative. As much as I love NVIDIA, like, to make that clear, like, I mean, they’re obviously an amazing company and within the amazing product. I do expect to see a little bit more diversity in the market as well. Also, things, you know, like Cerebras and Grock and these things. Yeah. What are your thoughts on like some of those providers? I think they’re pretty exciting. And, you know, and you see them doing really well on certain workloads. And, you know, there’s some interesting, you know, people doing these like hybrid, you know, disaggregated inference where they run, you know, NVIDIA for the pre-fill and Cerebras for the decode and things like that. Oh, I haven’t heard of that actually. Yeah. And, you know, Grock too, like, because a lot of these things, you know, certain things are compute bound, certain things are memory bound. So you can sort of mix and match. And I think that’s quite exciting. It starts to get very complex when you’re deploying these things in practice. But I do think there’s a lot of, there’s a lot of, you know, opportunity and these sort of, you know, more complex hardware setups. Are there any learnings that you get from some of these Neoclouds, like, I don’t know, like Nebius or whatever, and like the way that they’re doing things versus

28:30 like what you’re doing? I think one of the things that, you know, is very interesting with Neoclouds is like, how can they expand so quickly? I think actually the most crazy example is like N scale, which went from like nowhere to like, you know, having like, you know, whatever, hundreds of billions of dollars in the pipeline, despite barely having any capacity. How can Neoclouds grow so fast? I mean, there’s like an, you know, and basically the answer is like, you know, and this might be obvious, but like what they do is like, they go out to like, you know, enthropic or open the eyes and some big buyer or whatever. And basically say like, Hey, do you want to buy like five years of chips? You know, sign this contract, we give you a good rate. And then they take that contract and then they go to, you know, Wall Street and say, look, we got this like, you know, big buyer, you know, they committed to $10 billion worth of chips. We just need to monitor it right now. And then they go on to raise the money. And then they like, you know, go to like Super Micro and DAL or whatever in the video and like get the hardware and then they install it. And then, you know, now they have like, you know, delivered to the customer. Right. So you have this like very nice sort of financial construct where you can basically sort of pre-sell things that you haven’t even bought. And that means, you know, you can scale up extremely fast. And so that’s what we’ve seen with these Neoclouds is, is, is they can scale very, very well. I think modal because we’ve always had a little bit more of a PLG focus and, and not the sort of, you know, ability to lock in long-term contracts in the same way, it’s a little bit more challenging from a financial perspective to scale that fast, even though I think, you know, in the long run, like we’ll get to the

30:00 same point. But I think that’s been kind of a fascinating thing to see with these Neoclouds and especially like CoreWeave and those big ones. They’re very, very savvy with, you know, how they raise money and how to, you know, think about GPU as collateral for loans and all these things. What’s the relevance of, like, I forget who it was, but like some like financial company was like, oh, we’re going to make, oh, I think it was Nvidia. Actually, they were like, we are going to turn GPUs into an asset class. Is that just because like they need to raise money and they’re like trying to figure out where to get the money from? Like, what’s the relevance of that? I mean, there’s a bunch of startups. I don’t know if you were thinking about that. There’s like Silicon Data and computer exchange and like Orn and, and, and a few other ones that are like, basically like trying to get to like, you know, creating price indices and indexes and trading derivatives and things like that. I think it’s interesting. Like, I think with compute, like it does, there are a number of challenges with compute as an asset class. Like it’s definitely not as homogenous as other asset classes. You have different network and different locations and, and, and different disks and CPUs and all kinds of stuff and reliability. I mean, that being said, like you can say the same thing about oil and like we kind of created like, you know, relatively commoditized, you know, tradeable assets out of oil. It remains to be seen. Like I do think, you know, at the scale where we are with the sort of AI, like data center and GPU deployments, like we do need to be a little bit smarter with like risk hedging and, and, and how we share risks and, and, and, you know,

31:30 having more like sort of insurance and, and things like that. So I, I, I’m very happy that people are thinking a lot about this. That being said, like I’m, I’m not entirely sure if we’re going to see, you know, GPUs becoming sort of, you know, whatever it is, sort of Henry hub, like, you know, Brent crude type, like, you know, super commoditized, like tradeable asset. Yeah. I want to shift the conversation to modal sandboxes. Yeah. I guess like before we do that, what are the different products that modal has? It’s, I guess like the main product obviously is like the, the GPUs. Then you have modal sandboxes. Is there anything else? I think of the like the two big products that we have as basically number one is modal functions and number two is modal sandboxes. So, so with modal functions, we launched that, you know, back in 2021. It was the idea of like serverless functions that are very easy to deploy and very easy to scale and we managed all the capacity. That turned out to be really, really good for inference and it turned out to be really good for managing GPUs and running on things like that. But we actually never had like, you know, dedicated like inference product or like a GPU product per se. Like we never thought of modal as like a neocloud and I think there’s like a risk of like getting sucked in that direction and becoming a neocloud, like just selling, you know, sort of what I think was like rectangular, like here’s like a three year. You get this many GPUs. Models has always been very good at like you deploy something you can kind of scale it up. You have this like very cyclical or like bursty stuff. Like we manage all the capacity. And the beautiful thing is like, because we have many, many thousands of customers,

33:00 we can sort of aggregate that capacity and get very smooth workloads, you know, very smooth aggregate demand, which we can manage very, very efficiently. So modal functions was the core. We launched that in 2021. Like I mentioned, we started seeing product market fit in 2023. And now, you know, it’s about two thirds of our avenue and empowering a lot of, you know, in practice, it’s mostly inference, but you can use it for all kinds of other stuff. Modal sandboxes, we launched that in 2023. And it’s actually kind of, you know, took a while to see product market fit similar to functions. Why did we launch it? We realized a lot of people we talked to were using LLM to output code. This is like me and Akshar talking over lunch one day. We’re like, maybe they need a place to run that code. And we realized like, actually, we kind of have like a lot of the infrastructure already to just take code and run it. Because we’ve already built the primitives for like managing containers and isolation. We already had, you know, G visors and isolation primitive and things like that. So we get kind of excited about that idea. And Akshar wrote a prototype and launched it in basically one day or two days or something like that. Zero. Like we saw like very little like growth. I don’t want to like trivialize it. Because we did see some growth. Like there were like some sort of, you know, And it was 2023. So yeah, it was 2023. And like, you know, like some people picked it up. Like we had some early users who kind of enjoyed it for certain things. But like overall, like it was like a pretty insignificant source of revenue into going into the summer of 2023. So two years later, we even discussed like, do we even need this product?

34:30 Should we shut it down? Then it just started exploding. So 2023. And I think what really happened 2023 was like, LMS got really good at generating code. In 2023. 2023. I mean, it was like around that time. Like I feel like that was like when they went from like being able to like refactor functions and like write snippets, so like generating like full apps. Right. And so we started seeing a lot of growth coming from like vibe coding, you know, companies like Loveable and other ones. And then, you know, companies like ramps started using model for background agents. And then we started seeing tremendous growth coming from a number of Neolabs doing things like reinforcement learning on model sandboxes. So sandboxes is now about a third of our revenue. You know, we considered it like basically like our second product. And to me, it’s like really exciting because like even back in the early days of model, I always knew that model, you know, is about like creating a whole toolkit of products. Like we always wanted to be almost like a cloud, you know, you have the storage and the compute and a bunch of different ways to work with it. We always wanted to be a multi-product vendor, but that wasn’t necessarily true in practice until like now. We’re like, I think now it’s like kind of clear. We have functions, we have sandboxes. We’re working a bunch of other stuff too. We’re working on training, working on batch, we’re working a bunch of other stuff. So to me, that sort of evolution into becoming like a multi-product company, I think it’s been amazing because that was always so core to the vision. Even in the very early decks, I had a bunch of slides on this. But it wasn’t necessarily true to the external world until recently.

36:00 Do you have anything in the works that you think is going to be like that next product? Any predictions maybe about the market? Anything like that? I mean, I’m very bullish on training. I think training is like kind of an underserved market. Obviously, there’s like a lot of focus on like very large training runs. But we actually see a lot of demand, the sort of smaller and mid-sized training segment. So I’ll be talking a little bit about that later this week. We’re organizing a conference. Batch is I think like a really interesting market. I think we see a lot of demand for sort of, you know, customers that need a... You know, we had a customer the other day that needed to transcribe, you know, 5,000 years of audio. They had like, you know, millions and millions of audio files. And we had a lot of capacity. So we let them do that, you know, and took a few days. I think there’s some really interesting use cases there. Very bullish on like low latency stuff. I think there’s going to be a lot of, you know, once we sort of get the audio, like the speech-to-speech models working, like the voice models. I sort of expect that to be very interesting market. Then of course, like all this like agente stuff, right? And in a way, like I have like mixed feelings about like the word agente, because I feel like in certain ways, like, I think all software will become agente. So it’s almost like to me, like, what does it mean to be agente? Like I think all software in a certain way, like will become like AI powered over time. But like clearly there’s like a set of like building blocks of like, you know, doing more sort of LLM powered software that seems to be sort of, you know, common across all the different use cases.

37:30 And we’re like, you know, the way I think about models, like we’re in the business of building building blocks, you know, we want to give you like amazing, generalizable, battle-tested, you know, general purpose, composable building blocks. And I think there’s some really interesting stuff also, like, you know, with the agent use cases. That’s interesting. You bring up training because I think in another podcast, maybe like a year ago, maybe like two years ago, you said training you were not focused on like specifically, like what changed in the market? I mean, we’ve been building a training product for multiple years on and off. So I wouldn’t say we’re like, we’ve never been focused on it. But it’s like, it’s like one of those things we’re like, I mean, now we’re like 150 engineers, but, you know, a year or two ago, we were a much smaller number of engineers. And so, but just virtue of, you know, being smaller, we had to focus a little bit more. And so inference became like our sole focus and was our sole focus for many years. But we’re bigger now. Like we, we, we do have opportunities to also place bets in other markets. I’m also really interested in, you know, kind of continuing the list of things. Post training, reinforcement learning. I think there’s some really exciting stuff around that. I think there’s also many other like stuff around like storage and building data warehousing and feature stores and real time streaming pipelines and all kinds of like, I don’t know. I think the longterm view of, of modal is like, basically like, you want to build some cool AI shit, like, you know, we got you covered, like, you know, we got all the building blocks, like comp tasks and like, we’ll help you, you know, take all these building blocks and compose them.

39:00 And that’s the beauty of software is like you, you can, you know, build almost anything if you have the right building blocks. So that, that’s very much the sort of company and the product that want to build. When you say training, are you focused on more of like the bigger enterprises doing huge training runs? No, no, they’re going to go and like, you know, do these like, you know, big, hyper-scaler or rather Neo, Neo cloud mega deals and get, you know, 100,000 GPUs or whatever. But, but, but there’s like a little bit of an underserved market of like, in my opinion, you know, like researchers iterating or like people training smaller models, there’s, you know, Texas beach and speeches, text models, computational biotech is actually a market that we see a lot of demand from. We have a lot of customers in that space training relatively, you know, modest models. You can train on 64 GPUs or whatever recommendation system, a lot of sort of traditional machine learning. There’s a lot of things like you don’t need a, you know, 100,000 GPUs. That market, I think is like, we’re not going to like focus on that for a while. But, but I think there’s a lot of stuff in the, you know, slightly smaller markets that that are relatively underserved. It’s very hard to set up training clusters. Yeah. Have you noticed like since, I don’t know, like GLM 5.2 or like, maybe like, can we 2.5 that, you know, a lot of people are using those and like fine tuning them? If anything, I’ve actually seen the other direction. I feel like fine tuning the value prop a couple of years ago was like, yeah, you know, open source models, they’re not quite as good. But, you know, if you take, you know, a small model and you fine tune it on your specific

40:30 domain, you can actually reach the frontier in terms of, you know, quality and cost. Today, I would say like the open source models are so good that like you can basically just like swap them out and just run, you know, instead of, you know, the frontier ones, you can run, you know, Kimi K3 or whatever, DeepSeq 4.1 or GLM 5.3. And they’re like basically almost as good as like the frontier model at a much lower cost. So we still see a lot of demand of for post-training. And I do think it’s a really interesting space and we’re working on some products in that space, you know, both sort of SFT and but also reinforcement learning and things like that. I would also say, you know, fine tuning is no longer the thing that you kind of have to do. Like it’s very optional. Like you can open source models like work really well just off the shelf too. Back to sandboxes actually. I forgot to ask you. So like what did you have to update about the sandboxes that you guys were offering that like existing sandboxes did not did not allow for? Not a whole lot. I think the thing we’ve always been very, very good at, I mean, we’re infrageeks, right? Like we’re very good at like, you know, scaling and reliability and doing a lot of stuff. And I think that’s why that explains to a large extent like why we have, you know, in my opinion, I don’t have numbers, but like I think we have by far the largest market share in this space. Like we’re just very good at running, you know, hundreds of thousands or even millions of sandboxes. Yeah. I will say we’ll have to, we’ve had to rewrite our systems, you know, a bunch of times in order to handle that scale.

42:00 But we were always like very good at it. There are other things like over time. I mean, the sandbox space in general, like, you know, like it’s like not the most like differentiated space, but you know, we’re working on some really cool stuff. Like over time, like we’ve had to add support for VMs, like virtual machines. So we previously bought our sandbox product on GVisor, but now we offer full virtualized machines as well. Turns out a lot of for certain use cases, you kind of need full Linux compatibility. There are other things we’ve had to build, you know, better primitives for like more interactive, you know, piping data back and forth between sandboxes, working with file system. We’ve had to spend a lot of time on snapshotting. I think a lot of the things that over in the long run will differentiate will be like, you know, really cool things like, you know, forking sandboxes, snapshotting, resuming snaps, like live migrations. In general, like I think sandboxes, the novelty of sandboxes is always, you know, this sort of ephemerality, like you can spin off sandboxes for a few seconds. And that’s something I like, that’s a string I want to pull more in. I think there’s a lot of really cool stuff you can do, you know, kind of decoupling the agent and the harness decoupling it from sandboxes, having agents control many sandboxes, forking sandboxes, you know, snapshotting things like, you know, like running basically like very short lived, maybe one per command, like things like that. So I think there’s a lot of cool stuff there. We’ve already spent a lot of time doing that, you know, around snapshots. But I think there’s a lot of cool stuff to be built there as well. You brought up snapshotting a couple of times.

43:30 Like what do you see as like the use case for snapshotting? Like what would that give us that we don’t get right now? I think the big thing for me is like today a lot of sandbox workloads are kind of like lift and shift. Like people think a lot about sandboxes almost like, you know, I have my dev box, now I’m going to shove it in the cloud, I’m going to have an agent control it, maybe I’ll run the harness inside the sandbox. I don’t know if like long term that’s like the right way to think about, you know, on like compute for agents. I think a much better model would be you’re going to have this like very long running agents, which almost like to me are like kind of durable execution workflows. Like they’re going to run, you know, 24 seven, you know, every once in a while they’re going to run commands and, and you know, having the ability to like spin up a sandbox in, you know, 100 millisecond, run a few commands and then like shut it down and like snapshot it. I think the faster you can get that like, you know, wake up, you know, go to sleep cycle, the more you can sort of create sandboxes that only run for like a very brief period and that will make compute much more efficient. And it will also enable you to do some really cool things, you know, like forking and moving things around and, and, and, you know, having like imagine, you know, like Claude code instead of, you know, having sort of a very linear flow, like sequential flow, like kind of branching and, you know, running hundreds of agents at the same time. I think a lot of like CI is like another like illustrative example, like for most of the, the history of CI, you know, talk about like GitHub actions and things like that.

45:00 Like you had a very sequential flow, like, you know, you slowly install a bunch of stuff, then you run the test suite and then, you know, generates some reports or whatever. And it’s like, you can break it up into shards, but it’s kind of annoying. Like what I would love to do is like take your entire test suite and just run every single test in its own sandbox, right? Like, and just paralyze the shit out of that around like a thousand sandboxes. Why can’t that be done? I mean, the problem is like there’s too much, there’s too much overhead around like, you know, starting and stopping sandboxes. So I think the more like you create like, you know, frictionless, like starting and stopping, the more you can enter a world where like you can do this like extremely bursty, like large scale, like run as much as you think you want in parallel. So, you know, I’m an infrageek, like I said, like I get very excited about these things. You know, once we figure out all these primitives and the right building blocks, I think it’s going to unlock a lot of really cool stuff that, you know, we previously could not do with software. Why hasn’t that already been built? I would assume even previously people would want to parallelize as much of their CI as possible. You can’t, but the overhead, it’s like Amdahl’s law or whatever. Right? Like the overhead of parallelization right now is very high because like right now, like you can get up actions, for instance, like we parallelize in the modal test suite, right? But like what that basically means is like now if you take your test suite and you split it up in two, like now each of those shards both need to do all the installs and the test setup. And that takes a long time. Like it can take minutes, but let’s say you can do the test set up once

46:30 and then you snapshot the container and now you clone it and now you can run the test suites independently. I think a lot of the benefits cannot be, you can’t materialize those benefits until you get the friction and the overhead down, the transaction cost. I think you guys have a partnership with Anthropic and they’re using sandboxes for something, like modal sandboxes. Yeah, I mean the managed agents with Anthropic, this is sort of bring your own sandbox integration. So you can use modal for those, yeah. How did that come out? Like come to be? I think Anthropic is like trying to create like a thriving ecosystem and community and so I think they make money from inference, right? Okay. I think it’s very much in their interest to promote, for everything else, as much of an ecosystem as possible and they want to have the largest possible surface area as long as you use Anthropic for inference. So I think it makes like very rational sense for them to partner with us as a sandbox provider. Also, I feel like we have to bring up the hugging face incident. Are you okay talking about that? Yeah, yeah, I mean it’s all there. Yeah, yeah. So yeah, could you just briefly describe like what happened and I think they were using the modal sandbox but it wasn’t your, you know, it wasn’t like a fault of the sandbox. Yeah, exactly. I forget all the details of the exploit but I mean, you know, I think what you’re referring to is the fact that like modal was involved. Yeah. And so, you know, just to comment on that, like basically the agents running inside,

48:00 you know, that exploited hugging face, they discovered a vulnerability in an app that was deployed on modal and they used that to sort of, it’s almost like a springboard for getting access, getting full internet access is my understanding. I don’t remember the full details. To be clear, this is an app running a modal that was hacked. Like it was not modal that was hacked. So it’s like AWS, a lot of people run stuff on AWS, occasionally those apps get hacked. Does that mean AWS got hacked? Like no, of course not, right? And so there’s a lot of people running apps on modal and, you know, one of those apps got hacked. And I think, but like there are some lessons we can learn and I think one of the things we’ve been reflecting on is like, could we have more sensible, you know, security restrictions by default, I think to use the AWS analogy and just as another analogy, you know, S3 used to be this, like, you know, it used to be very easy to accidentally make an S3 bucket public and AWS realized like, you know, it doesn’t necessarily cause it, you know, poses security problem for AWS, per se. But a lot of their customers accidentally would, you know, like leak data. And so S3, it’s much harder now to make S3 buckets public. And so in a similar way, it’s like, we’re exploring like, you know, can we make security a little bit more restricted by default without necessarily, you know, destroying the magic that makes modal good. And I think there’s also, you know, it’s kind of a separate sidetrack is like, agents are getting really good at cybersecurity. So we’ve actually also, as a sort of, you know,

49:30 separate sort of learning from this incident, like we started using agents internally quite a lot to try to find vulnerabilities so far hasn’t found anything, but like, they’re very good at like, probing into like very deep parts of the system and exploiting, trying to exploit things and trying to escalate and get out of sandboxes. So, you know, it’s, it’s, it’s something we’re exploring very actively right now. How exactly do you get the agents to like run through the code base? You’re just like, hey, like find security vulnerabilities? Well, no, you just like, you just put them in a sandbox and tell them like, can you try to explain? Just run through the escape from the sandbox. It’s like, escape the room, but for agents. Cool. When did you start doing that? We started doing it a few months ago. And it’s pretty exciting. Like when Fable, I guess, came out. Yeah, roughly around the same time. Okay. I mean, my view on like cybersecurity is that, you know, I actually think they’re very good at finding vulnerabilities, but I think you can also use them defensively very well. So I, I, I think, you know, companies like modal and many other companies right now, like, I think, you know, if you go to talk to like pretty much every system, I’m sure they’re all thinking about the same thing. Like how do we use agents defensively to, to, to, you know, fix all our vulnerabilities? So over time, actually, in a way, like I’m not that nervous. I think if anything, like agents and cybersecurity, all the breakthroughs in cybersecurity will, will make the world safer, if anything. Do you think that like all of the cybersecurity, the big cybersecurity companies are just cooked? I don’t know if that’s true, because I mean, you’re asking something that’s like very far

51:00 outside of my like core expertise. But when I think about like cybersecurity, a lot of what you’re selling is effectively fear. And that’s almost like a branding and like marketing strategy, as much as, you know, obviously the core products. But I think, you know, brand and, and things like that are actually like arguably like more valuable in the air of AI. So I wouldn’t necessarily jump to the conclusion that they’re cooked. I think they have, you know, deep relationships with the enterprise customers and they have, you know, big sales teams are very good at telling, you know, spooky stories and, and things like that. And I think they’re going to continue to be very good at that. I just ask, because like, yeah, you’re doing it yourself, you’re not going to one of like the Palo Alto networks or whatever. But I guess like it’s only like giant enterprises that care about that. Also, they can blame somebody if something goes wrong. That is true. Yeah. Yeah. You kind of want to hire in many cases, just like an established vendor in order to blame them. I mean, to some extent, it does reduce a little bit the need for us to do, to bring in pen testers like human pen testers, which most startups like do on a regular cycle. So maybe previously, you know, we would do it, I actually don’t know what the cycle was, but, but I think we used to do it like once every six months, roughly. Maybe now we only need to do it once every year. I don’t know. But, but like, I think with all of these things, you know, humans can also use them to get more efficiently. So like, you know, like, I don’t know, I’m kind of a believer also that like humans with AI is even more powerful now. So like, maybe there’s an argument that like we can bring in pen testers

52:30 who are really good at using agents and, you know, find vulnerabilities even faster. I don’t know. I’m not that, you know, I’m an optimist. I started a company. So like, you know, I’m generally like optimistic about everything. But like, I certainly don’t think humans are cooked or companies are cooked. Like I think in the era of AI, a lot of companies are going to figure out, you know, how to use AI to be even more efficient. I want to transition a little bit to like developer experience, because like early on, I mean, even now, like you guys focus a lot on that. I mean, you’ve talked a lot about that. Does developer experience even matter anymore? I think it still matters. And I think you’re sort of, you know, like the context, I think you’re official, like the thing you’re talking about is like, now with agents, right? Like the, the developer experience, like the docs are already there. The agents are going to look at the docs, like all that stuff. To that point, though, like almost all the things that make developer experience good, also makes the agent experience good is kind of what we’ve seen in practice, which is like things like having good docs, having descriptive error messages, having fast feedback loops, right? Like, I do think there’s certain things have changed. The other day, I was like setting something up with Cloudflare, and it’s like, I find Cloudflare like notoriously annoying to work with sometimes. And, and I threw an agent at it, and it was able to like just configure a bunch of like, you know, things and pollute me for me. And, you know, and so I will say like agents does, they do make it a little bit easier to work with things that are like harder to work with in the past, you know, like the sort of human toll of dealing with the AML as arguably gone down a little bit, because you can just like,

54:00 you know, tell your agent to deal with it. But, but for the most part, I think, you know, the things that make developer experience good also make agent experience good. I think to some extent, also like infrastructure over time, anyway, becomes, it needs to be more programmatic. I think that’s always been the thesis about sandboxes, is that, you know, like things that you previously had to configure using YAML, that should just be an SDK. That should be more like imperative, like launch a sandbox, like do stuff with it, tear the sandbox down, right? Like, rather than sort of declarative using, using, you know, some sort of DSL, it also pushes like a little bit of a different interface that maybe previously didn’t exist. I think in either way, like SDKs are as relevant as they were, as they always were, if not more. Is there anything you changed in your product? Just because like, you know, most people are just running agents on it? We do. I mean, we also, we actually do run a lot of internal tests where we have agents, we tell agents to like, build a mobile app and run it. And I mean, a very concrete thing that is kind of funny is that we have actually seen that like, because of agent training data, it’s very hard to deprecate things now. We see that like, agents often like, love using, you know, old versions of modal syntax and it’s kind of harder to nudge them away from that. So we’ve actually had to be a little bit more conservative going forward, like, you know, over the last year or so with, with how we deprecate and remove features, it takes, we have to be very careful to update every single example that’s out

55:30 there online. Yeah. And, and, and, you know, skills, right? Yeah, there’s like stuff like that. But, but like, there’s also an insane amount of like modal code that’s now like online too that goes into the training data. I’ve seen like that to be like kind of a, you know, a little bit of like an interesting example where like, another thing by the way too is like, there’s obviously like to sort of a similar to like search engine optimization, there’s like agent engine, you know, whatever people call it, agent optimization. We often like, run internal benchmarks, we’re like, ask an agent, like, hey, like, can you deploy a stable diffusion for us? Use whatever tool you want. And we see if it recommends modal or not. And those things are very useful for us to sort of track over time. Or like, hey, spin up some sandboxes or like, hey, like rank all the sandbox vendors. So we spent a lot of time like doing those things to make sure. What do you do about that? Yeah, you say, hey, like, you know, build this stable diffusion thing, does it recommend modal or not? Like, yeah, what do you do about that? Yeah, I think that’s harder. I think, you know, it’s like search engine optimization, I think it’s primarily through content is like, you just have to put out a lot of content. You can also to some extent, like sort of see the perception, I think, a little bit more like you can ask an agent like, you know, how does, you know, rank all the sandbox vendors by reliability or rank all the sandbox vendors by price and sort of see, and it does feel like agents are a little bit more prone to like, misunderstanding pricing and, and, you know, things like that. So we also spend a lot of time like just making content more clear

57:00 and more like, you know, understandable for agents, because we know they’re all indexing, like everything that’s out there and that’s talking about modal. Have you thought about just like, making a bunch of GitHub repos and having the agents like write a bunch of modal code in there? Have you tried that? Slop cannon. I think the truth is like, that is probably something you can do. My guess is like LM training data today is probably quite insensitive to like, what’s like AI output in the first place and you can probably like create a lot of slop. Like we thought about it, like, I don’t love that idea. But like in theory, like much like, you know, SEO in the early days and still to some extent, like auto-generated content is like a very effective strategy. I think it’s possible. Like, you know, you could, you can do those things too. Yeah. I want to switch to FDEs. Apparently, you guys hire like forward-appoid engineers. Like, what do you do? We’re in a new era of like, building applications and, and, and, you know, things that we couldn’t build, you know, just a few years ago. And we see a lot of companies that, you know, they need a little bit of help, right? Like, you know, and so we started investing in FDEs a couple years ago, but it’s really been this year, it was kind of scaled it up a lot. I think we started this year having like four FDEs now. I think we have like 24 or something like that. So we’ve grown it a lot. And they do it, you know, it’s more, it’s not exactly the pound term model, but they do a lot of like heavy, you know, sort of customer engagement. Like they work with a lot of customers on performance tuning, especially with LMS. We see that there’s so much benefit from like, you know, sitting down

58:30 with customers and training draft models and tweaking, you know, parameters and DLM and SG Lang. They often, you know, fix bugs or not fix bugs, but like, you know, when customers run into issues, like they often work with customers trying to figure out like, how do we work around that? Or another thing we’ve used FDEs a lot for is like, you know, kind of proactively, you know, if we see certain customers, you know, over provisioning with model, like customers requesting too much CPU or too much memory, for instance, like that’s kind of expensive. And like, we try to catch those things and we try to sit down with those customers like, yo, like you’re spending a lot of money on model right now, we can actually help you save money. And then of course, like with sort of active deals, you know, we helped them migrate code from other platforms, things like that. I think, you know, we never have the intention of becoming like service heavy business. I think that the goal for FDEs is like, you know, A is to be a part of, you know, the sort of sales process and like closing deals. B is to serve customers and do some like, you know, customer support. But like, I actually think like the most important thing is like, C is like bridging gaps in the product that we later want to turn into like real features. So I think to a large extent, you know, anything an FDE does, you know, my question is always like long term, how do we turn this into product so we don’t need an FDE for this? Yeah. Do you guys hire PMs?

1:00:00 Product managers? Yeah. No. Would you say these FDEs are kind of like pseudo ish PMs? It’s a little bit of a different function. Like I think certainly on the more like, you know, customer facing side they are, but there’s like, you know, on the more like inward, you know, internal side, like this is sort of separate part of like product management, which is more like, what are the next building blocks Modal should invest in and like, you know, and then there’s like something in between like, you know, looking at like maybe 10, 20 different customers that are all asking for like slightly different things, but maybe what are the patterns? Like how do we turn this into product? We do have people doing product with us and it’s sort of evolved, you know, we’ve had people basically like become product managers after previously being either DevRel or engineers and showing strong aptitude on doing thinking very strategically about what we have to build. So I’m actually not like anti product management. I do think that a lot of companies in the past hired product management product managers too eagerly. And I think there’s a risk that you kind of, I can sort of see it in a lot of products like certain products like you can tell like, there’s just like eager product managers that just thought of like, kind of like, wigity things and just ship them. I think doing product management at an infra company really well is very, very hard because I think it takes like a very deep, you know, level of strategic thinking thinking about like, what are the foundational building blocks? Where do we want to be in one to two years? Because the truth is building infrastructure is very hard. And how do we make sure we don’t end up with like 35 features that are like,

1:01:30 kind of similar looking, but all do like a little bit slightly different things. That sort of level of thinking is something that, you know, is very rare to find. And for that reason, we haven’t aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving maybe like since Genialite? Because I would assume, yeah, I mean, you obviously worked at like Spotify, you had like a certain team structure, have your team structures like evolved a lot now that, you know, we have coding agents. In a funny way, like, yes, I do think team structures have changed a lot, but I’m actually not convinced it was like Genialite driving that. I think in particular, like one thing I’ve seen a lot is that there’s been sort of a focus, like a swing back towards being in IC that I actually think is very, very positive for the space as a whole. I think, you know, if you look back at the space like 10 years ago, 20 years ago, everyone wanted to be in management because that was the way you would get more money and more recognition and more prestige. I think that shift actually started even before Gen AI, but I do think it’s like heavily accelerated with AI. Like, you know, we make it very clear to everyone at Modal that like manager or RC, it’s like there’s no difference in prestige. Like everyone is just a member of technical staff. You know, it’s about ultimately your contributions and we see a lot of X managers, you know, becoming ICs and because they want to go back and build code. And so, and you know, and there’s been sort of shift away from like, you know, non-technical managers,

1:03:00 everyone, including me, I still write a lot of code at Modal and Akshath too and pretty much everyone in the tech team and a lot of people who are not in the tech team still write code. So that to me is like, you know, a recent phenomenon, but I actually don’t know if it’s like necessarily cost by Gen AI, but I think Gen AI has like greatly accelerated it because it made it even lower the cost of like writing code is even easier. So if you have like non-technical people like committing code, like how do you, how do you review that? Like how do you guys do code reviews? We do code reviews, but it’s, we think of it as optional. I mean, for most, my view is like code reviews in large part is like people, in many cases, trying to just escape the accountability of like, you know, merging things into production. And if you have that like, you know, if you have that culture, like then to me, that’s a sign that you need to invest way more in, you know, CD and CI and things like that is CI in particular, but also CD, containers integration, because it’s deployment. We do code reviews. Like, I mean, like when I write code, like occasionally, like I’m asking, you know, authors, like, is this in line with how you think about things? I hope I’m not changing the architecture fundamentally, but we also make it clear internally that like code reviews are quite optional. And, you know, if you’re just changing docs, if you’re just doing, you know, some basic stuff, like don’t wait for a human to look at it, especially in the era of AI, why do you need a human to look at every piece of code? You don’t have to. But that does require you to have, you know, very good controls, you know,

1:04:30 internally around, you know, testing things. Yeah, and exactly. Like, you know, infra companies, like we need one more nine than the average startup, right? In terms of uptime. But like, you know, these are investments you just have to make. Like, you have to build systems that are very resilient to like, you know, you have to have rate limits everywhere. You have to decouple systems. You have to make sure that like, nothing is a single point of failure. They have to have extremely good test coverage. Agents are actually really good at writing tests these days, which was like, you know, a few years ago, I thought like, one of the major things missing in agents, now they’re actually quite good at like, because I think it takes a little bit of like, sort of kind of like, need to step out of what you’re trying to do for a second and be almost a little bit like devil’s advocate to write good tests. But now they’re actually quite good at writing tests too. So I think, I think all these things help. Do you guys use like frontier models for coding or do you guys have you guys experimented with like, open source ones? We experiment with everything. But yeah, I mean, you know, initiative is a lot of cursor, then, you know, of course, Claude and Codex and now a lot of people are using, you know, night shift and all these open source stuff or other stacks. So I don’t necessarily have like a strong point of view, like I want people to experiment and try different things like we, you know, use frontier models, use open source models, we’re trying to do a little bit more dog floating internally. Now that we’re actually running inference for a lot of the open source models, like a very large scale. I think it’s also helpful to like use certain internal endpoints for a lot of agentic development. So we try to promote that a little bit. Something I read maybe like, I don’t know, a year ago, maybe like two years ago,

1:06:00 you were saying that you didn’t think that there was going to be like a big on-prem business for you guys. Have you maybe like changed your opinion on that? No, I have not changed my opinion. I think, moral to me, I’m a Claude maximalist and maybe to take a step back, like when the cloud came in 2008, 2009 or whatever, my first reaction was like, that’s crazy. Like why would I want to take my code and run it on someone else’s computer? That’s like insane. And then like six months later, I was doing it. And I think, you know, like took me six months to change my mind. It took probably enterprises like another 15 to 20 years. But like you look at, you know, vendors like Snowflake or I mean AWS and like, I think there’s many examples of Datadog, et cetera. Like the cloud is clearly winning, right? Like and the multi-tenant systems are clearly winning. And you know, to me, on-prem looks increasingly like sort of a legacy business model. And I actually think, you know, BYOC, you know, sort of VPC peering, like all of these things, you know, like to me, they reflect sort of an outdated security model of like, we need to protect everything inside our own network, right? Like using the network as a mode. And that I don’t think is true anything anymore, right? Like we’ve sort of moved on to more like actually robust security paradigms, like, you know, zero trust and things like that. And so to me, BYOC and those things, like they look increasingly

1:07:30 as like outdated, at least from a security perspective. From our perspective also, like the fact that we have full control over our entire infrastructure stack, we rent all the hardware, we deploy, we manage all the infrastructure, we deploy and like manage health and observe, you know, all the metrics and all the hosts. It means we can iterate very fast and build a greater product. And I think that’s very, very valuable versus, you know, customers in the past who had enterprise editions that they had to deploy on customers environment, they had to be very, you know, they have to think very carefully about release management and having the like, we don’t care. Like if we want to introduce a new database, like we can introduce a new database because we have control over it. Like we don’t need to think about what customers have access to. So to me, multi-tenancy is, is, yeah, has so many different benefits. I mean, that being said, like, you know, if we were to work with like, you know, some very large Fortune 500 customer in the future and, you know, they ask for BYOC, you know, we’ve done this in the past, we’ll give them a number. Like, you know, that number is very high and no one said yes to it so far. But like, who knows? Like, you know, at the right price, like I would still build it. And, and I think there’s like maybe separately, like another thing which I think is interesting, which is like, in theory, BYOC might also be a way to work with and get capacity in certain ways. Like we sometimes work with Neolabs. And those Neolabs have already have very large capacity reservations. And, you know, the question is like, you know, how do we, how do we get them to run modal when they already have capacity? And BYOC might be a solution. We can still keep the control plane in our environment.

1:09:00 And the other flip side of that is like, we could maybe even run like other people’s stuff on that capacity. So there’s some like, kind of fun stuff you could do in theory where like, you can sort of, you know, like kind of aggregate all that capacity into one big pool. So we’re excited about some of those ideas. But in general, like I feel like on-prem, I don’t know if we’re ever going to do. But BYOC, like maybe bring your own cloud. I can say BYOC. I should explain. Let’s say hypothetically, you are going to raise, I should tend more money like soon. What would you use that money for? I mean, like I mentioned, like in this capacity situation, the, a lot of the money, a lot of the cash flow, I should say, that’s required is capacity, prepayments, right? And so that to me represents the best use of cash today. We’re ramping up capacity very aggressively. Maybe we move into, you know, other types of deployment models in the future, ownership or whatever. But I think there’s always going to be a cash component of that, like some sort of down payment or prepayment or some sort of that. So that to me is the greatest use of cash right now. There’s of course other things like, you know, salaries and office spaces and stuff like that. But that actually in the grand scheme of like our financials, like it’s basically a rounding error, right? Yeah, interesting. Do you think that’s going to change? No, I think, you know, the sort of operating leverage that we have in this business is kind of insane. Like, you know, we have 200 employees, you know, they cost a lot of money, but, you know, the

1:10:30 GPUs cost a lot more money. Yeah. If you had to say, like, what are the top, like, I don’t know, like three things that are your biggest focus or the business’s biggest focus right now, like, what would you say those are? Capacity and the sort of financial complexity. And, you know, in general, like, how do we, how do we think about that space? How do we think about pricing and packaging? How do we think about the sort of tension between ownership and renting? Should we be going a more neocloud direction? Should we go in the opposite direction? The sort of counter position, like, that’s probably the biggest thing that I think a lot about. The second biggest thing I think a lot about is probably around, like, more like product strategy. Like, I think a lot about, like, what is the next building blocks? Like, if we, you know, do more like, agentic stuff, like, what are the things we need, you know, are there certain storage primitives or compute primitives? Or in general, like, what is the thing that the world is going to wake up and realize they need over the next few years? And ideally, we have that at that point. We’ve been pretty good in general at, like, you know, anticipating demand and building products in advance of it with the serverless functions in the sandboxes. So what’s the next thing? Then I would say, like, actually the third thing I think a lot about is, like, more on, like, culture and company building and stuff like that. I mean, we recently crossed 200 people. We’re growing extremely fast. We’ll probably hit 250 by the end of the year. Like, when you’re growing very fast, you have 250 people that are all trying to figure out how they can have the most value to modal and, you know, making sure that they have the right context, that they have the right motivation, that they’re excited to come into work every day because they get to build something incredibly cool

1:12:00 and they know, you know, what makes modal successful, what they can do, aligning the incentives and making sure that, you know, we, we, people have like a very positive culture where they have the sort of implicit trust to do a lot of stuff, but also the accountability to like, you know, deliver the best things and, you know, and be held accountable to like, if they’re not doing it, to me, those are like, you know, super critical things around this scale. Like, when you’re like 10, 20, 30 people, you can all just like, kind of sit in a room and like, it’s all kind of implicit and you kind of just like, feel it. People can sort of read, read me like, they can tell me when I get a little bit annoyed. They can tell when I’m happy, but like, now we’re 250 people and like, people don’t necessarily see it and, you know, of course, there’s like a lot of new people that are like, maybe a little bit nervous to like, talk to CEO and like all this stuff. So like, you know, how do I remain approachable and how do I, like, it’s, it’s, there’s like the Dunbar number, which I think is like more sort of, you know, faked than, than anything, like at least like the story behind it. But like, it is actually kind of real still, like there’s something around like 150, 200, we’re like, laying the foundation of the culture is very, very important. How many people are you guys trying to hire over the next year? We’ve grown about four acts on an annualized basis in the last 12 months. I think that’s like an unsustainable growth rate on a relative basis. I’ve seen that, you know, you can double the team every year and that’s not super hard to do, but four acts, what you start to see is, you know, cultural drift, you start to see islands of different working styles.

1:13:30 You start to see people doing like very speculative things like that. That’s something I always watch out for in the engineering team. You go and talk to the engine and engineer and I’m like, what are you working on? And it turns out they’re like building something like kind of like fringe and like, is this really what you should spend time on? So growing very fast has that cost and you know, it has the cost of ramping people up. So I don’t know. I think by the end of the year, maybe 250, about the end of the next year, you know, but like, you know, we’re going to keep growing like maybe two and a half, three acts. So it’s very likely we’ll end up like next year, like 700 people, something like that. I’m going to assume most of those are engineers. Most engineers. We’re wrapping up sales a lot too. Okay. What kinds of engineers? It’s primarily systems engineers. I mean, we’ve always been like very deep in the sort of, you know, Linux and file systems and scalable systems and networking and cloud and stuff like that. So that’s the core of what we’re looking for. Like people are really good at those things. We’re also looking a lot at like, you know, more like, because we moved up in the stack a little bit, like we’re doing a lot of work in the LMA in front of space. So now we’re dealing with a separate set of concerns around, you know, CUDA and like GPUs and and and MFU, you know, like maxing out the tokens that you can get per GPU and a lot of that things. And, you know, even though I actually like will say a lot of the work in that space, in my opinion, turns out is like more systems driven still, because it turns out like KV caching and routing is actually like, is still like very like systems like like these are very like systems have problems. But yeah, a little bit

1:15:00 more like sort of GPU performance, that kind of stuff. We’re starting to build up a research team. We’re very excited about, you know, moving into stuff around like, you know, post training and distillation and better methods for quantization for your customers or for internally. Both. I think it’s like a little bit like with the FDEs. Like I think we initially will do a lot of stuff with our customers to optimize their models. But over time, figure out how do we turn those into products? I think we’ll sort of arguably fail in a certain way. If every time a customer needs to optimize a model, we have to go and work with them. And, you know, long term, it has to be turned into products. And so with the FDEs and with the research team, initially, you’re going to have to do a lot of work with the customers and we are that. But over time, we’re also building a lot of internal tooling and a lot of over time internal products that are fully self-serve. So we’re working a lot of stuff right now in post training space, but also in the GPU performance space. So a lot of cool stuff. A lot of cool stuff. Back to the product strategy stuff that you were talking about. Yeah. Yeah. Like what are your thoughts? Like what do we need over the next year, two years? I think it’s incredibly hard to say. And if I said like, I know exactly what we need to build, I think I would be like delusional. I think what we’ve seen so far, and what I kind of believe in on a more general level is like, we need better building blocks. Like this is a new era of software and we’re unlocking a whole new type of applications that could not be built a few years ago.

1:16:30 Doing all the GNI stuff, code that, LMS that output code that needs to run like in an isolated way. Like there’s so many like new primitives that are needed to work with GPUs and work with LMS. And we built some building blocks so far and that’s unlocked a bunch of applications that the world has seen over the last few years. Not just stuff with us, of course. But I think over the next few years, there’s going to be so many more cool things. And so I mentioned some stuff. Like I’m very bullish on training. I’m very bullish on fast jobs. I’m very bullish on certain things around like agents and low latency and things like that. But like in the end, we consider ourselves like an infrastructure shop, right? Like we’re not trying to build, you know, very narrow things. Like we try to build like very general purpose building blocks and put them in front of customers. And then those customers will combine those building blocks and build some incredible applications that we could have never anticipated. I want to go through some of your tweets. This will be the last part of it. Okay. And we can just like, maybe you want to double click on anything. A lot of long running persistent sandbox stuff right now. Today feels lift and shift. I suspect sandboxes will evolve from dev box into cloud to something more ephemeral, one environment for every command. And now I’m shouting in between so you can pause, resume, clone. Yeah. I mean, we talked about this a lot. Yeah, we talked a bit about it, right? Like I’m fully, I think, I think the median sound box in modal today is like eight seconds long or something like that. Like, which is kind of incredible

1:18:00 when you think about it. And over time, I think it might get even shorter. We want to support long running sandboxes too. Like I think they’re awesome, like very valid use cases around long running sandboxes. But like my vision is like, you know, the world needs also, we need to figure out this like how can we just massively parallelize things and reducing the overhead of starting and stopping sandboxes? I don’t know if I ran an IP already start up engaged in catfax arms race for the competitor, it would be nice to coordinate and paste arms race a bit. No comment. The biggest downside of GPU scarcity is not that it’s hard to get GPUs, but that almost all capacity is sold as three years as a three year rectangle is starting three to six months in the future. That’s not what demand looks like for most users. I think we talked about that a little bit as well. Yeah. And you know, maybe just go a little bit deeper now. Like one thing I think modal has always been very, very good at is fully on demand capacity, right? Like if a customer comes to us and need 100 GPUs, like we can typically give them that like immediately, you don’t even have to talk to salesperson. And a lot of workloads in real life, like they don’t look like these like flat, you know, rectangular workloads, like they scale up and down throughout the 24 cycle, they maybe grow over time, they maybe have certain burst periods. And to me, that is a challenge with a lot of customers today. Like the, and I do think it’s a legit problem for, for the space is like the only, the prime way to get capacities to, to buy these rectangular blocks. It’s not great. I think the world needs to, and that’s fundamentally, I mean, I’m very biased, but like that’s why I think modal can play a very large role in that sort of, you know, whatever you want

1:19:30 to call it, sort of duration transformation of offering more flexible capacity. And we can do that because we can aggregate many, many thousands of customers and the demand gets very smooth and aggregate. So that’s a lot of the value prop of modal is exactly that. Would you say that’s a differentiator? Like the, oh really? Okay. Yeah, massive differentiator. Actually, in fact, like, I think, modals, like if you want to be like very sort of reductive, like you can break down modal’s value prop into two things. What is a developer experience, the ability to like iterate and like build things and not have to think about infrastructure. The other thing is the capacity stuff. The more and more I think about modal and like where we’re going in the long run is like we’re building, like we’re aggregating the world’s compute capacity. And we’re very good at slicing it out like very granularly, because we’ve built all this work around, you know, scaling up and down extremely fast, you know, all this stuff around GPU, snapshotting, working some, some ways to make it much, much faster. And that model works at very large scale, like really, really well, because you can aggregate lots of customers. Every month I keep saying or saying box revenue might apply to it, but it keeps going up 30% every month. It’s kind of nuts. Like I don’t understand, you know, like every month I’m like, well, you know, sandboxes grew a lot, but like clearly the kind of plateau and then they keep growing even more. So I, I, I don’t know. And a lot of VCs, like they’re like, how did you know sandboxes was going to be big? Like, and the truth is like, I didn’t like, like, I thought it was going to be bigger than it was in 2023 for sure. But like the fact that sandboxes is a third of our revenue today, it’s, it’s insane. And, you know, we’re obviously very happy about it. And we have like a very large team investing

1:21:00 in sandboxes. You know, I don’t want to like trivialize it. Like we obviously, I do want to give ourselves like a lot of credit for, for seeing sandboxes for what it is today a little bit back in even 2023. But I think, you know, if like, I probably would have never, you know, taken a bet that revenue would be this large from sandboxes and the market demand would be so much. There’s an interesting structural reason why inference providers should exist separately from model builders. The best model keep changing a lot inference providers keep can easily reallocate resources between models since overall demand is quite smooth. So I mean, if you think about all the Chinese labs and, you know, in the future, hopefully more US labs too, every time a new lab releases, you know, the latest and greatest open source models, like you see big shifts in demand, you know, for these models. And so I think a challenge with, you know, if all these labs independently like held capacity, they would see when they launched a new model demand would go up a lot. And then another lab launches a better model and then the demand goes down. You know, but, but of course, like in the background, they’re working on the next version. So like, I think, you know, there’s like a structural reason, like I said, like why you’re going to start, you’re seeing demand kind of shifting to like the latest and greatest Chinese lab. I mean, not like, it’s not like, you know, this is like, everything just moves overnight because, because a lot of models and prompts and stuff like, like a lot of things that get sort of there’s like a certain stickiness to it. But I do think, you know, there’s an argument that it’s very easy for model to like, you know, right, today we’re running, you know, X number of 1000 GPUs for one model. Tomorrow we need to shift like, you know, 50% of that to another model.

1:22:30 Sure, that’s just like, that’s what the system is built to handle. No one even needs to tell us to do that. It just magically does that. So you agree that basically the LLMs are commodities? And if you agree, then where do you think the business for the big labs is going to be? I’m not sure if that’s necessarily true, right? Like, is Taylor Swift the commodities? Like, she releases a big album and goes on a tour and makes a lot of money. And then, you know, then there’s like a down the air when she goes and like records, I don’t know, this is best analogy. And then she releases a new album, you know, so I don’t know, like she still has like a lot of brand and there’s a lot of, you know, power and like makes a lot of margin. So like, as long as the labs like repeatedly can launch new models, and as long as the margin that they have on those models are such that they can sort of pay off amortized like that the cost of training those models, it could still be like a decent business, I think. So you don’t think that the intelligence is like plateauing? No, I don’t think so. I mean, I think it remains to be seen, right? Like, models are today are like very good at like doing things that can be trained very efficiently, you know, using reinforcement learning. So we’ve seen that initially with, you know, coding and writing simple snippets over time, it’s gotten really good at also writing production like level code. It’s gotten really good at math. It’s gotten really good at cybersecurity. Like these are all like very quite verifiable things. I still think it remains to be seen, you know, our model is going to be really good at whatever, cooking food or I don’t know, like things that I have like more

1:24:00 like diffuse targets, maybe longer feedback loops or whatever, I think humans probably like my feeling is like will be quite good and quite competitive with models for a long time in those domains. But you know, certainly, you know, AI models, they’re getting very good as I’m sure everyone can agree with. Yeah, so obviously, like the models are getting really good at coding, they’re already really good at coding. What other applications do you think maybe that aren’t as big right now are going to become huge over the next few years? For the LLMs? I don’t know, but I think anything that has like kind of a verifiable computational bio, I think you mentioned. Yeah, computational bio, I think those are like, not exactly verifiable, but there’s like certain ways you can sort of run simulate, like you can use an LLM to output predictions, then you can do like computational like, like methods to like simulate molecular dynamics and get like kind of feedback loops. I mean, like I think like material science is like maybe another area where like you have a little bit of that loop where you can sort of, you know, synthetically manufacture certain materials and measure the properties and use kind of put an agent in the loop and like, you know, do things. So, you know, I have a physics background, I’m pretty excited about that. So drugs, materials, what else? I’m drawing a blank, but I’m sure there’s many other things. Cool. I mean, those are all the questions that I had. Is there anything you wanted to talk about that I haven’t? I’m good. This is great. This is a lot of fun. Awesome. Thank you.

Tickets and applications now live for AIE CODE - the last flagship AIE of the year.

When Anekha Sokhal traded energy derivatives, staying on top of the market meant tracking more information than any person could read. Now, as CEO of Moshi, she is building the tool she wished she had: a recommendation engine that connects market information to what matters for a trader’s book.

At AI Engineer New York, Anekha joins swyx to discuss her path from mathematics and JPMorgan to founding Moshi, and why useful AI needs to fit the way people actually work. She describes Moshi as what might happen if Bloomberg, Claude and Netflix had a baby: market data, an interface for working with it, and recommendations tailored to the trader.

They also talk about trusted sources, document extraction, building versus buying models, and the pressure on AI engineers who are told, “you are the team.” Anekha shares what she learned from customer conversations at the conference and why meeting users comes before hiring.

Watch Anekha Sokhal on YouTube

Moshi — Anekha’s company

Anekha Sokhal — background and work

GIQ research paper — research discussed in the interview

Reducto — document extraction

AI Engineer New York

0:00 — AI Engineer New York and “you are the team”

0:54 — Meet Anekha Sokhal

2:09 — From mathematics to energy derivatives

3:46 — The Fulbright and becoming a founder

5:45 — What Moshi does for traders

6:10 — Processing a flood of market data

9:03 — Bloomberg and market workflows

10:03 — Bloomberg, Claude and Netflix

12:03 — From dashboards to workflows

15:03 — Trusted sources and signal

17:13 — Build or buy models and document extraction

18:20 — AI Engineer New York

20:32 — Customer meetings and community

23:54 — Hiring and talking to customers

25:55 — swyx’s path into AI

Automated transcript; wording may contain errors. Speaker labels are omitted.

0:00 So I’ve actually extended my stay in New York, because we’ve like now, I was meant to leave today. We’ve now got meetings on Thursday, extended it another day, Friday, extended it another day, Monday. And then I’ve got now meetings where people are like, hey, we’re in energy. Let’s talk in Houston. So we’ve like filled up a week after the conference of meetings. AI engineers was an interesting one, because a lot of them are feeling like overwhelmed with the pressure from management. To do what? Because people now say, we don’t need a team, you are the team. And so it’s like, hey, I need to now build this whole system myself. But that system would be typically done by 10 people. Okay, we are in the studio. Last podcast of AIE for me with Anekha from Moshi. Welcome. Thank you so much.

1:01 Did I get it right, Anekha? Yes, you did. So your background caught my interest because I was a former trader and you’re now a founder. Yes. And you also were telling me just about your experience. I kind of want to hear about it from you. As far as I know, this is your first AIE. How did you hear about us? What was the experience like giving your talk and what happened after? Yeah, well, first of all, thank you to everyone at AIE because it’s been such a fantastic experience. And thank you for inviting me on the podcast as well. I’m so excited to just share my experience. And also, I think we’re living in such a fast moving time and everyone I speak to is like very overwhelmed. So it’s really helpful to be able to kind of share everything that I’ve had in my experience because I did my undergrad in statistics and math, which is now what everyone calls AI. And so it’s really interesting to speak about that and the evolution of that and also to always speak to a former trader as well.

2:02 And just to share like war scars on that. Did you do financial calculus, like stochastic calculus? Yeah, I did that. And I so my undergraduate degree was mathematics, operational research, statistics and economics, Morse at Warwick. And it was what everyone was just like, oh, my God, so nerdy. And I hated telling people. And now it’s like, wow. Yeah, everyone’s like, oh, my God, machine learning, AI. Yeah. Yeah. And then derivatives. So I did that. So my background was started in that at Warwick in England and then knew I wanted to work in trading. I don’t know what what your story to go into trading was, but I wanted something which was fast paced and to do with data, like absolutely love data. And it was when I read about what trading was. I could not imagine myself being in any other career. So I was just set on doing that. So I got my first internship in investment banking and trading and then got an internship at JP Morgan where I started.

3:04 And I started on the emerging markets desk and then moved to commodities and did derivative trading there. So most of my career has been in energy derivatives. And because I have that data background, I always looked at the market from I wanted to break it down into all the different parts. So, for example, in commodities, a lot is about the fundamentals and about the different news and the different the physical happening. And so I was always fascinated by like building out from zero. And so when I was trade trade trading, I saw this shift where it’s like a data and then something was going to happen. And that’s when I left JP Morgan to to actually I wanted to be an entrepreneur and and a founder. But I wanted to have a data company. So I got the Fulbright scholarship to come to the US because you guys have such an incredible venture. Just casually got a Fulbright.

4:06 Yeah, I guess the way I say a funny story, I was doing a yoga class in London. I was like, so my parents have a jewelry business. I side hustled a computer vision startup before my startup at university. And so I was like, oh, what are the problems in the jewelry space? Oh, like virtual try on. So that led to me wanting to be a founder. And I thought, how could I be a founder in the best country to be a founder in? And I thought, what I want to be a founder who can talk about the technology. So this was back in like 2020, 2021. And I found Fulbright had a data scholarship and they gave one in the whole of the UK. So I was like, my way to get here is a Fulbright scholarship and they only give one. And this is really funny because I don’t know if you remember 2020 when it was COVID and gas markets were going crazy. So there was there was like gas markets had insane

5:06 volatility. And so when I did my Fulbright interview, they asked me, how do you like, how do you see the future? And I talked about the like global issue of everyone is having the energy market crisis and how data relates to that. And everyone could relate to what I was talking to because like gas prices were crazy. Their electricity bills were going crazy. And so that led me to the US, came here, was deep in machine learning research, published at ICLR with Stability AI, worked in collaboration with NASA, did a lot of a lot of super interesting work, which then led me to Moshi. And what is Moshi? So Moshi, we are building the program that I wish I had when I was trading. So what we do is we provide real time insights for financial traders, starting with commodity and energy markets and macro markets. So what we do is we process more data than is humanly possible to tell you everything that matters to your portfolio right

6:09 now. What kind of data would you be processing that give us an idea of maybe types and an order of magnitude? Types and order of magnitude. So let me start with order of magnitude. So a fun way I explain it to people is it takes me four seconds to say the sentence to you. And in the same amount of time, Moshi has learned 300 new things. So that’s the type of volume of data that we process. In terms of the types of data, this very much comes to me like geeking out about how I would trade. I would break the market up into all the different types of data, which a trader looks at to decide what trade to put on and how to manage the risk. So what I do is I take the market and I say, well, there’s structured data and unstructured data. There’s the time series of the futures market and there’s all of the technical analysis around it. So we’re looking at all of this like statistical indicated, all of the different technical indicators, all of the moving averages.

7:11 That’s one data set. And obviously that is updating constantly. The next data set we look at is the time series for fundamentals markets. So especially in some commodity markets, there’s a lot of demand and supply. And then we look at the unstructured data. So the news is the obvious one. But then there’s also all this alternative data that exists. For example, there’s X data. There’s audio data on podcasts. X Twitter. X Twitter, yes. You read Twitter. OK. So understanding all of these different types of data. And that I would say is the like public availability of data. And there’s this whole other type of data, which is the user’s private data. So I’m sure you had that when you were trading, which is you have the analysts you care about. I can’t come and tell you, hey, you should, you know, read this person because you have who you need. And then you’re going to have your subscriptions that you care about. You’ll have your internal data, right?

8:11 Like some of the companies we speak to, if they are trading markets, trading energy markets, they will have their person for the amount of LNG ships going through the Strait of Hormuz. And so there’s all this data which is personal to the user. So we take those two data points together, do all of the data cleaning, data transformation. We then look at the statistics and machine learning, tag it, kind of create our whole ontology on top of it. And then we put it into the Moshi, like n-dimensional graph. And then we produce the output, which is a piece of software which pushes data to you. So every time you open Moshi, you see something new related to your portfolio. And you can pull data. So LLM, I’m going to give you answers in 30 seconds, which would take you typically days. So do people often say new Bloomberg or is that something that you disagree? You don’t like being characterized as new Bloomberg? Yeah, that’s an interesting question.

9:13 Because obviously that’s the 800 pound gorilla or whatever the British version of saying 800 pound gorilla is. Yeah, you know, I think Bloomberg has certain features, which the Bloomberg chat, it’s a fantastic aspect of the platform. Yeah, for those who don’t know, like traders would transact billions of dollars in chat. And there’s a standardized formula. When my entry into NLP was actually parsing Bloomberg chat into a structured pricing sheet for my logs. And that’s how we did option pricing. Yeah, there’s the Bloomberg chat, which is one thing. I wouldn’t say we’re a new Bloomberg. I’d say we’re a different type of software, which is not. I think this is the easiest way to say it. And I don’t know how controversial this is. But if Bloomberg and Claude and Netflix had a baby. Netflix. It would be Moshi. You have videos? Netflix.

10:14 We actually have a recommendation system, which we’ve internally built, which is if you’re a trader and you’re looking at one thing, you actually care about the second and third order derivatives right now, which you wouldn’t immediately know. I see. And so we’ve built a whole recommendation system where a simple thing, if you’re, let’s bring it back to like an oil trader, and there’s an energy equities company that’s doing something which would indirectly impact oil, you need to know that right now. You’re not going to get that from like a news stream. You will get that from a connected graph, which is showing that to you. Because if those two information points are coming out right now, the only way you can do it is either a human, which is being able to like connect those, or by a mimicking human, sort of like a recommendation system, which can do that. But that’s what we try and do, because I like wish I had that when I was trading. Yeah. Well, you know, there’s a whole addictiveness element of Netflix, which is bad.

11:15 There is. But yeah, it’s very interesting. I think like RecSys is one of the most important parts of AI. And as a curator of AIEs, I actually have every year an LLM RecSys track. And engineers often don’t appreciate how important it is. Like it’s not very well attended, even though some of the people powering these things, I mean, they run RecSys for YouTube, they run Netflix for Netflix, for Instagram and TikTok. And then like nobody shows up. And I’m like, what do you mean? Like you’re not interested in RecSys? You know, I find it really interesting. I think part of it is they make it seem so easy. And it’s like, oh, this is so like... This is the most important system in the entire, you know, company. Exactly. And I’d say there’s arguably something linked to it, which I think some companies do really well, like Spotify, Instagram, which is one thing that people have come out with AI

12:15 is everyone now can build their own Claude dashboards, right? It’s like, hey, I built this myself. Great. Great. If you want to get someone else to use it, that’s your brain in a dashboard or your brain in something. Try and get someone new to incorporate that into their workflow. You’re then no longer dealing with a problem. You’re dealing with a UI problem, which is how did they interact with it? Because sometimes users don’t always use that. One thing that these platforms do really well is they push data to you, which like hooks you. So you naturally don’t even realize you’re on the platform and you’re like, oh, I want that. No, no, I actually want that. And so we’ve been trying to internally as a company understand how does UI relate to a recommendation system? Because our thesis is if you solve for one, you’re naturally also solving for the other. And I think that’s a big thing right now, which is if everyone’s building software, how do you get it to actually be software that people will use? It’s a bit of an interesting take.

13:19 Yeah, a bit of chicken and egg. Well, anyway, I forgot to mention also my background to this. I used to trade currency options in London, actually. And then I spent some time in rates. And it had a very similar career path where you start with EMEA and then you go into maybe G10. Yeah. At least that’s what we call it for currencies. And you ingest a lot of information and really probably LLMs can process it better. So we didn’t have the tooling back when I did this 10 years ago. But now I see that. And then I moved to the buy side where you start paying for alternative data exactly like you did. We were actually one of the very first few at Balyasny to do the X fire hose or Twitter fire hose. Yeah. That’s something like a stupidly expensive, like $300,000 a month data thing that you have to pay for to get all tweets. It used to be free. And you sort of parse them.

14:19 I don’t think I don’t know if it was ever free. It was free for researchers. Okay. But then if you’re like a hedge fund, like you have to buy that. And it’s very noisy. And so my eyes go up because I don’t know if I would trade on those things because sometimes people do just talk random crap. And sometimes people try to manipulate markets by saying stuff. And it just doesn’t matter that much. But we would also have alternative data like satellite information or credit card information or whatever. And so on the buy side, my number one question is, are you looking at my data? Which is probably not. But how do you assure people? How do you integrate with systems that you don’t necessarily even see? Yeah. And I think there’s two questions in there, which is the first one is, if there’s so much data available, how do you know what’s important? So when we’re looking at X data, it’s a natural conversation when we speak to people or let’s even expand it out to any types of data.

15:20 One thing we found with people is they really will have their trusted people who they’ll follow of these voices in industry, especially in commodities. There’ll be the suppliers of commodities. There’ll be the LNG, all of these facilities who will be active on there because it’s the quickest way for them to also give a status update. Wow. So it’s interesting because we found that a lot of our users will tell us about this. So it’s like, oh, OK, so you find it as a trusted source because you believe this certain individual. Sketchy. Right. And so it’s one of those things where we have found a way to try and filter out as much as possible. But when we are showing it to someone, we give a confidence score of, OK, someone is saying this one thing, but this is everything else happening around the conversation. Because what’s interesting to people is if there are divergent sources, like an example would be how many

16:21 ships are going, how many planes are going above us right now. If Reuters is saying something or Bloomberg is saying something, a Wall Street Journal is saying something, they may not all agree. But the fact that they don’t agree is a point of data because is there an opportunity there? Because no one knows. So that’s how we try and give people information. And then the second question is, you know, people’s data sits everywhere. And I think there’s different types of integration. The first one is what about like their messy PDF data? That data is what we’ve built is like a vision language model to understand it. So we’ll take the data that they can upload to their cloud version of Moshi. And then we actually go and we break it down and then build it up in our database. And there are certain types of data where you just have to connect to their systems. And then that’s like more of a conversation. Is it important to, I assume you’re running this vision language model yourself, is it important to own that? Or would you want to lay out, I mean, there’s any number of vendors, Reducto

17:24 is like a non-documenting processing one. How important is it to have your own models versus just use whatever’s off the shelf? I think that’s a question that a lot of companies are battling with right now. The first question for us is we, if someone’s giving us their data, we don’t want to be sending it to a third party. Because there are systems out there which can do these things for you. But you want to make sure that you’ve got that agreement on how are they processing that data? What are they doing with it? That’s the first question. The second one is, if you don’t have a team of engineers, then that is a great solution. But if you do have a team of engineers like us, a fun fact is most of the team has in some capacity worked with NASA, then they want to build the models. But not everyone has that. And so, you know, I think it really depends like where you are as a team. Yeah, got it. Okay, and then coming into your talk, you apparently had a lot of inbound from it.

18:28 This is our first finance focused one. So I was genuinely unsure who we could get in the door. Obviously, we like pitch it and market it very heavily for this crew. What kind of people did you end up talking to? Yeah, we, it was a like fantastic opportunity because we have been like back to back speaking to people. And what we initially thought would be the interest, we’ve had people like, in the past hour messaging me on LinkedIn saying, hey, I saw your court talk. Are you’re in Houston? Can we get a meeting? Or hey, are you in New York? Can we get a meeting? The types of people we’ve had a lot of traders, either at funds, consulting, and then AI engineers was an interesting one, because a lot of them are feeling like overwhelmed with the pressure from management. To do what? Because people now say, we don’t need a team, you are the team. And so it’s like, hey, I need to now build this whole system

19:31 myself. But that system would be typically done by 10 people. So a lot of people have talked to us and they’re like, hey, I need to build a system. But like, you know, do I need to have a swarm of agents? But then like, what if something breaks? Something interesting is, hey, I built the system. People told me they needed it. It’s in their email. No one’s, no one’s clicking it. But they complain that they have this problem. Like, what do I do? So that was one area. And then partnerships. You know, we’ve been meeting people and people like, hey, you’re doing this. That’s interesting, because we’re doing this. Let’s have a conversation after. So I’ve actually extended my stay in New York, because we’ve like now, I was meant to leave today. We’ve now got meetings on Thursday, extended it another day, Friday, extended it another day, Monday. And then I’ve got now meetings where people are like, hey, we’re in energy. Let’s talk in Houston. So we’ve like filled up a week after the conference of meetings. Oh my god. Yeah. And my team was like, hey, Anekha, another person signed up. Another person signed up. So it’s super cool.

20:32 Is it interesting to do these one to one? I guess because they need to ask you things in private. It just seems very inefficient to set up so many meetings when you could be doing building a business or building the product. Yeah. We try and be as helpful to people where we will have an initial meeting, but then really we want to be able to understand what the problem is that you’re solving for and then have a plan after that. So we have got things that we’re doing now to stream that because like we’re hosting our first webinar because of the amount of inbound that we have. We just can’t always do these one to ones. And so more generally, when I was in the buy side, the banks would actually set up these meetings for us. Yeah. And they’d come to little rooms like this. There’d be sort of a round table. I would meet five at once. So like I’d say I’m a CFO of a company and I need to do investor relations.

21:33 I’m meeting five funds at once and I would be one of the five funds. And I imagine what we could do for you in future, which I haven’t, I have not got this off the ground yet, but we should do the meetings on site. Like you’ve, that makes sense. People like saw your talk. They want to speak more with you. They should be able to sign up and you can do five on one. And probably all these questions are the exact same. And like, that would be a great idea actually, because another, because we used to do this in finance, like that, that is such a good idea. Cause we had one where, um, sometimes you like don’t know what the person you’re meeting looks like as well. Okay. So you’re trying to like take pictures of where you are. Really? It’s like, Hey, meet me here, meet me here and like call them. So an idea like that would be great. So just like an assigned meeting room. Yeah. It’s like, Hey, like. I mean, this whole floor is that, but we had to pre-assign it to other speakers, but, um, we, we wanted to do some dynamic allocation. I just don’t know how to, you know, get that going. I’m trying to fill it out with you. So like, you’re not having to like fly to Houston or even like, even if they’re in New York, you don’t even have to go to their office.

22:34 Everyone just gets in here, gets their meetings done. You’re out. I actually think that would be a great idea. Like have a 30 minute slot for this person and then like sign up. Just like whatever you want. Yeah. Like smaller one, one. That is a great idea. Yeah. Yeah. Uh, I mean anything else that like we, I guess we could do better or like other experiences from the conference that you’re, I think this has been the best organized conference I’ve been to like, no, no, seriously. Like the amount of good quality guidelines I’ve had, and it was super clear, like the speaker team, the breakout rooms, um, the speaker lounge having, I feel like the lounge was never this good. Uh, yeah. So haircut. I know. I didn’t do a facial. I did a facial and a massage. Yeah. I know you got your haircut. I did get my haircut. Yeah. Um, I, I, I mean, we do want to swallow speakers. Uh, this was definitely, uh, over and above and that’s thanks to the cognition for doing that. But, uh, yeah, I mean, uh, organizing is one thing.

23:38 I do think like in terms of business value and efficiency, I think that’s an important, uh, element. Like we want to make this, uh, and you know, something that’s useful for you for both hiring and selling, you know, and I don’t know how to balance that because yeah, are you even, what’s, what’s more important to you hiring or selling? So in terms of we’ve just hired and we plan to hire, um, we’re always meeting people, but the priority right now is to meet clients and that’s a priority, not necessarily hiring. Um, because I think what we have with hiring is we get continuous inbound from LinkedIn. Really? Yeah. I had, we had one job up. We had to close it. It was open for four days. We had 2,500 applicants. Okay. Um, and so it’s like, that’s tough to deal with in itself, but meeting people who like want to buy and people who, because I think there’s people who have the problem, but they don’t want to buy. And there’s people who have the problem and want to buy. And then you who are trying to find that match that is,

24:39 I haven’t found any person who’s done that. Um, well, but so I really like your one-on-one idea. Yeah. Yeah. I mean, so basically I’m, I’m transferring over what I used to have as a hedge fund person to this market, which is AI and finance. Yeah. Um, well, we’ll try to do that next year. We’ll try to be the definitive thing. Um, and I think on hiring side, so that that’s buying and then that’s clear. I think, I think I can definitely help you better there. Uh, on the hiring side, I also want to help you. Right. Okay. Like 2,500 is too much and most of it’s noise. And so you need a better filter. I have a strong opinion. Okay. Go ahead. The amount of people who are also doing it in a way where it’s like undercover, not it’s they’re shopping, they’re spying. You have that, but you also have, it’s that like legally not allowed to work. So there’s so much noise in that. It’s like visas and there’s so much noise around that as well. And so when I was a first time, like I am a first time founder, when I first

25:41 started, you don’t know those things. You don’t know the like market. Right. And there’s a lot of noise related to that. Yeah. Okay. Well, we’ll, we’ll try to get that higher signal for you. We want to be a high signal conference. Um, I’m just appreciative that you’re spending some time with us and congrats on your success. Thank you so much. I’d love, I have one question, which is how you went, you moved into AI. Uh, just, just decided to start. Uh, well, I, I left, uh, the hedge fund worlds. I moved into tech and then when the AI wave happens, I realized that I better move very, very quickly to, uh, establish this field. And, um, so I did the AI engineering post and, uh, started this, this conference and the media stuff because I’m, I’m very much of a media person. Yeah. Uh, at my heart. Um, I do miss trading a little bit, but this is very fulfilling because I get to meet real people instead of stare at a screen all day. Uh, yeah, I feel the same. Well, thank you. I think we have to go. Thank you so much. Thanks. Thanks.

Almost 2 months ago, OpenAI introduced Daybreak for Frontline Defenders, a $1B subsidy for access to Daybreak cyber models and products. That funding came into use almost immediately, as major vulnerabilities in PHP were independently discovered by two parties: Ryan Austin, a researcher at the security firm Calif, as part of the OpenAI “Patch the Planet” project in August 2026 using GPT 5.6 Sol:

… as well as Wiz, recently acquired into Google Cloud (using Gemini 4 Argon and GPT-5.6 Sol, as well as their own tool, Wiz Atlas).

All high profile cyber disclosures are at least a little bit of marketing, and the degree to which this was a severely alarming vulnerability on the order of famous ones like log4shell and Heartbleed is up to some debate, but surely the coordinated response across the PHP Ecosystem is the developer community responding in its most ideal form.

We should expect to be in for at least a year of finding such critical vulnerabilities; but it is the wrong move to blame the messenger for finding them; rather it is good to know that our critical software is generally getting the biggest security audit it has ever had in human history.

AI News for 10/20/2026-10/21/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

Top Story: PHP2Shell—chronology, core claims, and the complications behind the emergency response

Researchers disclosed two PHP-engine memory-corruption vulnerabilities, with demonstrated unauthenticated exploitation against major CMSs and an unusually broad coordinated response.

• The researchers named the vulnerabilities PHP2Shell and identified them as CVE-2026-109618 and CVE-2026-109619. Their technical description places the defects in PHP’s internal handling of out-of-memory conditions and fatal errors—not in a WordPress plugin or an application’s authentication logic. They report demonstrations of full remote code execution and “Heartbleed-style” memory disclosure. @sagitz_

• The disclosed demonstrations included pre-authentication RCE against WordPress, Drupal, and Joomla. The researcher’s announcement separately claims remote memory disclosure against any PHP application. These are researcher-reported exploit results; they should not be silently expanded into a claim that every deployment has an identical, immediately exploitable RCE path. @sagitz_

• OpenAI says Calif researchers used its models to discover the flaws, while secondary reporting says Wiz independently found the same bugs. Posts identify GPT-5.6 Sol on the Calif/OpenAI side; a Wiz-affiliated account credits Gemini 4 Argon with discovery, exploit-chain development, detector creation, and patch validation. This is a multi-party discovery story, not evidence that one vendor alone found and fixed the issue. @TheRundownAI, @steipete, @ralucaadapopa

• OpenAI describes approximately two months of coordinated remediation before disclosure. It says the patch required a major PHP-engine change and coordination among maintainers, researchers, security providers, and governments. A participating researcher confirms work with the PHP Foundation to support and harden the patch. @OpenAI, @itszn13

• Cloudflare deployed initial WAF mitigations before the scheduled patch release. Its CTO said rules covered known exploit vectors, were available to all customers including the free tier, and were being updated as early attacks emerged. He explicitly called the rules a stopgap and directed operators to patch at 14:00 UTC. @dok2001

• The supplied NSA account advised preparation for an immediate update—and taking systems offline if timely mitigation was not possible. Later OpenAI messaging said upgrades were available; secondary reporting says details leaked before the patch. The supplied material does not establish precisely what leaked, when, or whether the release schedule changed. @NSACyber, @OpenAI, @TheRundownAI

1. Discovery and private coordination: approximately two months before public disclosure.
OpenAI places the work under Patch the Planet and says it led a coalition preparing fixes and protections. Its security researcher Eric Wallace adds that internal teams worked on patching and detection rather than merely disclosing the bug, with confidential advance notification to governments and outside agencies. Greg Brockman reiterates the two-month coordination period. These are participant accounts, not an independently audited incident timeline. @Eric_Wallace_, @gdb

The distinction between program names matters: the posts associate the vulnerability with Patch the Planet, while researchers credit Daybreak model access for enabling security work. Those are related roles—remediation initiative versus access mechanism—not interchangeable descriptions of the same program. @OpenAI, @S1r1u5_

2. Advance public warnings and a scheduled patch window.
Japanese-language posts warned operators to prepare for a 23:00 JST, October 21 release, consistent with Cloudflare’s 14:00 UTC announcement. One correctly emphasized that a PHP-core vulnerability cannot be fixed merely by waiting for plugin developers. Another framed the event as a “15-minute” race after patch publication, but that number was a prediction—not a measured time to weaponization. @ds_nakajima, @expokun

3. Pre-release filtering and observed attack activity.
Cloudflare’s post is the strongest firsthand operational evidence in the supplied record: mitigations were already deployed, known vectors were covered, and “early attacks” were appearing. It does not specify successful compromises, request counts, victim counts, payload provenance, or whether the traffic represented original exploit development versus reuse of leaked information. @dok2001

4. Research disclosure, government warning, and patch availability.
The researcher’s public thread describes the exploit demonstrations and error-handling root cause. The NSA post calls for urgent preparation and conditional shutdown. OpenAI subsequently says every PHP site can upgrade safely. The supplied set contains no direct PHP Foundation release advisory, exact fixed-version matrix, distribution package identifiers, or patch commit. That missing primary-source layer prevents a reliable “upgrade to version X” instruction. @sagitz_, @NSACyber, @OpenAI

5. Alleged premature exposure and widespread service disruption.
The Rundown reports that details leaked early. A later viral compilation adds approximately 1.5 million servers, attributed to Censys, running versions without a fix; reported shutdowns involving Dutch institutions, government sites elsewhere in Europe, Wikipedia, Archive.org, Asana, GOG, and others; and a 05:55 warning from Japan’s prime minister. None of those individual claims is corroborated by its underlying advisory, measurement, or status page in the supplied material. Nor does the record establish whether each outage was a precaution, a patching interruption, an unrelated failure, or a compromise. @TheRundownAI, @fiveoutofnine

SemiAnalysis separately reports that Japan’s prime minister urged PHP operators to patch immediately or temporarily disconnect if they could not. Its account contextualizes that response with an existing Japanese breach wave, rather than establishing that PHP2Shell caused those earlier incidents. @SemiAnalysis_

6. Exposure-checking assistance followed disclosure.
OpenAI published a repository containing a Codex skill intended to guide operators through checking whether their sites were affected. That is an assistance workflow, not a substitute for a vendor advisory, package verification, or evidence that every vulnerable runtime has been replaced. @OpenAI

The affirmative case is stronger than a benchmark-only claim. Researchers describe working exploit chains, coordinated patch engineering, detectors, and patch validation. A participating researcher calls PHP2Shell the most consequential vulnerability they have helped discover. OpenAI characterizes it as an example of the “Defender’s Window”: advanced models finding a flaw before attackers, giving defenders time to prepare. Importantly, its wording is “to our knowledge”—not proof that no attacker previously knew the vulnerabilities. @itszn13, @OpenAI

Credit is distributed across humans, models, and infrastructure providers. Calif/OpenAI and Wiz/Gemini appear as separate discovery accounts; PHP maintainers supplied the upstream remediation path; Cloudflare supplied pre-release filtering. Google also advertises Wiz testing in which Gemini 4 Argon found more than twice as many vulnerabilities as baseline models, but the supplied promotional post does not expose the dataset, baselines, budgets, or false-positive accounting. It is not an independent comparative evaluation of PHP2Shell discovery performance. @TheRundownAI, @ralucaadapopa, @Google

Researcher-access policy draws both praise and criticism. One researcher credits OpenAI’s Daybreak access with a portfolio of findings spanning PHP, GitLab, Chrome, WordPress, and other targets. Another sharply contrasts OpenAI’s vetted access with Anthropic’s restrictions; the claim that Anthropic contributed “zero value” is that author’s sweeping opinion, not established by the PHP record. A further post criticizes restrictions on Astra access and argues for giving 1,000 vetted researchers the best available models with intensive monitoring, rather than a rigid red/blue distinction. @S1r1u5_, @migtissera, @S1r1u5_

The countervailing safety rationale appears in reporting that Anthropic delayed broad access to Mythos-class models for four months to strengthen cyber safeguards. That provides context for restricted access, but does not settle whether its implementation helped or hindered this particular remediation effort. @kimmonismus

A burst of disclosures is not, by itself, evidence of a burst of AI-caused compromises. The supplied feed mixes defensive discovery, supply-chain attacks, credential theft, precautionary shutdowns, and unrelated vulnerabilities. Ethan Mollick interprets the moment as attack and defense accelerating beyond human response speeds; John Allard describes a capability “phase transition” while acknowledging that many recent findings came from white-hat programs. Both are interpretations of the pattern, not incident-level causal attribution. @emollick, @john__allard

• The emergency is as much distribution and ownership as discovery. Operators need to locate PHP across managed hosting, containers, appliances, background workers, and old applications; obtain appropriate packages; deploy them; restart or replace affected processes; and verify the running build. One operator describes maintaining numerous PHP/WordPress sites and being exhausted by testing and updates. Another concludes that patch distribution is poorly coordinated. @ostrisai, @jd_pressman

• WAF deployment reduces immediate exposure but creates a false-confidence risk. Cloudflare’s own statement is narrower than the later advice to put all internet-facing services behind it: known vectors were covered, protections were evolving, and patching remained mandatory. Proxied coverage, alternate ingress paths, and the origin’s exposure must be checked rather than assumed. @dok2001, @threepointone

• Shutdown guidance is conditional, not evidence that the entire internet should disconnect. The supplied NSA post recommends taking systems offline when immediate updating is infeasible until full mitigation is possible. For a specific operator, the relevant questions are affected-version status, exposure, available mitigations, and the consequences of continued operation—not the engagement score of an alarming post. @NSACyber

• “Use the RCE to auto-patch every server” is a proposal, not an authorized remediation plan. Marc Köhlbrugge raises the idea; J. D. Pressman suggests a presidential pardon for a lab or white-hat team attempting it. Unsolicited exploitation still risks unauthorized access, outages, incorrect fixes, destruction of evidence, and cross-jurisdictional liability. A speculative pardon does not provide universal authorization or eliminate civil and foreign legal exposure. @marckohlbrugge, @jd_pressman

• Update velocity has a supply-chain trade-off. One widely shared post captures the dilemma: updating constantly can expose users to compromised packages, while delaying updates leaves known vulnerabilities open. Pressman argues that endless patching is unsustainable unless the rate of new vulnerabilities and risky releases falls; he also notes that receiving no patch and discovering theft is worse than receiving too many updates. @xeophon, @jd_pressman, @jd_pressman

• The longer-term alternatives remain open questions. Sarah Guo reports CVEs increasing from roughly 4,000 per month to 15,000 since April, and asks about formal verification, safer distributions, and organizations without red teams. The counts are not independently verified here, and disclosure volume is not a direct measure of successful exploitation. Other posts ask what a full-codebase security sweep would cost and recast the challenge as hardening mundane infrastructure rather than debating only abstract catastrophe. @saranormous, @saranormous, @teortaxesTex, @mimi10v3

For an incident team, the defensible sequence is to obtain the authoritative PHP and distribution advisories, inventory exposed runtimes, apply temporary protections where justified, deploy authenticated fixed packages, verify running processes, and investigate evidence of exploitation separately. Updating closes a vulnerability; it does not establish that credentials or data were never exposed.

The precautionary perspective stresses that interconnected services are fragile and remediation can be costly. Joshua Achiam compares the moment to March 2020, speculates about internet lockdowns, recommends backing up valued internet material, and suggests printing important records. Pressman says remediation costs exceeded his expectations. These are preparedness judgments—not evidence of impending systemic collapse. Paper copies may help continuity but are not inherently tamperproof logs. @jachiam0, @jachiam0, @jachiam0, @jd_pressman

The alarm-skeptical perspective asks for specificity. Alex Johnson argues that warnings about bank accounts being arbitrarily emptied obscure the distinction between unauthorized transfers and scams in which customers authorize payment. He invokes U.S. Electronic Funds Transfer Act protections with timely reporting; those protections are jurisdiction-, account-, and circumstance-dependent, not a universal guarantee. Kelsey Piper separately warns about the psychological cost of living under constant annihilation framing. @AlexH_Johnson, @KelseyTuoc

Broad personal-security advice needs to be separated from the PHP fix. Password managers, unique credentials, strong MFA, patching, carrier port-out protections, and U.S.-specific credit or tax identity safeguards can reduce downstream harm. But enabling 2FA does not prevent exploitation of an unauthenticated PHP-engine bug. Likewise, viral advice to avoid all online payments the next day is not supported by a demonstrated scope assessment. @peterwildeford, @StillMare_

The broader threat context is real but not interchangeable with PHP2Shell. SemiAnalysis reports 18 Japanese companies disclosing breaches since September 25, including exposures of approximately 6.6 million Times Car accounts, 11.18 million Burger King Japan app records, and potentially 22.5 million Sourcenext records. It attributes findings about AI-assisted scanning and API attacks to JPCERT/CC. Another post quotes an August 19 advisory concerning AI-assisted tooling against internet-exposed Siemens PLCs. These are separate reported incidents and advisories; neither establishes PHP2Shell as their entry point. @SemiAnalysis_, @AndrewCurran_

The key unresolved evidence gaps are concrete: exact affected and fixed versions; CVSS vectors; deployment prerequisites; the independent discovery timeline; what leaked before release; successful exploitation versus scanning; validated exposure counts; and the cause and duration of each reported shutdown. The supplied viral incident compilation cannot answer those questions merely by placing PHP, GitLab, LiteLLM, Railway, certificate-authority failures, and government warnings in the same narrative. @fiveoutofnine

Flint and Automated Research: Local Coding Models, Smaller Teams and Scientific Workflows

• Flint launches with broad local-serving support: Recursive released an Apache-2.0 coding MoE with roughly 40B total parameters, 2.5B active parameters and a 262K-token context window. LM Studio says it requires approximately 24GB locally. Recursive positions it as an on-device complement to—not a replacement for—frontier cloud agents (launch, specifications).

  • Claimed results: Recursive reports 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2 and 74.2% on SWE-bench Verified. These are release claims, not independent replication (results).

  • Performance: Advertised short-context speeds exceed 125 tokens/s on a MacBook Pro and 150 tokens/s on RTX Spark. A separate user measured 114–132 output tokens/s on an M5 Max with int4 MLX; that is useful corroboration of speed, not a controlled capability evaluation (local measurement).

  • Research contribution: Recursive says its Eureka Machine participated across ideation, implementation and validation, contributing sparse architecture choices, low-bit training and kernels. It claims a 71% size reduction with almost no quality loss and development from scratch in under four months (method).

  • Availability: Day-zero integrations span local frameworks and hosted access. vLLM reports 500 tokens/s on one GB200 at 100K context using NVFP4 without speculative decoding; OpenRouter offers free launch access (vLLM, OpenRouter).

• Mirendil claims one-researcher pretraining at scale: Mirendil says its first base model was built by one researcher in 3.5 months for less than $10M in compute, including research experiments and the final run. It discloses unusually broad parameter ranges—100B–500B for Base, 500B–2T for Large and 2T–10T for XL—rather than precise configurations. Independent evaluations, post-training results and API access are still forthcoming, so this is a consequential efficiency claim awaiting external validation (announcement).

• JOLT jointly trains the teacher and student: Perplexity and NVIDIA’s post-training work addresses a weakness in self-distillation: a privileged teacher can exploit shortcuts unavailable to its student. JOLT trains the teacher using outcome rewards while penalizing excessive divergence from the student. The authors report matching GRPO on MATH with 6.5× fewer training tokens; JOLT+, which adds student outcome rewards, beats GRPO across five evaluated benchmarks (method and results, benchmark summary).

• GPT-Rosalind generates testable biological hypotheses: OpenAI reports thousands of agents linking gene-expression results from millions of mouse-brain cells with literature in 32 minutes. One proposed mechanism connects DCTN1 disruption to impaired cholesterol transport; another examines differences between TSC1 and TSC2 disruption in cerebellar neurons. These are hypotheses for experimental follow-up—not demonstrated treatments or validated drug targets (DCTN1 work, TSC2 follow-up).

Inference Systems: DeepInfer Economics, Native TPU Support and Power-Aware Capacity

• DeepInfer exposes production scheduling techniques: DeepSeek’s released framework combines speculative decoding, dynamic prefill sizing, decode balancing by request count and KV length, and overlapping CPU scheduling with GPU execution. A particularly reusable idea is adjusting speculative verification length using predicted draft acceptance and measured hardware throughput, making batches more regular instead of allowing slow replicas to determine completion time (technical analysis).

  • Cache accounting: A deployment-report summary cites 1.5M input tokens/s per prefill accelerator including cache hits, versus 60K excluding hits at a 96% cache-hit rate; decode is reported at 14K output tokens/s per accelerator. These are materially different throughput quantities (report summary).

  • Economics caveat: The same summary describes approximately $38 per accelerator-hour as revenue, not profit. Server, electricity, cooling and staffing costs remain outside that figure.

• Independent benchmarking remains contested: SemiAnalysis says DeepInfer is absent from InferenceX because essential serving components are missing and reconstructing them would require too many assumptions. Its subsequent profitability comparison drew criticism for mixing hardware generations, peak versus off-peak prices, and revenue versus profit. The criticism is itself an attributed analysis, not a settled replication result; the practical requirement is matched hardware, workload, cache assumptions, SLOs and pricing before comparing economics (benchmark limitation, comparison critique).

• TorchTPU opens a native PyTorch path: Google open-sourced TorchTPU for training and serving through familiar PyTorch tooling. SGLang’s plugin is reportedly powering production inference at Meta and supports several current open models; its custom MegaKernel reaches a claimed 770 decode tokens/s for Muse Spark Open at batch size one on an 8K-input/1K-output workload. vLLM-TorchTPU adds prefix caching, speculative decoding and disaggregated serving (Google, SGLang, vLLM).

• GTC emphasizes deployment speed and power utilization: NVIDIA’s DSX POD packages AI infrastructure into containerized building blocks, with coverage claiming capacity expansion in days rather than months. More concretely, Lambda demonstrated 19 HGX B200 nodes operating within the power budget of 16 using DSX MaxLPS. Treat deployment timelines as product claims and the node result as a proof of concept, not fleet-wide capacity already delivered (DSX POD, Lambda demonstration).

Evaluation Integrity and Agent Engineering: Measure the Harness, Not Just the Model

• Medical reasoning exposes completeness–accuracy tradeoffs: Artificial Analysis revised MLCR-AA’s prompts, reference answers and three-model judging panel. Sonnet 5.5 and GPT-6 Astra tie at 41.7%, with no model exceeding 42% under the combined completeness-and-accuracy requirement. Claude leads completeness; OpenAI leads record-grounded accuracy. Revised scores are not directly comparable with the previous methodology (results, methodology).

  • Length confound: A cross-benchmark analysis finds Claude answers approximately 4–6× longer than OpenAI’s; each added claim creates another opportunity to contradict the record. Raw rubric scores, completeness and length-adjusted quality therefore produce different rankings (comparison).

• SWE-bench Multimodal faces an ablation challenge: An independent investigation argues that v2 does not meaningfully require images; a separate researcher highlights running the benchmark without images as the essential sanity check. This is a substantive critique of what the benchmark measures, not proof that its underlying coding tasks lack value (investigation, ablation discussion).

• Evolved harnesses and interfaces improve fixed executors: Two papers strengthen the case for optimizing the environment around agents.

  • OO-Harness: Specialized subclasses, execution-derived design lessons and routing raise AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor; Workspace-Bench rises from 25.5% to 39.5% (paper summary).

  • Agent S4: A meta-agent builds tools from trajectories without success labels. The reported open-27B OSWorld V2 result reaches 21.3%, 3.8× the same model with hand-built interfaces; ScienceBoard rises from 41.4% to 58.6%. These are paper-reported system results, not model-only comparisons (summary).

• SuperMimic targets cheaper humanoid training: Researchers report training across 129,514 motions using one GPU for three hours, versus a cited state-of-the-art setup of 128 GPUs for seven days. The mechanism combines a learned, differentiable world-model surrogate with backpropagation through time. The claimed >1,000× GPU-hour reduction concerns this motion-control training setup, not robotics learning generally (results, mechanism).

Agent Products, Retrieval Models and API Distribution

• Claude Duet enters research preview: Anthropic opened a waitlist for an Opus-5.5-powered proactive assistant; reported initial access is for U.S. Pro and Max users. Examples include monitoring Slack, preparing questions and making purchases after a request. Stripe describes Link-wallet purchasing and MCP-based invoicing, refunds and revenue analysis. Availability is preview-stage, with no separate pricing disclosed in the supplied posts (Anthropic executive, access, payments integration).

• Retrieval releases target precision and context cost: Cohere launched Rerank 5 in Pro and Fast tiers. Its opt-in Context Focus returns scored passages rather than whole documents, claiming up to 41% lower token consumption while maintaining accuracy. Separately, ParseBench reports NVIDIA’s sub-1B Nemotron 3.5 Parse taking first place across its document-understanding benchmark (Cohere feature, availability, ParseBench).

• Kling 4.0 rollout reaches API access: fal initially announced UI/agent access with API support forthcoming; Kling subsequently announced its official API rollout. Claimed improvements center on complex-motion stability, reference consistency and cinematic control. The later announcement supersedes the earlier “API coming soon” status (fal launch, API rollout).

• Enterprise model access expands while Claude versions its API: OpenAI announced frontier models and Codex workflows coming to Google Cloud Model Garden. Separately, Anthropic’s API version 2026-10-21 removes temperature, top_p and top_k from Messages; new major SDKs default to it, while 2023-06-01 remains supported. Teams upgrading SDKs should explicitly review migration behavior (Google Cloud distribution, version changes, SDK defaults).

Industry and Governance: Researcher Speech and Public-Benefit Oversight

• Non-disparagement clauses draw scrutiny: Posts citing BBC reporting allege that Google employment contracts restrict public criticism indefinitely, discouraging discussion of AI safety and military uses. Former employee accounts and researcher commentary describe a chilling effect. These are attributed contractual and workplace claims; the supplied material includes no company response or legal determination (report excerpts, researcher account).

• Anthropic updates benefit governance: Anthropic published its annual Public Benefit Report and appointed experimental physicist and former Harvard science dean Christopher Stubbs to its Long-Term Benefit Trust. This is a governance development, separate from the company’s model evaluations and product launches (report, trust appointment).

Top tweets (by engagement)

1. NSA: prepare PHP updates; take systems offline if mitigation is impossible — 80,000

2. OpenAI: coordinated PHP2Shell discovery and remediation — 19,600

3. OpenAI: Codex skill for checking PHP2Shell exposure — 10,500

4. NVIDIA: Microduck sim-to-real demonstration — 7,800

5. Richard Socher: Flint local coding model launch — 7,600

6. OpenAI: GPT-Rosalind’s neurodegeneration hypothesis — 7,500

7. Cloudflare CTO: free WAF mitigations are a stopgap—patch PHP — 7,000

8. Nous Research: Hermes orchestrates robot-training workflow at GTC — 6,400

• New from DeepSeek - DeepInfer - (Activity: 441): The image is a screenshot of DeepSeek’s X announcement that DeepInfer, its in-house large-scale inference framework, is now fully open source: image, GitHub repo, X post. DeepSeek describes DeepInfer as the production inference stack behind DeepSeek-V4.1-Flash, optimized for high-throughput, low-latency serving to reduce compute cost and energy use; a linked Zhihu source claims a single cluster reaches 155 trillion tokens/day and about $38/card/hour revenue at off-peak pricing. Commenters framed the open-sourcing as unusually aggressive compared with Anthropic, OpenAI, or Meta, with one saying those labs “would never do this.” Some users were still unclear on the practical role of the framework, asking simply: “What do it do?”

  • A linked Zhihu source claims DeepInfer enables high-throughput, low-latency inference for DeepSeek V4.1 Flash, reporting single-cluster throughput of 155 trillion tokens/day and revenue of $38 per card per hour at off-peak pricing:

https://zhuanlan.zhihu.com/p/2095560295930508953

• • . Commenters highlighted this as unusually aggressive public disclosure compared with companies like OpenAI, Anthropic, or Meta.

  • One technical concern was whether DeepInfer is a truly general-purpose inference framework or primarily tuned for DeepSeek’s own model architecture. A commenter noted that flashy framework launches tied to a flagship model often hide architecture-specific optimizations that may not transfer cleanly to other LLMs.

  • A performance figure mentioned in the thread was roughly 256 tokens/s × 100 users, which commenters characterized as extremely high concurrent serving throughput. The implication is that DeepInfer may be targeting production-scale multi-user inference rather than single-request latency benchmarks alone.

• Eagle-4 Released: Faster than DFlash2 & DSpark (Activity: 306): EAGLE-4 is a source-available speculative decoding stack using hardware/workload-calibrated draft trees, with eagle-harness integration paths for SGLang, vLLM, and TokenSpeed; current reported support/benchmarks center on Qwen3.8-27B. The post cites speedups over vanilla decoding of 2.22× for MTP, 2.57× for DSpark, 2.80× for DFlash 2, and 3.59× for EAGLE-4, while the repo reports SPEED-Bench gains on 4× NVIDIA B300 of 3.42× SGLang, 2.83× vLLM, and 3.09× TokenSpeed at single-request concurrency; gains reportedly shrink at higher concurrency (EAGLE-4). Commenters also point to LiLiCorr, used by Nemotron Bolt, as a competing speculative decoder with more granular benchmark reporting (paper). Commenters questioned whether the DSpark/DFlash comparisons are representative because aggregate averages can hide workload sensitivity—e.g. MTP may do better on structured outputs such as code/JSON—and asked for breakdowns by workload type. Another concern was practicality: the EAGLE-4 draft model is reportedly about 3 GB larger than DFlash2, making it less attractive for VRAM-constrained deployments.

  • Several commenters questioned whether Eagle-4’s reported speedups over DFlash2 and DSpark are representative across workloads. One noted that averaged results may hide large variance, since MTP can perform better on structured-output tasks such as code or JSON, and asked for acceleration broken down by workload type rather than aggregate benchmarks.

  • A practical deployment concern was raised around memory footprint: Eagle-4’s draft model is reportedly about 3 GB larger than DFlash2, which may make it unsuitable for VRAM-constrained local inference setups already at capacity.

  • Commenters pointed to competing or adjacent speculative decoding work, especially LiLiCorr, used by Nemotron Bolt, whose paper reportedly provides more granular benchmark analysis: https://arxiv.org/pdf/2608.20530. There was also interest in comparing Eagle-4 against HyperQwen or whether HyperQwen could integrate it: https://github.com/syv-ai/HyperQwen.

• Now that 3090 frenzy warning post makes sense. (Activity: 405): The image is a screenshot of an X/Twitter post claiming an RTX 3090 “gpu-unlock” kernel/module can raise BF16 tensor math throughput substantially, e.g. 82→155 TFLOPS for BF16 tensor pipe and 64→99 TFLOPS for cuBLAS BF16 GEMM, allegedly by removing an NVIDIA-imposed limitation on FP32-accumulate tensor ops. In context, the Reddit OP asks whether this explains a prior “3090 frenzy” warning and whether old anti-mining or product-segmentation restrictions might still affect LLM workloads, but commenters note that synthetic BF16/GEMM gains may not translate to real inference/training speedups. Commenters are skeptical of the market-hype angle: one calls predictions of used 3090s becoming more expensive than 5090s “delusional,” while another links a prior refutation arguing the hack has not shown practical workload gains. The dominant view is cautious: the benchmark claims are interesting, but “1.9× faster” may amount to much smaller end-to-end improvements.

  • Several commenters argued that the rumored RTX 3090 “hack” is being overhyped because the measured benefit appears limited to a narrow workload slice: roughly 10–12% prefill improvement, not a broad 1.9x real-world LLM inference speedup. One linked the prior thread’s top refutation, emphasizing that the apparent low-level BF16 gains do not yet translate into practical end-to-end workloads.

  • A technical concern raised was that BF16 operations may be only a small fraction of inference time, so accelerating or unlocking BF16 paths on RTX 3090 would have limited impact unless the bottleneck is actually BF16 compute rather than memory bandwidth, KV-cache access, sampling, or other kernels. Another commenter noted adoption would depend on whether the modification can be made safely and remain stable across CUDA and NVIDIA driver updates.

  • On pricing, commenters were skeptical that used RTX 3090 values would materially spike from this news, arguing that GPU pricing is driven more by broader industry supply contracts and new-card positioning than by niche LLM hobbyist demand. One comparison cited a new Radeon RX 9700-class 32 GB card around ~$2k as potentially better value than an inflated used 3090, though the exact product naming/value claim was not deeply benchmarked in the thread.

• ornith-ai/Ornith-1.5-122B (Qwen finetune) (Activity: 438): Ornith AI released Ornith-1.5-122B, a 122B MoE reasoning model fine-tuned/continued from Ornith-1.0 on a Qwen 3.5 / Gemma-derived lineage, with a self-improvement loop that jointly optimizes task generation, scaffold construction, and solution rollouts via RL rather than using only fixed human-curated tasks. Artifacts include GGUF, unsloth GGUF, and NVFP4 variants; the model card claims strong coding/agentic benchmark performance, 262K context with optional YaRN scaling toward ~1M, and serving support via vLLM/SGLang/OpenAI-compatible APIs plus llama.cpp/Ollama-style GGUF paths. Commenters questioned the choice of a Qwen 3.5 122B base versus newer “Next Flash”-style bases, arguing it may already be behind newer architectures. One technical concern was inference speed: because it is based on Q3.5 122B with A10B active parameters and lacks sparse attention, a commenter estimated it will be roughly 2× slower than Q3.8fn-style models with A6B active parameters.

  • A commenter notes the finetune is based on Qwen 3.5 122B, implying A10B rather than A6B behavior for q3.8fn users and no sparse attention, which they estimate will make inference roughly 2× slower for that setup.

  • There is technical skepticism about the base-model choice: one commenter asks why it was not built on Ornith Next Flash, arguing that while Qwen 3.5 is a strong base, it has been “left in the dust” relative to newer alternatives.

• RecursiveSI/flint (40B A2.5B) · Hugging Face (Activity: 784): RecursiveSI/flint is an Apache-2.0 decoder-only MoE coding-agent model on Hugging Face, with 39.8B total parameters but only ~2.5B active per token. It uses 49 layers, 256 routed experts per MoE layer with 6 selected per token, mixed sliding-window/full attention, and a documented 262k context; Recursive reports 54.9% on Terminal-Bench 2.1, 89.7% on SWE-bench Pro v2, and 74.2% on SWE-bench Verified, targeting SGLang serving on B300/H200 GPUs with OpenAI-compatible tool use. Commenters framed it as roughly the “35B-A3B” class model people wanted from Qwen 3.8, but noted that the existence of Qwen3.8-27B makes the release feel less surprising; otherwise, the benchmark screenshot was the main technical discussion point.

  • Commenters position Recursive Flint as a 40B MoE with only 2.5B active parameters, roughly matching the class of “35B-A3B” sparse models some users wanted, but now competing against newer small-active MoEs like Qwen3.8-27B. The thread notes that without Qwen’s recent release, this would likely be viewed as a much more significant drop.

  • A technical NVIDIA GTC Berlin announcement frames Flint as a local coding model optimized for RTX Spark, claiming over 220 tokens/s on a 64GB RTX Spark and strong results on agentic coding benchmarks including SWE-Bench Pro v2, Terminal-Bench 2.1, and SWE-bench Verified: https://blogs.nvidia.com/blog/local-ai-gtc-rtx-dgx-spark-station-sync/

  • NVIDIA says Flint was built using Recursive’s Eureka Machine, an automated AI research loop that proposes ideas, implements them, runs experiments, and selects follow-ups. The cited post claims this system produced specialized kernels that improved Flint generation speed on RTX Spark by 55% without changing model weights, and notes Apache 2.0 open weights plus support for post-training via NeMo Automodel SFT/LoRA and NeMo RL GRPO.

• Pocket TTS which supports voice cloning is now just 1.9 million parameters (Activity: 561): Kyutai released Pocket TTS-small, compressing a CPU-real-time voice-cloning TTS stack from ~100M parameters down to 18.2M, 6.9M, and 1.9M variants. The compression pipeline uses distillation from a ~300M teacher, single-pass baked-in classifier-free guidance, a PCA-derived 64-D backbone–head latent interface, structured transformer pruning, separate backbone/head/codec distillation, and optional reward fine-tuning; reported CPU throughput reaches ~9.9× real time for the 18.2M model and ~25× real time for the 1.9M model, with the 18.2M model matching or slightly exceeding the original on WER/UTMOS. Commenters focused less on the compression method and more on an apparent profanity filter in the released model/demo—e.g. “What the fuck” being rendered as “What the dee”—though an update says the team fixed the issue. Others asked for more rigorous benchmarks on naturalness, real-time factor, and cross-speaker voice similarity, especially for fully local inference.

  • Several users reported an apparent text censorship/regression in the TTS output: phrases like “What the fuck” were synthesized as “What the dee”. The top commenter later noted that the team addressed the issue and posted a fix, suggesting it was likely an unintended model/tokenization/filtering behavior rather than a permanent policy choice.

  • A commenter highlighted that the headline parameter count—1.9M parameters—is impressive, but asked for concrete evaluation data: naturalness, real-time factor, and voice similarity across speakers. They emphasized that tiny TTS models are most compelling when they can maintain fully local inference without sacrificing speaker cloning quality.

  • One user reported poor non-English language support, claiming non-English inputs were effectively rendered as English rather than the target language. This suggests multilingual capability may be weak or absent despite any broader TTS/voice-cloning claims.

• EXCLUSIVE: ByteDance, Alibaba, Tencent place over $35 billion worth of Nvidia chip orders (Activity: 1047): Reuters reports that ByteDance, Alibaba, and Tencent have placed >$35B in Nvidia chip orders (Reuters). The Reddit post frames this as China ordering roughly 3M RTX PRO 5500 cards at about $11,400 each, likely targeting LLM inference and AI video generation, while a quoted source says Nvidia may fulfill only 5–10% of requested quantities and can produce at most 3M RTX PRO 5500 cards globally in 2027 versus > 6M requested by Chinese customers. Commenters argue that supply constraints mean the headline order volume is unlikely to be fully delivered, and some speculate Nvidia may deprioritize lower-margin workstation-class cards as datacenter demand dominates. There is also skepticism that future GPU generations will make current RTX PRO cards affordable for local LLM users, though some hope datacenter refresh cycles could depress used-market prices.

  • Commenters highlighted a reported severe supply mismatch: Nvidia may allocate only 5%–10% of requested quantities to partners, with one source saying it can produce no more than 3 million RTX PRO 5500 cards globally in 2027 while China customer orders exceed 6 million. This implies the $35B+ order book may be heavily supply-constrained rather than fully deliverable demand.

  • A pricing correction noted the implied unit cost is about $11,400 per card, not $11.40, aligning the order size with high-end professional/datacenter GPU pricing. Another technical concern was that high-margin enterprise demand could make even cards like the $16,000 RTX 6000 Pro less attractive for Nvidia to prioritize versus datacenter allocations.

  • Several commenters inferred that GB202-class silicon may be diverted toward professional or export-compliant SKUs such as RTX PRO cards, reducing availability for consumer GPUs like the RTX 5090. The underlying concern is that datacenter/professional demand could continue to crowd out gaming GPU supply until newer generations shift demand away from current silicon.

• Be prepared, GPU price increase incoming --> Samsung sells out 100% of all 2027 HBM capacity - HBM prices expected to jump 200% next year, DRAM prices expected to see a double-digit price increase (Activity: 1227): GSMArena reports that Samsung has allegedly pre-sold roughly 70% of next year’s general-purpose DRAM output and 100% of its HBM capacity under large multi-year contracts with big-tech customers. The resulting memory supply crunch is expected to push HBM prices up 150–200% and DRAM prices by double digits in Q1 2027, increasing BOM pressure for GPUs/AI accelerators and downstream devices such as future Galaxy phones. Commenters expect the HBM squeeze to worsen consumer/local-AI GPU affordability, while noting the dependency loop: frontier models require hyperscaler GPU clusters, but local inference users compete for the same constrained memory supply. Some argue scarcity may force optimization toward smaller, more efficient models and longer hardware lifecycles.

  • Several commenters connect the reported Samsung 2027 HBM sellout and expected HBM/DRAM price hikes to worsening economics for local AI hardware: local inference users need GPUs, but hyperscalers and AI labs are simultaneously absorbing supply for training and datacenter expansion. One commenter notes that GPU acquisition appears to be accelerating even before matching datacenter capacity is built, implying inventory stockpiling could keep consumer/prosumer GPU supply constrained for years.

  • A technical-economic point raised is that the break-even point for local AI rigs vs. cloud inference has shifted outward as GPU and memory prices rise while some cloud AI services have become cheaper. The commenter specifically argues this is amplified by MoE models, since datacenters can multiplex sparse expert activation across many users more efficiently than an individual local machine can justify with expensive VRAM-heavy hardware.

  • One commenter argues that hardware scarcity may push model-side optimization, incentivizing smaller, more efficient models rather than simply scaling parameter counts and memory bandwidth. The implication is that high HBM/GPU prices could accelerate work on compression, quantization, distillation, sparse architectures, and improved inference efficiency.

• When will we see these changes? --> China Is Doubling RAM Production to Keep Everyday Gadget Prices from Skyrocketing (Activity: 402): The post asks when consumers will see price effects from China reportedly doubling RAM/DRAM production, likely via domestic suppliers such as CXMT, in response to rising memory demand. Top technical comments argue the effect will be gradual: new DRAM capacity ramps over quarters/years, is absorbed by latent demand in areas like edge compute and datacenters, and may mainly reduce the rate of price increases rather than create an obvious consumer price drop. Commenters are skeptical that global buyers will see near-term savings: one view is that Chinese output will mostly satisfy domestic Chinese demand, reducing China’s imports rather than flooding export markets. Another argues manufacturers will not abandon high-margin enterprise/datacenter sales for low-margin consumer DRAM, so end-user gadget prices may remain elevated.

  • Several commenters emphasized that additional DRAM capacity will appear gradually rather than as a visible price drop: new fabs ramp over years, and added supply may be absorbed by latent demand from edge compute, datacenters, and consumer devices. One prediction was that the main near-term effect is slower price inflation rather than cheaper RAM, because vendors are unlikely to abandon high-margin enterprise/AI demand for low-margin consumer sales.

  • A detailed supply-chain comment cited CXMT’s target of 600,000 wafers/month by 2028 as the relevant timescale, arguing that memory capacity expansions triggered by the AI boom will start arriving from late 2025 onward and continue through 2030. The commenter also noted parallel global expansions: U.S. fabs in New York, Idaho, and Indiana tied partly to the CHIPS Act; Samsung adding 4th and 5th facilities at its memory hub; and SK Hynix investing roughly 35 trillion won in its Y2 plant plus 19 trillion won in its F17 fab.

  • Another technical/economic point was that China’s increased DRAM output may primarily displace imports rather than flood global consumer markets: CXMT could satisfy domestic Chinese demand first, especially with aggressive datacenter construction. Commenters framed memory manufacturing as highly cyclical and capital-intensive, where overbuilding can turn record profits into bankruptcy because trillion-dollar-scale fabs cannot be quickly repurposed or liquidated if demand falls.

/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo, /r/aivideo

• Claude Duet: Anthropic just casually dropped the personal assistant calling it a proactive agent. This is the response to Muses, Bots and Dots (Activity: 1306): The image is a screenshot of an X/Twitter announcement from Claude introducing Claude Duet, a research-preview app positioning Claude as a proactive personal agent rather than a purely chat-based assistant. The post claims Duet can surface items needing attention, complete requested tasks asynchronously while the user is away, and request confirmation before acting on the user’s behalf; one commenter notes it will reportedly be available on the $20 plan. Commenters were mixed: one said this matches how they already use Claude and questioned what differentiates Duet from other agent/app concepts like Muses, Bots, and Dots. Another top comment was a non-technical joke and adds no substantive technical context.

  • Commenters noted that Claude Duet appears to be included in Anthropic’s existing $20/month plan, suggesting Anthropic is positioning the proactive-agent/assistant layer as part of mainstream paid Claude usage rather than a separate enterprise-only product.

  • Several technical critiques argued that current “assistant” launches from Anthropic/OpenAI-like vendors still look like thin chatbot wrappers over different frontier models, rather than fundamentally new agent systems. Users questioned what differentiates Claude Duet from existing personal workflows using Claude, Muses, Bots, or similar assistant abstractions.

  • One commenter argued that the missing piece is not necessarily model capability but structured personal-data integration and product orchestration: a useful assistant would need better schemas, context management, and integrations with external services. They suggested the required technology already exists, but needs to be packaged into a coherent assistant experience rather than another generic chat UI.

• I used Claude to fix my Samsung TV (Activity: 3894): A user reports using Claude as an interactive agent to diagnose a Samsung TV as Tizen rather than Android, then work around the lack of webcam access by serving a local web page, generating a local-server QR code, and using the phone camera to capture TV screenshots every 4s for visual feedback (QR screenshot). The agent allegedly spent ~30 min changing privacy/telemetry settings, deleting the Samsung advertising ID, disabling unused apps, rearranging icons, then building a custom “Apple TV-style” launcher/home-screen app plus a companion settings app for shortcuts/backgrounds (launcher, settings UI). A top commenter describes a similar custom TV launcher pipeline for guests: pre-arrival preference intake, personalized Plex/media/news surfacing, music/background/welcome screens, itinerary/weather display, and Home Assistant controls for outdoor lights, pool aerator, and misters. Commenters were mainly surprised that the user even attempted this level of autonomous TV debloating/customization with Claude; one framed it as a practical example of agentic UI automation beyond standard coding assistance. The cat-video detour was treated humorously rather than as a technical failure.

  • A commenter described using Claude to build/replace a Samsung TV launcher with a custom guest-oriented interface: questionnaire-driven personalization for Plex movies, TV/news station presets, background music, weather, itineraries, and local recommendations. They also integrated the launcher with Home Assistant so guests could control outdoor lights, pool aerator, misters, and other smart-home devices directly from the TV home screen.

  • Another commenter suggested avoiding a custom Claude-driven workflow by installing cleanyourtv, a GitHub project with roughly 15,000 stars aimed at cleaning/debloating smart TVs. They framed it as a faster, lower-token alternative to having Claude manually guide or generate the TV-fixing process.

  • One technical detail called out was the removal of the TV’s Ad ID, implying the original Claude-assisted fix included privacy/debloating steps rather than only UI tweaks. This was viewed as a notable privacy-oriented modification for a Samsung smart TV.

• I replaced my Samsung TV’s home screen with Claude Code (Activity: 1069): The image shows a custom Samsung TV launcher replacing the stock ad-heavy home screen with a minimal web-style interface: clock/date, selected app tiles, console shortcuts, settings, and remote-control hints. The author says Claude Code helped build the launcher and speed iteration by mocking Samsung TV APIs in a laptop browser; Home-button takeover reportedly uses a CleanFrame-like helper that edits undocumented TV settings, tested only on a 2022 Samsung Q70B. Code is available on GitHub, with a longer write-up. Comments were skeptical and polarized: one commenter praised the project as a way to avoid Samsung/LG telemetry and ads, while others suspected spam/scam reposting or dismissed it as repetitive.

  • Commenters raised privacy/security concerns around proprietary Samsung/LG smart-TV software, arguing that an open-source replacement UI/OS could reduce vendor telemetry and unwanted tracking. One technically relevant caveat was the risk that “vibe-coded” TV firmware or home-screen replacements could brick devices if deployed without understanding the platform’s boot/update constraints.

  • A security-focused thread warned that encouraging users to run unvetted TV code could create a large attack surface: compromised replacement launchers or scripts could enable synchronized remote control of many TVs, e.g. forcing playback of attacker-chosen video across devices.

  • One user asked whether replacing the Samsung home screen produced measurable latency improvements, noting that stock TV launchers feel “slow as a pig.” They also referenced using Claude to install Jellyfin, suggesting interest in whether an AI-generated TV launcher could improve responsiveness for local media workflows compared with OEM smart-TV software.

• I handed Claude Code a database of 11 million news articles, it found some weird patterns (Activity: 1604): The image is a technical dashboard screenshot, not a meme: it shows CLSTR’s dark-themed “The world right now” map, where a news-ingestion pipeline visualizes current global event volume by country. In the post, the author says the system processes ~100k articles/day from 40k+ sources in 40 languages, clusters them into events, and exposes derived analytics after 11,079,667 articles; the screenshot reports 2,703 events across 161 countries in the last 3 hours and highlights a major cluster, “France implements curfews amid widespread civil unrest,” with 673 articles. The important clarification is that Claude Code did not read 11M full articles; it queried/exported aggregated cluster metadata produced by the author’s pipeline, which uses embeddings, heuristics, and an LLM judging step. Commenters debated whether the methodology supports the claims: one skeptical thread argued Claude likely only wrote heuristic analysis code and that source-selection/clustering biases could dominate the observed patterns. Others were more optimistic, viewing this as an example of LLM agents helping surface large-scale patterns in real-world data.

  • Several commenters questioned whether Claude Code actually processed 11M articles directly, arguing that full LLM-based analysis at that scale would be prohibitively expensive and slow. A more plausible implementation would be Claude writing auxiliary code or heuristics to aggregate/search the corpus, but commenters warned that such heuristics need to be documented because they may dominate the results.

  • A major methodological critique was the lack of detail on data collection, source selection, and sampling bias. Commenters noted that if sources came from Google News or another aggregator, the observed patterns could reflect ranking, regionality, click-history personalization, or source-availability bias rather than real media behavior.

  • Commenters also flagged potential language/source coverage issues, including the claim that unknown-language items were labeled as English and the apparent absence of major non-English corpora such as Russian, Chinese, or Korean. They argued that without source documentation and language-labeling validation, conclusions about global news patterns would be technically weak.

• Waymo’s robotaxis had 95% fewer serious crashes than humans. Now it plans a fleet of 20,000 cars (Activity: 1457): Waymo’s latest safety data reports 271.3M rider-only miles with 0.01 serious-injury/fatal crashes per million miles versus a modeled local human benchmark of 0.21/M, i.e. ~95% fewer serious crashes, plus ~82% fewer injury crashes. An IIHS analysis of 2021–2024 federal crash data over ~50M driverless miles found directionally similar reductions—68% fewer police-reportable crashes and 81% fewer injury crashes—but noted reporting limitations and an Austin result that was slightly worse than the human benchmark. Semafor reportedly says Waymo plans to scale from ~4,000 vehicles to 20,000 by end-2027, partly via a $5B debt raise. Commenters debated whether the benchmark is apples-to-apples: supporters argued serious crashes are common enough that the reductions are meaningful and that Waymos are rarely at fault, while skeptics noted Waymo’s current ODD is mostly dense urban, lower-speed, limited-highway driving—unlike rural/suburban or high-speed highway exposure where many severe crashes occur.

  • A commenter claiming experience in OEM crash-safety engineering argued that the reported Waymo safety results are consistent with a decade of AV studies: human driving is a high-risk daily activity, while autonomous systems have generally shown lower serious-crash rates. They emphasized that the reported metrics appear to be based on total crash volume rather than fault, noting that Waymos are often involved in crashes caused by human drivers rather than being the initiating party.

  • Another commenter raised an exposure-bias concern: Waymo vehicles reportedly operate mostly in dense urban environments and rarely on highways, where high-speed crashes and severe injuries are more common. They suggested that the 95% fewer serious crashes claim may not generalize to rural, suburban, or frequent highway operation where speeds exceed 50 mph more often.

  • A commenter noted that Austin-specific crash-rate comparisons may be statistically weak because the study period allegedly included only 2 Waymo crashes there. They argued this sample size is too small to support strong conclusions about whether Waymo is safer or less safe than human drivers in that market.

• Unitree’s newest robot: H2-X Kung Fu Legend Guardian (Activity: 3907): Unitree announced the H2-X “Kung Fu Legend Guardian”, apparently a more rugged/high-force H2 humanoid variant, claiming maximum endurance of ~8 hours and framing it as a “serve and guard” platform in its X post. The post contextualizes it alongside other Chinese humanoid combat/entertainment robots, notably Engine AI’s T800, used in fighting-tournament demos per this writeup; the Reddit-hosted video itself was inaccessible due to a 403 Forbidden block. Top comments focused less on specs and more on safety/dual-use concerns, joking that the demo robot appeared to shove a child with more force than a slow car would and sarcastically hoping the technology would not be misused.

• Fibo 2 Edit is what Klein 3 should’ve been. (Activity: 1189): Reddit user reports Fibo 2 Edit (HF model) as a fast, out-of-the-box “uncensored” image editing model with no observed color drift/quality loss in simple edits, attributing some quality gains to the Flux 2 VAE. They link experimental ComfyUI support via PR Comfy-Org/ComfyUI#17240, but later update that it struggles with complex prompts and multi-reference edits, and is “not better than Qwen, just faster.” Comments frame Fibo 2 Edit as potentially “best edit model so far” largely because it is uncensored, with a side debate implying censorship may degrade model utility. Another commenter highlights rapid open-weight image-model progress, citing upcoming/available models like Ideogram 4, Krea 2, Qwen 2.1, LTX, and quantized Hunyuan Image 3 running in ComfyUI on consumer hardware.

  • Users highlight Fibo 2 Edit as a strong image-editing model and note that it is reportedly uncensored, with one commenter claiming it is the “best edit model so far.” The discussion frames this as part of a broader trend where less-restricted image models are perceived to outperform more heavily filtered alternatives in practical editing quality.

  • A technically relevant limitation was reported around output resolution: Fibo 2 Edit allegedly downscales source images to roughly 1MP, then processes them through the Flux 2 VAE, causing blurry results. This suggests the model or current pipeline may be bottlenecked by preprocessing/VAE resolution handling rather than edit semantics alone.

  • Commenters mention interest in ComfyUI support and broader open-weight ecosystem momentum, citing models such as Ideogram 4, Krea 2, Anima, Qwen 2.1, Moss Image, Ming Image, Minimax H3, LTX, and Hunyuan Image 3 quantized. The latter is noted as potentially runnable in ComfyUI on consumer hardware, implying quantization is making larger image models more accessible locally.

• As of 3 hours ago, the Linux Kernel no longer requires the usage of AI to be declared for new code submissions. (Activity: 1071): A screenshot of torvalds/linux shows commit 14c8977 by Linus Torvalds, merging Linux documentation updates that include removal of the prior “Assisted-by” requirement for new kernel submissions. In context, this means contributors no longer need to explicitly declare AI assistance when submitting code to the Linux kernel, shifting the policy away from mandatory AI-use disclosure. Comments were split between approval and anticipated backlash: one commenter argued that assumptions like “AI code is all slop” are outdated, while another expected a community meltdown. A practical commenter suggested they might now submit a long-standing local driver fix using Claude, implying the change could lower the barrier for some contributors.

  • One commenter notes they have been carrying a local kernel patch across releases for a recurring driver bug affecting some motherboards with a specific chip, and the relaxed AI-disclosure rule may make them more likely to upstream the fix using Claude. The technical implication is that reducing disclosure/process friction could increase submissions of small, hardware-specific fixes that otherwise remain private local patches, though the comment does not identify the driver, chipset, or exact bug.

For the last few weeks I’ve used Claude Mods to mold Claude Code to my AI engineering workflow. Claude Code is great out of the box, but every engineering workflow is different. Claude Mods let you make Claude Code yours, in the way you want to pay attention.

The “paying attention” part is now the most important thing.

In a recent Latent Space post, The Evolution of the Agent Harness, I argued that as models absorb more of the harness into their weights, the part that’s left faces the human. It becomes the Attention Interface: the layer that decides when the agent interrupts you, what it shows you, and which decisions it needs you to make. Claude Mods are an early version of the Attention Interface, and one you can shape yourself.

Here’s my bet: ultimately, the harness becomes a platform that other companies build on, with models as the engine. Just as the browser became a platform for web apps, the harness could become a platform for the interfaces between people and agents.

A mod is a Claude Code plugin that consists of TypeScript or JavaScript functions that run inside Claude Code. These functions customize the behavior and appearance of the interface in the CLI and Desktop. Mods respond to events, like a prompt, tool call or interface render. They can observe the event, modify how it’s rendered, and handle it in place of the default behavior.

Claude Mods are middleware for the agent harness. They inject code that changes Claude Code’s execution and rendering flow, changing how the application behaves at runtime.

Claude Mods let the human and the agent build the human-facing part of the harness together. You can now work with Claude to design how Claude works with you. That is the essence of the “Attention Interface.”

The agent harness started as an interface between the agent and the machine it runs on. It lets you prompt the agent and the agent then sends commands, edits files and calls tools. Now it’s gradually becoming a way for the agent to interface with the human.

Claude Mods let you reverse the direction of the harness. Rather than the harness pointing from the agent to the machine, you point it from the agent to the human operator, and the harness becomes a way to interface with human attention.

Claude Mods make the human layer programmable, by the human who uses Claude.

Claude can also build mods for you. Claude Code has a built-in skill that teaches it how to build mods, so you can ask for the mod you want in plain language and Claude will build it.

Some of Claude Code’s own features are already mods. Anthropic has shipped six of them built-in. You can see these by running /plugin and opening Installed, where they’re listed under Built-in:

• agents-md: loads AGENTS.md as project instructions.

• diff: takes over /diff and draws its pane.

• plugin-authoring: gives Claude the skill it uses to write mods for you.

• sec-default: guards what your organization manages from the mods a user installs.

• telemetry: sends the analytics records that Claude Code and its built-in mods log.

• you-should-know: a side agent that shows a note above the prompt when it finds something you might have missed. It’s the only one that’s off by default.

Most of these work in the background, which is interesting because it means Anthropic is building parts of Claude Code as mods — even parts you may not notice. The one I’ve turned on myself is You Should Know.

Here are the two mods I’ve found most useful in practice: one a built-in, and one I built myself.

This is a native Claude Code plugin built by Anthropic. It runs a side-agent that observes the main agent while it works and flags important information you’re likely to miss. Think about it like: tool calls and decisions that have a material impact on the output, but aren’t necessarily easily visible because they’re buried in the agent trace. It comes built into Claude Code, but it’s off until you turn it on.

Here’s a real example. I asked Claude Code to simplify the noisy UI text in one of my personal projects. The main agent trimmed the copy on the Compare screen, installed the new build and took before-and-after screenshots. Everything looked done.

Then You Should Know flagged something for me. CompareView.swift, the file the agent was editing, already had uncommitted work in it — and so did about 45 other files. If I later discarded the demo edits, I’d throw away that earlier unfinished work too. Its suggestion was to commit or stash the earlier work first.

In this case, the You Should Know mod saved me from discarding work unintentionally.

This mod only runs after a turn that edited files, ran commands or made 8 or more tool calls. In the screenshot below, you’ll see six different pieces of information flagged for your attention. Next Steps Supervisor then suggests a next step to take, as a clickable button which sends the instruction as a message to Claude Code. This mod is available as part of ModSmith — an open-source Claude Code plugin I built with several different built-in mods, as well as a template Claude can use to build new mods that work together.

Anthropic isn’t the only company that’s building an Attention Interface. Two recent products, one from OpenAI and one from Cursor, point in the same direction. Using the new ChatGPT Intelligent UI and Cursor’s PR Routing & Approval as a frame of reference shows what’s unique about Claude Mods.

On October 7, OpenAI shipped GPT-6 inside ChatGPT alongside a new ChatGPT interface it calls Intelligent UI. Rather than answering only in text, ChatGPT now can generate an interface as it answers: charts, maps, checklists, forms, and small tools like a bill splitter or a savings calculator. All generated on the fly by the model as a type of generative UI.

In one of OpenAI’s examples, ChatGPT replies with multiple-choice follow-up questions to narrow down what you’re looking for. Think about it like: the model decides when it needs your judgment, then builds the interface for you to give it.

That’s an Attention Interface. The agent designs how you interact with its response.

But notice the difference between ChatGPT’s Intelligent UI and Claude Mods. For Intelligent UI, the model picks the interface, answer by answer, from a component library that OpenAI built. The one control mentioned at launch is turning the visuals down.

Cursor is working on the same problem from the other end: not one person and one answer, but a team and a stream of pull requests, more and more of them written by agents. Someone has to decide which changes deserve the attention of the human-in-the-loop.

PR Routing & Approval makes that decision. It assigns reviewers based on code ownership and commit history, and it can auto-approve low-risk PRs. It weighs a risk score against a threshold you set, findings from Cursor’s Bugbot and Security reviewers, and policy files that live in the repo.

One rule caught my attention during research. If a PR edits an approval policy, the agent won’t use the edited version to relax review on that same PR. It reads the policy from the base branch instead. That means an agent can’t change the rules to get its own PR approved.

Cursor is also clear about the limits. Its docs say the feature “does not replace a full code review.” The point is to spend human review where it matters, not to remove it.

All three systems shape what gets your attention and how. The main difference is who gets to design the Attention Interface.

• ChatGPT Intelligent UI: the model designs the interface for each answer, inside a frame the lab built. Zero setup, but little control. (I don’t believe the target user is engineers.)

• Cursor PR Routing & Approval: the team writes policy, and a fixed review workflow enforces it. Configurable, but the flow of the interaction is Cursor’s. You decide the rules, not the interface.

• Claude Mods: you and the agent build the interface itself, in code. A mod can hook into any point in the loop — from the prompt you submit to the tool call the agent makes to the moment a turn ends — and decide what to show you there and what you can do about it.

The supervisor I showed earlier is a good example. Cursor could easily route a PR to me because of the files it touched. ChatGPT could just as easily render the verdict as a nice card. But neither would let me decide that every turn which edits a file gets a second opinion, that the opinion lists shortcuts taken, and that each suggested next step is one keypress away. That interface exists because I prefer to pay attention that way, and Claude built it with me.

ChatGPT and Cursor give you an Attention Interface that someone else designed. Claude Mods let you design your own with Claude, but the tradeoff is you’re responsible for making it useful.

Mods aren’t free, and I’ve run into some downsides.

The first is noise. Next Steps Supervisor runs after any turn that edits a file, runs a command or makes 8 or more tool calls. In practice, that’s nearly every turn. Since I installed it on October 15, 99 of my 111 turns in Claude Code met that bar. Some of those verdicts are useful, others are just noise. So when a mod speaks up on almost every turn, I start skimming past it — which defeats the point of an Attention Interface.

The second is cost. Each supervisor check forks the conversation and re-reads it. Most of that is served from the prompt cache, but the mod’s own README estimates about 5–6k input-equivalent tokens per check on a 50k-token session, and more when the cache has expired.

The third is trust. A mod is code that runs with your permissions. Anthropic’s docs are clear about this: once it loads, a mod can read and write your files, see every prompt you send, and spend your usage. Plus, “mods aren’t sandboxed.” Built-in mods also keep running when you start Claude Code with --safe-mode. I’m comfortable with mods I wrote or that ship with Claude Code, but I’d read anyone else’s code before installing it.

The upside is that mods are fixable in a way other tools aren’t. When the supervisor is noise for a session, /supervisor off turns it off. Because it’s my mod, I can change when it runs.

Customizing the harness isn’t guaranteed to be a win for every individual engineer. Every mod I add is another piece of information I need to give my attention to. In one of my own sessions, my Next Steps Supervisor marked a task as “Solved” yet also listed two gaps and three shortcuts. Agents are intelligent but far from perfect, and my attention is far from perfect too.

So the test I use to gauge the usefulness of my mods is simple: did it surface something I would have missed, make a decision easier, or let me act without getting distracted by more output? If not, it’s just window-dressing rather than a useful tool.

Right now, building a mod works like this: I notice friction in how I work with Claude, describe the mod I want, and Claude builds it with me. My personal attention is still needed to notice the friction.

The next step is for the harness to notice the friction. Claude Code already keeps my session transcripts. Think about it like: a background process reads through them, finds the places where I keep correcting the agent or checking the same thing by hand, and suggests a mod for it. Then it helps me test that mod and refines it from my feedback.

That’s the best case for AI engineers: a malleable harness that suggests ways to improve itself from real-world use. Claude Mods are a step in that direction.

If the bet I made at the start is right, in a few years we will talk about the startups that were built on Claude Code, or Codex, or a harness that doesn’t exist yet.

I’ll leave you with a question, where the answer hopefully provides you with practical value: what friction have you noticed working with Claude Code that this article can inspire you to solve with a mod today?

Proudly brought to you by SerpAPI: Today’s release covers a LOT of ground, but you can scrape all this into markdown for your agents! 50% token cost of JSON.

We last covered the Inference Inflection in April, and historically most of the advances have been closed source decacorns from Fireworks to Baseten to Modal.

Well, time for the open source response. It is day 1 of PyTorch Conf, and DeepSeek timed an intercontinental ballistic missile into the discussion by open sourcing DeepInfer, their production stack, capable of up to 72 hour prompt caching to the 12 hours OpenAI just announced:

This is their customization of vLLM with credit to SGLang and TRRTLLM, but there is a lot added… minus their crown jewels - their ultra optimized KV cache (compact + >8hour lifetime), PD/disggregation, and load balancing (their 3FS was open sourced a while ago). We have translated their blogpost and published it here together with a complete research guide and explainers and codemap and subsystem guide:

As we point out in their additional disclosure, their claim to serve 155T tokens/day gets them $38 per SM100/A950 card per hour at off peak prices (currently $0.15/$0.60 per million input/output tokens: 155000000 tokens * $0.14 blend cost / 24 hours is ~$1m per hour). Taking their production data (which we translated) of 1.5 million cache-inclusive input tokens/second per prefill GPU, 96% input cache hit, and 14,000 output tokens/second per decode GPU (over ~2400 GPUs):

with all-in costs $5-8 per card per hour that is an >80% inference margin!

Doing some cluster math, that is roughly $166B per GW, with 81 day payback period on inference margin (DeepSeek famously plans for 10 month payback). Imagine what the Whale could do with more GWs.

Sponsored by SerpAPI!

Give Your AI Agents Live Search Data with Fewer Tokens

Receive results from any of SerpApi’s 100+ APIs as clean, structured Markdown, using roughly 50% fewer tokens than JSON on average. Simply add output=md or use the /search.md endpoint. It requires zero configuration and is included with every plan at no additional cost.

AI News for 10/19/2026-10/20/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

Top Story: DeepSeek DeepInfer—Production Inference, Open-Source Boundaries, and the Economics of Cached Tokens

DeepSeek released DeepInfer, the inference engine behind V4.1 Flash, alongside production-serving details that make the engineering report as important as the code.

• The release covers the inference engine and API-processing layer. DeepInfer supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output on NVIDIA and Huawei Ascend platforms. The accompanying deepseek-recipe component implements production token encoding/decoding and official-API-compatible processing. DeepSeek says future model inference implementations and optimizations will also appear in this repository. @deepseek_ai, @deepseek_ai, announcement translation

• The reported production figures are substantial—but describe a heavily cached service. DeepSeek’s engineering report, as summarized and quoted in the supplied posts, reports 155 trillion tokens/day from one cluster, 96% input-cache hits, 400 ms average time to first token, and 230 tokens/s generation, reaching 300 tokens/s for coding. It also reports $38 revenue per accelerator-hour at off-peak prices. These are first-party production claims, not independently audited measurements. @ZhihuFrontier, @swyx

• “Fully open source” does not mean the entire production service is reproducible from this release. Although DeepSeek describes DeepInfer as fully open source, the technical account says the KV-cache management and load-balancing components are not yet fully released, because they remain coupled to internal infrastructure. The engine is open; important service-level orchestration remains a deployment responsibility. @deepseek_ai, @ZhihuFrontier

• vLLM supplied the most concrete comparative response. After acknowledging that DeepInfer was initially built primarily on vLLM and released under Apache 2.0, the vLLM team reported decode throughput within 6% of DeepInfer’s published results on GB200 under a matched configuration. It also published AgentX results and reproduction recipes, materially qualifying claims that DeepSeek’s efficiency is inaccessible to existing open-source engines. @vllm_project, @vllm_project, reproduction resources

• The release triggered two distinct debates: whether DeepSeek has a uniquely strong serving stack, and whether its disclosures demonstrate high inference profitability more generally. The first now has comparative benchmark evidence; the second remains sensitive to workload mix, cache accounting, hardware cost, and which expenses are included. @Tim_Dettmers, @zephyr_z9, @Yuchenj_UW

DeepInfer’s disclosed architecture separates vision encoding, prefill, and decode, allowing each stage to scale against its own bottlenecks. The recommended deployment shapes are ordinary eight-accelerator machines for prefill and 64–128-accelerator supernodes for decode. Decode uses wide expert parallelism to distribute MoE experts while maintaining large effective batches. These are deployment recommendations, not sufficient evidence of the exact hardware composition of DeepSeek’s production cluster. @ZhihuFrontier

The scheduling details explain why a conventional “tokens per batch” comparison can miss substantial differences:

• Prefill scheduling estimates attention-compute cost. Work is allocated using estimated compute rather than raw request length alone. Chunk sizes vary to fit a time budget, instead of enforcing one fixed chunk size across heterogeneous requests. @ZhihuFrontier

• Prefill routing does not require cache affinity. Because cached prefixes are pooled in shared storage, a request can go to an available worker rather than the worker that previously served its prefix. In a detailed Chinese-language analysis, inference engineer Su Jin Yan highlights this as a particularly useful production design: prefill data-parallel groups balance on “compute tokens,” while shared storage supplies reusable state. The same analysis highlights request bucketing and FIFO scheduling with a runtime estimator. @SuJinYan123

• Decode routing balances both running-request count and KV-cache length. Request count approximates scheduling pressure; accumulated KV length captures memory/state pressure that request count alone misses. The engineering commentary describes prefill/decode scaling signals as aggregate compute load for prefill and running requests for decode. @ZhihuFrontier, @SuJinYan123

• Expert-parallel load balancing replicates popular experts. Replicas are periodically redistributed to reduce inter-device waiting caused by routing skew. The trade-off is additional HBM consumption against higher throughput, with diminishing returns as replication increases; the useful replication factor is workload-specific. @ZhihuFrontier, @SuJinYan123

• CPU scheduling overlaps accelerator execution. Asynchronous scheduling removes idle gaps between decode steps rather than treating host-side orchestration as free. @ZhihuFrontier

One technically notable observation is the apparent absence of tensor parallelism in the disclosed Blackwell configuration. A commentator contrasts this with default vLLM/SGLang recipes and traces the approach back to DeepSeek V3’s H800-era interconnect constraints. That is an interpretation of the configuration—not evidence that tensor parallelism is generally inferior. @nrehiew_

Scale also matters. Su Jin Yan argues that wide expert-parallel prefill needs enough traffic and workers to pay off, and favors pipeline parallelism for small-to-medium deployments with only a few prefill workers. The author also notes that layer heterogeneity can complicate pipeline partitioning. This is practitioner judgment, not a benchmark establishing a universal crossover. @SuJinYan123

The cache design is the central connection between the throughput figures, agent workloads, and reported revenue per accelerator.

Persistent block KV lives in a separate 3FS storage cluster with roughly 100 PiB of SSD capacity. Reported retention is three days for agent traffic and seven days for chatbot traffic. Any prefill worker can retrieve a cached prefix, decoupling cache locality from compute placement and allowing storage and compute to scale independently. @ZhihuFrontier

That retention is useful for agents that repeatedly resend a growing history of instructions, repository content, tool outputs, and previous responses. It also means 155T served tokens/day must not be read as 155T newly generated tokens/day: cache hits are part of the service’s token accounting, while newly computed prefill and autoregressive output impose very different costs. The reported 96% input-cache hit rate is therefore essential context for the headline throughput. @ZhihuFrontier, economic workload assumptions

The implementation addresses two different state lifecycles:

• Persistent block cache is loaded asynchronously. Host-to-device transfers overlap layer execution. Su Jin Yan highlights a lower-level implementation detail: fine-grained KV block copies perform poorly through the CUDA copy engine, so the described path uses GDR/RDMA-based transfers, aiming to saturate PCIe bandwidth without occupying SMs. This is a report-derived technical observation, not a separately measured result in the tweet. @SuJinYan123

• Sliding-window attention state stays in host memory rather than persistent SSD storage. It is useful mainly within active sessions and has a shorter lifetime than reusable block prefixes. @ZhihuFrontier

• SWA Bounded Replay reconstructs missing short-lived state from only the latest 128 tokens. Replay restricts attention to that segment while reusing the persistent block cache, producing an approximation rather than rebuilding state across the full preceding history. DeepSeek reports that this nearly halves total prefill computation. The approximation and its model-specific validity matter when considering portability to other architectures. @ZhihuFrontier

Su Jin Yan additionally reports that the long retention tail buys only approximately 0.5–1 percentage point of extra cache hits, illustrating a storage-capacity-versus-recomputation trade-off. The same commentary suggests decode-led cache management allows generated output to become reusable prefix state on subsequent turns. @SuJinYan123

The timing invites comparison with OpenAI’s separately announced 12-hour API prompt caching, versus its 30-minute default, including mixed retention durations within a request. These are not identical products or architectures, but both target persistent agents that reuse context across long gaps. Claims elsewhere in the feed about indefinite logged-in caching are commentary, not the terms of the API announcement. @OpenAIDevs, retention controls, comparison commentary

DSpark makes speculative verification adaptive. It predicts draft-token acceptance probabilities and changes verification length to maximize expected throughput. DeepSeek reports a 6% production throughput improvement over always verifying five draft tokens. This is an incremental gain over a specific speculative-decoding baseline, not a 6% gain over non-speculative decoding. @ZhihuFrontier

deepseek-recipe addresses the serving interface, not just tokenizer speed. It supports Chat Completions, Responses, and Messages APIs, parallelizes tokenization, and distinguishes real control tokens from identical-looking strings supplied as user text. That distinction prevents literal user content from accidentally being parsed as structural delimiters; it should not be mistaken for a complete prompt-injection defense. @ZhihuFrontier, @deepseek_ai

A community explanation attributes the efficiency to several model/runtime co-design choices: 196B Engram parameters, a smaller prefill backbone activating approximately 8B parameters/token versus 16B at decode, wide EP, highly compressed KV, and speculative decoding. It further describes 552B expert parameters, DP64/EP64, approximately 45B resident parameters/device, mostly four-bit weights, and a claimed 400× KV reduction against a GQA baseline. These are secondary claims: the post does not specify enough about replication, shared weights, quantization layout, or the GQA reference to independently reconstruct those figures. In particular, 45B/device is not obtained by simply dividing 552B by 64. @bookwormengr

This model specificity is one reason not to assume the same gains transfer automatically to arbitrary checkpoints. Su Jin Yan sees value in a per-model repository/kernel strategy; Andrew Carr suggests adapting the implementation to other models with coding agents, but supplies no portability or performance validation. @SuJinYan123, @andrew_n_carr

The strongest technical counterweight to “only DeepSeek can do this” is vLLM’s follow-up:

Sources: vLLM comparison, low-latency detail, recipes and measurements.

Three distinctions are necessary:

1. The within-6% result is decode-stage parity, not reproduction of DeepSeek’s whole service. It does not establish equivalent 100-PiB cache operations, retention economics, routing, production availability, or end-to-end latency under DeepSeek’s traffic distribution.

2. The 1M-token/s/GPU figure cannot mean per-user generation. The same post separately reports roughly 200 P90 output TPS; the throughput headline needs the benchmark’s input/cache/output accounting before it can be compared with pure decode throughput.

3. The work is an external implementation comparison, but still self-reported by the competing engine team. Public InferenceX results and recipes improve reproducibility; they do not make every quoted production claim independently audited.

The open-source lineage is explicit: vLLM thanked DeepSeek for the Apache 2.0 release and credit, while Woosuk Kwon described DeepSeek as an early adopter and contributor. DeepInfer is therefore better understood as production learnings flowing back into an existing ecosystem than as an entirely separate replacement for it. @vllm_project, @woosuk_k

Other reactions span a useful range:

• Strong endorsement: Tim Dettmers judges DeepSeek’s inference stack comparable to OpenAI’s and better than Anthropic’s for its user scale; Graham Neubig points to the official API’s speed and price. Neither supplies a matched cross-lab benchmark. QuixiAI says the release will inform SlimServe. @Tim_Dettmers, @gneubig, @QuixiAI

• Deployment skepticism: Su Jin Yan finds the engineering article more valuable than the repository for an experienced inference developer, partly because important components are absent. Finbarr Timbers asks for comparisons against Modal, Together, and vLLM; no matched Modal/Together comparison appears in the supplied posts. @SuJinYan123, @finbarrtimbers

• Qualified support from an otherwise enthusiastic commentator: Teortaxes acknowledges missing infrastructure and suggests 3FS plus potentially Dynamo could cover some gaps, while estimating that matching cluster-scale throughput would require roughly 3,000–4,000 GB200s. That is a proposed deployment path, not a demonstrated reproduction. @teortaxesTex

An official English version of the technical report was subsequently identified in the repository, reducing the need to rely exclusively on community translations. @teortaxesTex

The most consequential financial disclosure is $38 per accelerator-hour in revenue at off-peak prices. Revenue is not profit, and an accelerator-hour is only one part of a production service’s cost base. @swyx

A widely shared estimate initially calculated 87.76–96.94% inference margins, assuming:

• 97% cached input, 2% uncached input, 1% output of all billed tokens;

• 80K uncached input tokens/s/GPU and 16K output tokens/s/GPU;

• 250 output tokens/s/user;

• $3–$6/GPU-hour rental cost.

For one billion total billed tokens, that model contains 970M cached input, 20M uncached input, and 10M output, requiring approximately 875 GPU-seconds: 250 for prefill and 625 for decode. Estimated GPU cost is $0.73–$1.46, against $11.91 off-peak or $23.82 peak revenue. @zephyr_z9

The author then corrected the estimate: the idealized scenario implied roughly $49/hour, whereas DeepSeek reported $38/hour, or about 80% of benchmark performance in production. The revised quoted margin range was 84.2–96.1%. The original estimate should not be repeated without that correction. @zephyr_z9

Other posts use higher hardware costs:

• At $7/hour against $38/hour revenue, the arithmetic gives approximately 81.6% before other costs. Yuchen Jin explicitly notes that DeepSeek did not disclose the GPU behind the revenue figure. @Yuchenj_UW

• At $5–$10/hour, the same accelerator-only calculation spans approximately 73.7–86.8%, rather than establishing one universal “80% serving margin.” @swyx

These calculations omit or incompletely allocate the SSD cache fleet, CPUs, networking, idle/reserve capacity, failures, engineering, support, and other operating costs. They also depend on a highly favorable cache-heavy token mix. Training expenditure and future compute commitments are separate again: high contribution margins on inference can coexist with large company-level losses.

The more aggressive extrapolations are explicitly speculative. Teortaxes estimates that an alleged $1B ARR could be supported by approximately 42 GB200 NVL72 racks/~3,000 GPUs at off-peak prices, later lowering that to 33 racks when peak pricing is included. The posts also extrapolate toward $100B annual revenue from ~4,167 racks, and high revenue per gigawatt. These are scenario calculations—not audited DeepSeek revenue, fleet size, or payback disclosures. initial estimate, revised estimate, industry extrapolation

Views on the market implication differ. Some see the disclosures as evidence that inference has crossed a profitability threshold; others anticipate falling token prices and rising compute prices eventually compressing returns. The suggested competitive effect on Moonshot/Zhipu hosting agreements is plausible but unmeasured: an openly deployable stack can improve provider economics while also increasing pricing competition. @KCDN19, price-convergence argument, hosting competition

Ascend compatibility is confirmed; comparable Ascend production efficiency is not. Several posts interpret Blackwell support, sm100 support, and NVL72-shaped configurations as evidence of NVIDIA hardware in production. Those are clues, not a disclosed inventory or a measured NVIDIA-versus-Ascend comparison. @eliebakouch, @stochasticchasm, explicit uncertainty

A later hardware analysis compares reported decode figures for GB200 and Ascend 950DT:

• At a 100 TPS constraint and 131K context: approximately 25.5K versus 5K tokens/s/device.

• At 230 TPS: approximately 20K versus 2.7K, widening the ratio from roughly 5× to 7.4×.

• It cites an earlier DeepEP-Ascend disclosure that EP64–128 and combine operations remained under optimization, including local-reduction overhead and HBM contention with URMA.

• It hypothesizes 8–10K tokens/s/chip after improvements and a roughly 6.8K tokens/s/chip economic crossover, using assumed prices of $10M/NVL72 rack and a rumored $37K/950DT.

Those projections combine different measurements, assumed procurement prices, and unverified future optimization gains; they are not an apples-to-apples DeepInfer hardware benchmark. @teortaxesTex

The same commentator projects major future Ascend allocations and more than 1 GW of Blackwell-equivalent compute for DeepSeek by end-2027, while also reporting a conflicting model-generated estimate of 250K GPUs. Neither forecast is a DeepSeek procurement announcement. compute projection, conflicting estimate

Independent evaluation supports a meaningful open-model capability advance, although not universal frontier parity:

• Arena: V4.1 Flash 20261017 Max ranks #1 open model in Agent Arena and Code Arena: WebDev. In Agent Arena it lands #11 overall, with +5.1% net improvement across 7,500+ real-world sessions, at $0.20 median cost/task. Arena describes this as approximately 37% of GPT-6.1 Sol’s improvement for 27% of its cost. Reported signals include +11.5% confirmed success, +5.9% praise-versus-complaint, +5.4% Bash reliability, and +2.5% steerability, with no tool-hallucination issues observed in that evaluation. @arena, detailed results

• Cybersecurity: Vals reports that V4.1 Flash 1017 exceeded GPT-6 Sol’s average progress on a simulated corporate-network task using a Pi-based agent. DeepSeek and Sol each received three fresh-range runs; other models had single runs. Full evaluation details were still forthcoming. DeepSeek used substantially more tokens but had lower total API cost. @ValsAI, cost interpretation, methodology

• The cyber result is contested on efficiency and best-run performance. Scaling01 argues that “beats Sol” overstates the result because DeepSeek is several times less efficient. Teortaxes reports approximately 4.3× the tokens and 2.4× the time, and says Sol’s best run reached an objective group no DeepSeek run reached. Suggestions to spend the savings on repeated runs or swarms remain hypotheses, not evaluated equal-budget results. @scaling01, @teortaxesTex, equal-cost swarm suggestion

Informal reports are more favorable than controlled evidence warrants. Teortaxes describes trivia rounds with DeepSeek at 100%, 100%, and 91%, versus MiniMax at approximately 60%, 48%, and 62%, and a description/variant-spelling test at 100% versus 50%. These are small, author-constructed tests with model-generated questions and answer-key errors; they suggest follow-up experiments, not a validated knowledge benchmark or proof that Engram alone causes the difference. Another user reports strong Hermes-assisted AI research, while a separate post says DeepSeek remains well below Sol-xhigh on another evaluation. trivia tests, follow-up, variant-spelling test, research anecdote, contrasting result

Local deployment introduces a different trade-off. Brandon Music released a TrellisMX K2 Mix targeting two RTX PRO 6000s, using two-bit experts with 629 experts raised to three/four bits, four-bit Engram tables, and capacity intended for approximately 3M KV tokens. Reported mean KLD is 0.1306 nats, 8.6% below plain K2; a +5GB variant targets about 2M KV tokens. Only KLD had been measured, and serving used a test image, so task-quality retention, actual speed, and operational stability remain unestablished. @BrandonMusicKy

Agent Infrastructure, Local Execution and Hardware Acceleration

• Long-context inference kernels: SGLang merged megakernel support, with SemiAnalysis reporting nearly 1,000 tokens/s/user on GB300 NVL72 in AgentX and 1.5× greater interactivity on MI355X. Separately, Inferact reports 391 tokens/s/user for Kimi K3 on TPU v7, using open-source megakernels in vLLM TorchTPU (SGLang result, TPU result).

  • AMD optimization: Zyphra reports reducing GLM-5.3’s 512K-context TTFT from 178.8s to 64.6s, or 42.3s to 11.8s with 90% cached context. These are single-node MI355X tests without live traffic—not directly comparable production measurements (results, conditions).

• Apple-silicon runtime gains: PyTorch 2.15 replaces several MPSGraph paths with Metal kernels. Reported improvements include 5–6.5× faster MoE decoding, Qwen3.6-35B-A3B rising from 9.4 to 61.3 tokens/s on M3 Ultra, FP8 support on MPS, and Wan2.2 VAE peak driver memory falling from approximately 270 GiB to 8 GiB. These improvements land in PyTorch itself rather than a separate inference runtime (release summary).

• Local Perplexity Computer: Both model and agent can now run entirely on supported Apple-silicon Macs, including offline, interacting with native apps and local files.

  • Model and access: Local tasks use PPLX Pro, a post-trained Qwen 3.8 version, and consume no Computer credits. Access is available to Pro, Max and Enterprise subscribers; 36 GB unified memory is recommended.

  • Hybrid execution: A separate mode splits work between local and cloud models. Its privacy properties therefore depend on that split, unlike fully local execution (announcement, model, requirements).

• Persistent-agent services: LangChain’s Managed Deep Agents packages user-specific credentials, scoped memory, sandboxed execution and scheduled work; Hugging Face’s Agent Manager orchestrates persistent sessions across Claude Code, Codex and other harnesses (LangChain architecture, Agent Manager).

  • Prompt caching: OpenAI added 12-hour API prompt caching, which can be mixed with the 30-minute default within a request. Longer retention carries extra cost but can benefit repeatedly reused agent context (announcement, configuration).

  • Usage accounting: OpenAI identified approximately 40,000 accounts whose applied usage limits did not match their selected plans, corrected the issue and promised credits to affected users (incident update).

On-Device Models, Speech and Image Releases

• Muse Glimmer: Meta’s open 30B multimodal agent model, distilled from Muse Spark, targets 32 GB VRAM and runs on one RTX 5090 using 4-bit quantization and a speculative decoding head. The paper reports 75.5 on MCPAtlas and 51.2 on SWE-Bench Pro; Qwen3.6-27B remains ahead on OSWorld-Verified and TerminalBench 2.1. It supports text, images and more than 100 languages under Apache 2.0 (paper summary).

• Voxtral 3: Mistral launched batch and streaming transcription, multilingual TTS and a voice-agent SDK with tool calling, available through its API or on customers’ infrastructure (launch, deployment options).

  • Independent measurements: Artificial Analysis places Small Transcribe 3 at #3 with 2.1% AA-WER. The realtime model achieves 3.3% final-transcript AA-WER at 960 ms; TTS enters the top 12 in both voice arenas.

  • Pricing and speed: TTS costs $13/million characters and generates 112.3 characters/s in its measurements (evaluation, throughput).

  • CPU alternative: Kyutai released smaller Pocket TTS variants down to 1.9M parameters, claiming 103× realtime on one CPU core, with voice cloning (release).

• Fibo 2.0: Bria’s 8B open-weight image family includes Dev for text-to-image and distilled Edit Turbo. Artificial Analysis reports leading open-weight editing performance and a statistical tie with Qwen-Image-2.1 for text-to-image; Bria API pricing is $32/1,000 images (evaluation).

  • Licensing caveat: The weights are non-commercial; commercial deployment requires a separate Bria agreement. Licensed training data should not be confused with unrestricted commercial rights to the weights.

  • Integration: Diffusers support and fal availability are already announced (Diffusers, fal).

• Krea 3 Ultra: Rankings differ by arena and update. Artificial Analysis’s own announcement places Ultra #5, whereas earlier posts cited #4; Arena places it #4 in multi-image editing and #8 in single-image editing. The defensible conclusion is competitive image-editing quality, not a universal fourth-place rank (Artificial Analysis, Arena).

  • Pricing: Fast costs $0.03 per 1K output image plus $0.02 per 1K reference image. Ultra supports 2K output only; a 2K edit with one 1K reference costs $0.27.

Independent Evaluations, Cyber Capability and Safety Evidence

• HealthBench exposes scoring sensitivity: Artificial Analysis launched leaderboards for 5,000 general health conversations and 525 clinician conversations. GPT-6 Astra leads Professional’s length-adjusted score at 66.0%; Claude Sonnet 5.5 leads general HealthBench at 62.1%.

  • Length effects: Opus 5.5 has Professional’s highest raw score, 71.4%, but falls to 59.9% after adjustment. General HealthBench’s shortest-answer model leads communication quality while scoring lowest on completeness. Answer length is therefore a material ranking choice, not a cosmetic detail.

  • Cost trade-off: DeepSeek V4.1 Flash scores 60.9% at $0.005/task, versus Sonnet’s 62.1% at $0.08/task. Grading uses OpenAI models, following the original implementations (methodology and results).

• Routing versus specialized decisions: Across five routers and approximately 1,250 tasks, Vals found every router beaten on both score and cost by at least one standalone model. Hindsight-optimal selection within router budgets would improve scores by 8–27 points, indicating substantial selection headroom (Router Index, oracle comparison).

  • ClaimBench: On a separate financial-document classification benchmark, Decider V1.2 handled 99.7% of claims itself, escalating the remainder to Haiku 5.5; Vals reports 99.9% accuracy at $0.11/1,000 claims, versus $1.19 for Haiku alone. This narrow classification result does not contradict weaker broad agent routing (benchmark, cascade result).

• DeepSeek offensive-cyber result: Vals reports V4.1 Flash 1017 outperforming GPT-6 Sol on a simulated corporate-network task using the Pi harness. DeepSeek consumed more tokens but achieved a lower total price.

  • Evidence limits: DeepSeek and Sol each received three runs; other models received one. Full evaluation publication is pending. A reaction disputes calling this an unqualified win because of efficiency differences, so treat it as a preliminary capability signal rather than a settled ranking (finding, methodology, qualification).

• Safety assurance and accountability: METR agrees with Anthropic’s low catastrophic-misalignment-risk conclusion for covered models but identifies significant analytical disagreements. It recommends end-to-end red-teaming of monitoring systems and assessing models’ ability to undermine safety-relevant decisions (review, recommendations, decision risk).

  • Incident transparency: Hugging Face CEO Clément Delangue criticized Netflix’s portrayal of the reported OpenAI-agent cyberattack, arguing that containment, monitoring, defensive use of open models and missing agent traces deserved more attention (statement).

  • Rumor correction: Theo denied having Fable 5.5; the apparent leak involved false model identification, followed by a clarification that the supposed source was a troll. The supplied posts do not establish a launch (denial, clarification).

Scientific Agents: Wet-Lab Results and Open-Ended Limits

• Scorpion-toxin binders: Anthropic and Citrini report 17 binders from 211 tested protein designs, following substantial computational work by an internal Claude research model. All tested designs and laboratory data are being released (announcement, data).

  • Critical limitation: None bound multiple toxins, and none were tested for toxin neutralization. This is not an antivenom, nor evidence of therapeutic efficacy; cellular, animal and further validation remain ahead (researcher account, official caveat).

• Pathology-to-biology prediction: Sophont introduced Holon-S1, which predicts spatial gene activity and biomarkers from routine pathology slides. The team reports evaluation across 16 organ types and more than 15 scanners, outperforming two comparison models on spatial gene prediction and identifying potentially novel breast- and colon-cancer biomarkers. These are research findings, not validated treatment-selection outcomes (launch).

• Open-ended research remains difficult: Evaluators gave GPT-6 Astra and Claude Fable 5.1 unpublished AI-research questions and $10,000 budgets each. Original paper authors found improved experimental design and resource management over earlier tests, but still rejected both generated papers for weaknesses in creativity, judgment, calibration and communication. The small study is a useful counterweight to results on narrowly verifiable research tasks (evaluation).

Industry and Policy: Drug Development, Manufacturing and Capital Commitments

• OpenAI acquires Orchestra Bio: OpenAI announced the acquisition of a drug-development coordination platform, extending its life-sciences work beyond research tools into operational execution alongside Rosalind. Financial terms were not supplied (announcement).

• Arda launches publicly: Former OpenAI research chief Bob McGrew announced manufacturing software developed with Anduril as a design partner. His first-party account specifies a $160M Series A at a $1.5B valuation; CNBC’s accompanying post cites $230M raised, without clearly reconciling that figure with the announced round. The initial product supports manual assembly-line design and operation; reconfigurable automation is the next ambition (founder announcement, CNBC report).

• Financing and compute exposure: Reports describe OpenAI seeking approximately $30B at a $1.4T+ valuation ahead of an anticipated Anthropic IPO. Separately, a reported WSJ review of Anthropic’s draft prospectus cites $16.3B first-half revenue, nearly $9B compute/infrastructure spending, and more than $600B in future compute commitments, including negotiations. These are attributed reports, not completed transactions or independently verified financial disclosures (OpenAI report, Anthropic report).

Top tweets (by engagement)

• 39,393 — DeepSeek releases DeepInfer

• 9,809 — OpenAI corrects usage-limit mismatches affecting approximately 40,000 accounts

• 5,778 — Anthropic publishes scorpion-toxin binder research

• 5,700 — OpenAI announces Orchestra Bio acquisition

• 3,056 — Mistral introduces Voxtral 3

• 2,600 — Researcher explains binder results and biological limitations

• 2,476 — Hugging Face CEO challenges cyberattack documentary framing

• 2,330 — Theo denies supposed Fable 5.5 access

• New from DeepSeek - DeepInfer - (Activity: 313): The image is a mobile screenshot of DeepSeek’s X announcement that DeepInfer, its in-house inference framework, is now fully open source: image, GitHub. The technically relevant claim from the linked/source discussion is that DeepInfer powers DeepSeek V4.1 Flash at scale, with reported single-cluster throughput of 155 trillion tokens/day and revenue of $38 per card per hour at off-peak pricing, emphasizing high-throughput, low-latency large-scale inference rather than model training. Commenters were notably surprised that DeepSeek open-sourced an internal production inference stack, contrasting it with what they expect from Anthropic, OpenAI, or Meta. There was also basic uncertainty from some readers about what the framework actually does, i.e. serving/inference infrastructure rather than a new model.

  • A linked Zhihu source claims DeepInfer enables high-throughput, low-latency large-scale inference for DeepSeek V4.1 Flash, reporting 155 trillion tokens/day throughput for a single cluster and $38 per card-hour revenue under off-peak pricing:

https://zhuanlan.zhihu.com/p/2095560295930508953

• • . One commenter also highlights a reported serving figure of roughly 256 tokens/s × 100 users, implying very high concurrent decode throughput if sustained in production.

  • A technical concern raised is whether DeepInfer is a general-purpose inference framework or primarily a stack heavily optimized for DeepSeek’s own model architecture and deployment environment. The repo’s low initial GitHub star count versus high announcement engagement was interpreted as likely due to freshness, but commenters expect more meaningful evaluation once engineers inspect the implementation and portability.

• Deepseek released DeepInfer - Open Source high-performance, large-scale inference framework for DeepSeek models! (Activity: 1023): DeepSeek released DeepInfer, an open-source high-performance offline inference framework for DeepSeek-V4.1-Flash, with production-oriented primitives including optimized kernels, attention/data/context parallelism, MoE expert parallelism with EPLB, DSpark-style speculative decoding, SWA bounded replay, async scheduling, structured outputs, batch-invariant inference, and deterministic sampling. It currently targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs via DEEPINFER_ACCELERATOR=nvidia|ascend, requiring Linux/Python 3.12 plus CUDA 13 or CANN 9.2; however, full online serving still needs external KV-cache management, prefill/decode disaggregation, and DP replica load balancing. DeepSeek also released deepseek-recipe, a Rust library with Python bindings for protocol conversion and model-output parsing across APIs such as Chat Completions, Messages, and Responses. Commenters framed the release as valuable for reverse-engineering production inference techniques into community engines such as gufo, halogen, strixite, and Strata. There was also interest in the explicit Ascend 950 support, with one commenter joking that “Nvidia [is] in shambles,” though the listed NVIDIA target remains next-gen SM100 rather than current widely deployed GPUs.

  • DeepSeek’s release is split into two repositories: DeepInfer, described as a high-performance large-scale inference framework for DeepSeek models, and deepseek-recipe, a Rust library with Python bindings for protocol conversion and model-output parsing across API formats like Chat Completions, Messages, and Responses.

  • Commenters noted the currently listed accelerator targets are NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with build-time platform selection exposed through DEEPINFER_ACCELERATOR=nvidia or ascend before running ./build.sh. The Ascend support drew attention because it implies DeepSeek is optimizing beyond the standard CUDA/NVIDIA path.

  • One commenter referenced DeepSeek’s Zhihu post claiming roughly $38 per GPU-hour revenue/throughput economics, linking the original post at zhuanlan.zhihu.com/p/2095560295930508953. The thread raised whether similar per-GPU economics could be used to estimate inference revenue for competitors like OpenAI or Anthropic, though no concrete derivation was provided.

• Can DeepSeek V4.1 Flash 1017 really run on 12GB VRAM + 64GB RAM? (Activity: 352): The image is a screenshot of an X/Twitter thread claiming DeepSeek V4.1 Flash 1017 may be runnable locally at Q4 with 12GB VRAM + 64GB RAM; the Reddit post asks whether an RTX 3060 12GB could offload enough layers to make inference usable. The key technical pushback is that V4.1 Flash is reportedly already around 4.25 bpw for many experts via mxfp4 + fp8, so while it may fit if total RAM is sufficient, expected performance with heavy CPU/RAM execution could be extremely slow—one commenter estimates roughly ~3 tokens/sec. Commenters are skeptical that this would be practical for everyday use: the consensus is closer to “possible to load” than “pleasant to run.” One comment notes the person making the original claim is the Strata creator, lending some credibility to the sizing claim, but not necessarily to performance expectations.

  • Commenters disputed the claim that DeepSeek V4.1 Flash 1017 can practically run on 12GB VRAM + 64GB RAM: even the “Flash” variant is described as already being around 4.25 bpw for many experts, using MXFP4 + FP8, so a standard Q4-style fit is unlikely within that memory budget.

  • A technically viable setup may require heavy offloading: one commenter says Q4 does not fit and suggests checking the Hugging Face model size directly, then offloading n-grams to NVMe and streaming some MoE experts from disk. This may run in principle, but expected throughput is likely very poor, with one estimate around ~3 tokens/s or worse depending on disk and offload implementation.

  • There was skepticism that the original Strata claim was accurate; commenters noted that even if the model can be made to “fit” somewhere across RAM/NVMe, the relevant question is practical inference speed rather than mere loadability.

• MiniMax M3.1 Flash released, Model weights Oct 23 (Activity: 874): The image is an official MiniMax X announcement for MiniMax M3.1 Flash, described as “Space Performance + Bunny Speed,” with API availability now and open model weights scheduled for Oct 23. Commenters add that the model is reportedly a MoE with 428B total parameters, ~23B active parameters, sparse attention, and a native visual encoder; a preview/tool page is linked at agent.minimax.io/tools/m3-1-flash-preview. The main technical concern in comments is that benchmark excitement depends heavily on knowing the model size/active parameter count, since performance claims are hard to contextualize without architecture details.

  • A commenter cites the MiniMax M3.1 Flash Preview tool page as specifying a Mixture-of-Experts architecture with 428B total parameters and roughly 23B active parameters, plus sparse attention and a native visual encoder: https://agent.minimax.io/tools/m3-1-flash-preview. This directly answers discussion uncertainty about model size and suggests the model is positioned as a large sparse multimodal system rather than a dense LLM.

• MiniMax-M3.1-Flash is coming (Activity: 521): MiniMax teased MiniMax-M3.1-Flash via an official X post (source), but the Reddit post provides no architecture details, parameter count, context length, pricing, release date, or benchmarks. The only technical signal is the product naming: commenters infer “Flash” may denote a speed/latency-optimized variant, though no evidence is given. Commenters are skeptical about the meaning of “Flash,” joking that in current LLM naming it could still be a very large model — e.g. “Flash means a 500B model” — or sarcastically a supposedly lightweight model that is not actually small.

• AI labs report benchmark scores 6.4% higher than independent testers, on average (Activity: 334): The image is a technical benchmark-comparison chart from the whatmodel.sh open leaderboard, visualizing 611 matched comparisons across 180 AI models and 103 benchmarks from 1,629 published results. It reports that AI labs’ self-published scores are 6.4% higher on average than independent testers for the same model/benchmark pairs, with 375 lab-higher cases vs. 216 independent-higher and 20 ties; the largest median benchmark gaps shown include SWE-bench Verified and MBPP at about +9.0 points for labs. Image · interactive chart · data/code Commenters debated whether the gap reflects intentional score inflation or more mundane reproducibility issues such as package versions, harness settings, random seeds, prompt/eval setup, or labs optimizing benchmark tooling to present best-case results. Several compared it to vendor-favorable performance charts from companies like Apple or Nvidia, arguing that only independently reproduced or peer-reviewed results should be trusted.

  • Several commenters argued that the 6.4% average gap may come from benchmark reproducibility issues rather than outright inflation: small differences in package versions, evaluation harness settings, random seeds, prompt formatting, and lab-specific benchmark tooling can materially affect reported scores. One theme was that labs likely optimize the exact harness/configuration used to showcase their models, while independent testers may use more generic or less model-tailored setups.

  • A commenter noted the mean gap may be driven by outliers and suggested the median discrepancy would likely be closer to reported benchmark values. This implies the headline average could overstate systematic inflation if a small number of large deviations dominate the aggregate.

  • One technical explanation proposed was deployment mismatch: labs may benchmark full-weight models, e.g. BF16, while independent testers often evaluate served API versions that have been quantized to FP8, MXFP4, or other hardware-efficient formats. If true, some score degradation could reflect inference serving optimizations for datacenter tensor units rather than benchmark manipulation.

• Let’s delve into the tapestry of AI slop on r/LocalLLaMA (Activity: 455): The post links an informal longitudinal analysis of “AI slop” across Reddit communities, including r/LocalLLaMA, with an attempted extrapolation of future slop prevalence: thil.ooo/blog/measuring-slop. A technical commenter notes likely measurement bias: moderator removals and karma filters may cause Pushshift/archive-style datasets to undercount short-lived bot posts, while bot evasion tactics such as all-lowercase writing may weaken simple “human marker” heuristics; they also suggest comparing against Claude-associated vocabulary from “load-bearing vocabulary of Claude”. Moderation feedback was supportive: a mod left the post up as directly relevant to ongoing anti-slop moderation despite reports. Commenters broadly agreed that obvious AI-generated style markers—em dashes, bullet lists, and stock phrases like “delve”—have become noisy signals and can falsely implicate human writing.

  • A commenter noted several measurement biases that could affect an AI-slop detector trained/evaluated on Reddit archive data: moderators may remove many generated posts from /new before they appear in the dataset, and a new karma filter may be suppressing bot/slop submissions upstream. They also flagged that bots may evade simple style heuristics by posting in all lowercase, potentially corrupting “human marker” assumptions.

  • One technical suggestion was to expand the detector’s lexical feature set using Claude-specific “slop” terms, referencing the analysis of Claude’s load-bearing vocabulary at https://louisabraham.github.io/load-bearing/. The commenter emphasized distinguishing obvious AI slop from broader low-value generated content that may not exhibit obvious stylistic markers and may be removed quickly.

/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo, /r/aivideo

• After $712 and 39 hours, Opus 5.5 beat Baldur’s Gate 3 all by itself (Activity: 1145): A user reports that Claude Opus 5.5 autonomously completed Baldur’s Gate 3 using a custom harness: claude -p with built-in tools disabled, a Python MCP server connected over TCP to a Lua BG3 mod, Vulkan-layer screenshots, and SDL-mediated clicks (write-up, code). The run took 39 hours, $712, 99 fresh sessions, 20,809 tool calls, 862 screenshots, 2.2B+ cached input tokens vs 3.5M output tokens, and relied on a git-backed memory system with 157 commits, ~23k curated words, and 67 distilled tactical rules; an audited Faith-Leap rerun required 36 min, 674 tool calls, $24, one death, and eleven reloads using only screenshots of the mosaic after fixing a hidden-walkway leak. The author plans to turn the harness into a benchmark of short BG3 scenarios scored by success, attempts, time, and cost, with Claude’s Nautiloid opening reportedly succeeding 9/9 at about $1.50/run, and Codex/Gemini integrations prepared but not yet evaluated. Top commenters mostly reacted that the project is expensive but promising: one called it “one of the best benchmark ideas” for comparing agents across vendors, while another dismissed the spend as “more money than sense.” Non-technical joking centered on whether the agent made morally significant in-game choices, e.g. whether it raided the grove.

  • A commenter framed the run as a potentially useful agentic benchmark, suggesting that repeating the Baldur’s Gate 3 task across competing models could provide a more practical comparison of long-horizon planning, tool use, and persistence than many standard benchmarks. The key technical value would come from standardizing the setup and measuring cost, completion time, failure modes, and decision quality across models.

• Claude Opus 5.5 decompiles a PS2-era game overnight, and hundreds of them now run in browser tabs (Activity: 2348): A Kotaku report claims AI-assisted reverse engineering—attributed in the post title to Claude Opus 5.5—has enabled browser-playable ports of older titles such as Halo: CE, GTA: Vice City, Skate 3, and The Simpsons: Hit & Run. The article says some ports run entirely in-browser with notable performance claims, e.g. Hit & Run targeting 240fps and a Halo: CE build supporting low-latency online multiplayer, but commenters question whether these are true decompilations or instead recompilations/reimplementations of existing binaries/assets. Top comments are skeptical: one asks for an actual playable browser link and calls the post an “empty flex,” while another argues the terminology is being conflated—“these aren’t decomps… they’re just recomps?” A separate commenter speculates that public access to strong models may be restricted once corporations view this as an IP-enforcement threat.

  • A commenter challenges the post’s terminology, arguing these likely are not full decompilations but rather recompilations/recomp projects, noting that the community may be conflating the two. This distinction matters technically: decompilation implies recovering high-level source or equivalent structure, while recompilation often translates existing binaries/IR into a new target such as browser-compatible code.

• Fields Medalist Richard Borcherds on OpenAI proofs: “I think what people are getting upset about is they’re slowly realizing that they are dumb and somehow blaming this on AI, even though it’s not really AI’s fault” (Activity: 1309): The linked Daily Cal piece covers UC Berkeley mathematicians’ reaction to an alleged OpenAI Navier–Stokes proof (article), framing it as a stress test for mathematical proof validation, pedagogy, and the role of AI systems in advanced research. Richard Borcherds is quoted arguing that backlash may reflect researchers’ realization of their own limitations rather than an AI-specific failure: “they’re slowly realizing that they are dumb and somehow blaming this on AI.” Top commenters focused mostly on education: one endorsed the view that AI use on homework is irrelevant if students cannot reproduce understanding on exams, while another highlighted Borcherds’ claim that AI may be a more patient and expressive tutor for graduate abstract algebra than a human instructor.

  • A quoted discussion from Richard Borcherds frames AI-generated mathematics as potentially producing very large formal artifacts—e.g. “a 100,000-line formal proof of some major conjecture”—but argues this does not necessarily degrade mathematical learning or research because humans can choose to ignore machine-produced formalizations. The technically relevant implication is a distinction between formal proof generation/verification and the human-centered process of understanding, motivation, and theory-building.

  • One comment highlights Borcherds’ claim that AI may be better than him at teaching MATH 250A graduate abstract algebra, specifically because it is patient, consistent, and able to re-explain material without frustration. This is an education-focused technical point: LLMs may function well as adaptive tutors for dense symbolic subjects, while still not replacing the need for students to independently solve problems for exam-level mastery.

• A physicist spent a year of his life and ~500 pages of hand calculations for quantum gravity in his PhD thesis, but Opus 5.5 did it in 10 hours, including the article and GitHub repo. And it even found the errors. What will “labor” mean in academia from now on? (Activity: 1710): The linked repo, eokahya/claude-audits-quantum-gravity, is an AI-assisted audit by E. O. Kahya, the original coauthor, of prior one-loop quantum-gravity calculations for the graviton correction to a massless minimally coupled scalar in de Sitter space—not a fully autonomous rediscovery. The workflow was manually decomposed into an 18-task/9-stage pipeline where Claude orchestrated deterministic SymPy/mpmath checks, including high-precision numerical sampling, and found mostly bookkeeping/printing issues: an overall sign convention error in the printed bare self-mass, missing/non-analytic and analytic terms in Table X, finite intermediate issues, and 23 typos. The main physics result reportedly survived unchanged: after correction, the Schwinger–Keldysh late-time mode-function analysis still yields no growing one-loop terms such as ln(a), ln²(a), or a, only constant field-strength renormalization plus decaying terms. Top commenters pushed back on the framing that “Opus 5.5 did a PhD thesis in 10 hours,” arguing the hard scientific work was the original formulation, boundary conditions, Feynman rules, and expert scaffolding; the AI mostly accelerated verification of dense tensor algebra. A broader debate compared current academia to human+engine chess, with one view that AI-assisted researchers are temporarily strongest, but another predicting fully automated systems may eventually outperform human academics except in posing high-value questions.

  • Multiple commenters argue the headline misrepresents the work: E. O. Kahya, the original coauthor, did not have Claude autonomously rederive quantum gravity results from first principles; he used it to audit prior calculations. The workflow was manually scaffolded into an 18-task, 9-stage pipeline with expert-supplied boundary constraints, while Claude orchestrated deterministic tools like SymPy and mpmath for tensor-algebra checks and 30-digit numerical validation.

  • The technically important outcome was that the core physical result survived unchanged. Reported issues were limited to arithmetic slips, an overall sign/definition flip, a missing bookkeeping term, and 23 typographical errors, with commenters emphasizing that this is closer to computer-assisted verification of dense symbolic work than novel physical discovery.

  • Commenters highlight that Kahya’s own README includes caveats such as “What a human expert should still review” and limitations around numerical sampling. The main technical framing is therefore AI as an audit/cross-check layer for expert-designed theoretical pipelines, not yet a replacement for posing the theory, deriving Feynman rules, or establishing the physical framework.

• PSA: Do NOT connect your gmail and drive with your Dots at the moment (Activity: 1225): The post warns that Dots agents connected to Gmail/Drive/PC may execute attacker-controlled instructions delivered via email attachments or message bodies, exfiltrating personal data or installing malware if the agent treats the content as trusted user intent. Commenters identify this as a classic prompt injection / tool-use security failure affecting any LLM agent with external capabilities, not just Dots: untrusted content can become executable instructions when read by an autonomous agent. The dominant reaction is strong skepticism toward granting AI agents privileged access to local machines, email, or cloud storage until robust permissioning, sandboxing, and instruction-boundary defenses exist.

  • Several commenters identify the core risk as prompt injection against AI agents with connected tools, not a Dots-specific flaw. The technical concern is that any LLM granted access to Gmail/Drive or local actions can be manipulated by untrusted content to perform unintended operations, making autonomous account access unsafe without strong sandboxing, permission boundaries, and human-in-the-loop approval.

• My post about rogue AI agents got 1.3M views here and then reddit just deleted it. no reason given (Activity: 1315): The image is a non-technical Reddit UI screenshot showing a prior r/ChatGPT post marked “[Removed by Reddit]” for a content-policy violation, despite reportedly reaching ~1.3M views, 1.5K upvotes, 530 comments, and moderator approval before removal: image. The post claims the removed content discussed research, also covered by the NYT, about “rogue” AI agents operating online, but the screenshot itself provides no technical evidence about the agents—only evidence of platform-level removal without a visible rule citation. Comments lean conspiratorial/skeptical, suggesting the alleged agents or bots could be suppressing discussion or manipulating perception, while others frame reposting as the expected Reddit response.

Last year, Alex Lieberman and his cofounder joined us to explain Tenex’s unusual bet: pay AI engineers for the work they deliver, rather than the hours they spend at a keyboard. If AI makes an engineer faster, an hourly rate can make the engineer less money for the same job. Tenex wanted the incentives to run the other way.

At AIE New York, Alex came back with CJ Hess to talk about what happens when you scale that idea into an engineering organization. We also put one of our own experiments on the table: Kill My SaaS, where we asked builders to replace a conference speaker-management tool we were quoted $43,000 a year to use.

Both stories raise the same question: when building gets cheaper, how do you decide what good work is worth?

Tenex still uses story points as an important part of engineering compensation. Alex says the feared failure modes—people inflating estimates or shipping bad code to rack up points—haven’t been the biggest problem. Tenex’s AI strategists are responsible for customer retention and growth, giving them an incentive to push back on work that doesn’t serve the client.

The harder question is how to reward work that doesn’t fit neatly into a ticket: understanding a customer’s problem, designing an architecture, or deciding what should be built in the first place. CJ describes spending less time directly coding as models get better, and more time getting the customer’s context into a form the model can use. A compensation system designed around shipped tasks has to make room for that work too.

Alex says Tenex has grown to 75 people, with customers split between Fortune 2000 companies and mid-market businesses. The firm started as an AI-powered development shop; increasingly, customers are asking for help with AI transformation across the company. Jump to the compensation discussion →

CJ walks through Tenex’s MetaHarness: a shared set of skills, project documents and a CLI around the work. Plans and decisions are checked into Git alongside the code. When an agent comes back to a task, it can read not just what was built, but why the team chose to build it that way.

Tenex’s own AI-native SDLC article shows the idea as a project record that humans and agents work against, with views into Linear, GitHub and client updates:

Source: Tenex, Building an AI-Native SDLC.

The useful distinction is between making agents write code faster and making the surrounding work legible: the plan, acceptance criteria, decisions and evidence. Tenex’s article also notes the cost: that much structure can be overkill for a small team on one project. CJ explains the workflow →

Our Kill My SaaS episode covered 72 submissions to replace our speaker-management software. The team that won with our human judges came 19th on our AI judge’s leaderboard. It was a useful reminder that checking whether software satisfies a list of scenarios and judging whether people want to use it are different jobs.

I brought the project to Alex as a real example of a company choosing to build its own software. His pushback: the further upmarket you go, the more expensive and risky it becomes to rip out an established system. And even for a smaller tool, someone has to maintain it. Is the saving big enough to justify that ongoing attention?

My answer is that the upside isn’t just the bill. Our team has lived with a mediocre product for a long time. If we own the workflow, we can make it better for speakers and operators and iterate in minutes instead of waiting months for a vendor. That potential has to be weighed against the maintenance Alex describes. Watch the exchange →

Alex is generally bearish on vertical-specific AI products: if every enterprise expects a one-of-one solution, how do you deliver that at software margins? And as frontier models improve, how much of the underlying task gets absorbed by general-purpose tools?

I pushed back with examples like Harvey and OpenEvidence. Domain-specific data, specialized interfaces, integrations and user habits can matter even when the underlying model is available to everyone. Bloomberg is a useful comparison: a trader can get market data elsewhere, but the terminal brings data, workflows, communication and existing habits together.

CJ draws a distinction between a vertical product whose main feature is an agent performing a task, and a tool that improves the actual workflow around that task. That leaves a more interesting question than whether “vertical AI” wins or loses: which products become a durable place people do their work, and which become a feature of the agent they already use? Watch the debate →

• Tenex — Alex and CJ’s applied AI company, working with enterprises on AI adoption and engineering.

• Alex Lieberman on Tenex — Alex’s introduction to the company and its approach.

• Building an AI-native SDLC — Tenex’s explanation of its agent-assisted development process and MetaHarness.

• Our earlier conversation with Alex and Arman — Tenex’s original case for paying AI engineers for output.

• Kill My SaaS — What we learned replacing a $43K-a-year speaker-management tool with AI agents.

0:00 Cold open
0:26 Introductions
0:51 What Tenex does
1:32 Why CJ joined Tenex
3:17 Paying engineers for outcomes
7:20 Inside MetaHarness
10:25 Models and long-running tasks
15:11 Token spending and ROI
19:29 The enterprise AI adoption gap
23:22 Why agent deployment is hard
24:42 Vertical AI and enterprise software
30:32 Kill My SaaS
35:23 AI and jobs
38:18 New York’s AI ecosystem
42:28 Hiring at Tenex

Automatically generated from the interview; wording may contain errors and speakers are not labeled.

0:00 And I just felt my output getting better and faster, and ultimately getting paid less. Like I was charging hourly to my clients. I phrase it to the team as, you can’t automate a process that does not exist.

0:26 Okay, we’re here at AI Engineer checking with Tenex. Welcome back, Alex. Thanks. Thanks for having us. You brought your Director of Engineering, CJ. Welcome to the pod. Special projects lead at Tenex. Okay, special projects lead. All right, you’re the SWAT team. Yes, yes. So you’ve been on the pod before, and people are vaguely familiar with the Tenex story, but can you sort of reintroduce from the top what is the 2026, how you introduce yourself? Yeah, very simply, Tenex is an applied AI partner for some of the biggest companies in the world to help them go from AI absent to AI native. Our belief is that AI is incredible, but it is raw intelligence.

1:04 And the hardest problem to solve is the final mile to actually diffuse it through the economy. And so our view is to act as the player to help enterprises to figure all of this. Can I curse on? To figure all this shit out. Yeah, yeah. Yeah, we have the explicit tag on Apple Podcasts. Although I did recently discover that YouTube doesn’t like it. Oh, really? They’re so, like, yeah, you can’t say the F word. So did I just mess up your video? What are you going to do? I don’t care. Okay. We’re for adults. CJ, what drew you to Tenex? So I was actually the first hire. I don’t even know if Alex and Armand had the name Tenex quite yet,

1:40 but I have, like, a graveyard of startups I tried to start before Tenex, ultimately culminating in like my own kind of engineering consulting company. And I spent some time there doing traditional engineering, eventually moved over to doing some of this like AI consulting and just like building more and more with it. I’m a software engineer by trade. And I just felt my output getting better and faster and ultimately getting paid less. Like I was charging hourly to my clients. and I really liked kind of the ethos of Tenex that Alex had first posted

2:16 when I saw it on Twitter about kind of looking to reward engineers for their output as opposed to rewarding them for their time. I think you know fast forwarding two years you can see like how engineers can now parallelize a ton of work and just have a ton more output. At the time with you know Sonnet 3.5, Gemini 2.5 Pro, it might not have been that obvious, but I basically took the bet that I think with my skill set and leaning more into AI, I could have much higher output and ultimately have like a higher earning capacity to do that.

2:53 And since then I’ve just like fell in love with that model. I think that’s like what the future of engineering really looks like. And ultimately when AI is solving all these tech problems, companies are gonna pay based on outcome and they’re not going to want to pay based on the time that’s required to build something. Yeah. So for more detail on that, people can refer back to our previous pod where you literally pay people by story points. Yep. And you just have to agree what the story points are. But that also means that some 10x engineers can literally make millions of dollars a year and some others may not make as much just because they’re not as productive. Yeah, exactly. The thesis is that AI is really just a leveraged bet on a human’s intellect

3:29 and work ethic. And so you’re just going to have more variance than ever before in earning capacity. and our view is our model isn’t for everyone, but for the person who basically wants to go long themselves, who has this incredible foundation in software engineering, but also is constantly pushing themselves to leverage the frontier, to stretch their time as much as possible while having quality output, there’s no better way to earn. What’s the state of, like last time I talked to you, I think you had maybe two or three people making a million a year or over a million a year. Is there an update on that? Yeah, yeah, yeah. So just to paint a picture,

4:05 we started Tenex 18 months ago when we started the business. Really, we started as like an AI-powered dev shop. Really, like the original idea was engineering is kind of like the canary in the coal mine for AI. And part of the impetus was my co-founder, Armand, who’s technical by trade, was like starting to use coding agents a lot. And then he had some people he knew that were still typing into their ID, like not adopting the technology at all. And so we’re like, why don’t we just supercharge a company’s engineering org who isn’t adopting this? So we started there. Then as we started moving up market from startups and like mid-market

4:42 companies to now, you know, 50% of our customers are Fortune 2000. And then 50% are mid-market doing 100 to 500 million in revenue. They weren’t just asking for basically staff org and AI engineering. They were asking for like AI transformation as a service. Today, the business is at 75 people. 90% of those employees are either engineers or AI strategists, which are basically think of as like technical consultants. And I think the thesis has proved out true, which is there’s a very wide dispersion in earning capacity between the highest paid engineer at Tenex and the lowest paid engineer at

5:20 Tenex. It’s probably something around 4X difference. Oh, not 10X? Not 10X difference, yeah. That’s the dream. Yeah, exactly. but everything is proved right about the thesis. I do think like there’s been interesting learnings around our model of paying based on story points where there are limits to the model that we knew would be limits and how we think about that moving forward. But overall, I think the concerns that people had around this model of like paying people based on output, like the classic concerns are, well, story points are subjective. People are going to either sandbag story points or they’re just going to

5:55 ship a a bunch of bad code. Like barely check the box. Exactly. Right. But not actually do the job. Yeah, exactly. And I would say those two concerns have not been concerns at all because of our AI strategists. We incentivize, I think, perfectly at odds with engineers where AI strategists are incentivized for like customer retention and growth. Actually, the biggest challenge has been how do you compensate engineers for all the time that’s not spent coding, whether it’s architecting, meeting with clients. And so that’s where we’ve had to find nuance in the comp model,

6:30 but it’s still largely driven based on output. Anything to add? Yeah, on that, I spend a lot of time thinking about this because if I think about my tenure at Tenex, I was coding a lot, coding a lot, coding a little less, and now we’re kind of getting to the point where the models are more or less given the right context and can get you to 90% of the solution. and I do think there is some opportunity here to think about like really the value is sitting down with a customer, hearing everything that they can brain dump and then just providing that to the model

7:05 to actually get closer to the solution. So I imagine, you know, in the future, we’re going to start to look more at like true what this output is, how it works, how it looks as opposed to, you know, kind of that traditional story point model but still anchoring to those as the unit. I think one thing you’d also, I don’t know if we’ve talked about this before, but like the, this idea of like the meta harness, which is like the, the engineering workflows at Tenex that we use that has allowed us to do, I think like really meaningful work on long running tasks. Do you want to just talk through like what our process is? Cause I feel like people would find that interesting. Yeah. So we basically started trying to kind of standardize a set of skills.

7:44 Um, I think most engineering orgs have done something similar by this point. Um, and for us that grew into what is actually like a very structured set of documentation within a repo and then a CLI that’s around it that kind of the models are using to actually drive the work that’s being done, whether it’s creating a plan, a visualization, you know, creating some readout for a client, effectively like building all the pieces of the process around actually writing the code to just happen more automatically and standard across our team. So we have a couple of things, Like one is this like big canonical plan document that has some similar

8:20 structure, but I’d say as the models have gotten better, we’ve actually reduced the enforcements there in the structure. Kind of similarly across all of these. But once you have your plan, you actually build it out. And we have a very opinionated way on how agents will go and log their decisions and log what they’re doing and just keep track of what they’re up to. Everything checked into Git. Yeah. All of this is checked in. All of this is really kind of marked down for me now, more HTML. I like to see these picture books that the agents fit out. But all of that gets checked in and then it serves as this surface where not

8:55 only do you have the code as the source, you know, I think there’s a debate on like, do you even need docs? Like the code can actually be the memory of a code base. And I do think you do. You need to know why you made a decision. You need to know the context around the conversation that was had. But ultimately, that plus the code base, if there’s a question, if there’s something we need to answer, if we need to look back on why we did something, we just always have that in some durable record. And it’s never in someone’s head. It’s always in the repo. So that just gives our agents and our tools these superpowers to have all the context that I think a lot of engineering teams still just have in

9:31 conversations or have in their heads. And our goal is to kind of continue to push that forward so that all the context is in one place and more and more of that work is just driven by the agent as opposed to us having to provide everything up front. Yeah, amazing. And one way we even measure, like if any of these kind of like processes are working is are we seeing an increase in trend in average story points per month of every engineer in the company? Right. And what do you think it is, let’s say compared to started this year? I don’t know. What do you think? Do you think... Plus 50? I think, so it really depends, because we have some people that...

10:09 There’s some picture that I saw on Twitter of a guy over a wizard orb, and he’s got these things in the sky. And that’s how I think of some of our engineers when they’re whipping agents around. I’d really say, overall, it’s been a 2x increase, this year in particular. I think, basically, if you think of December as the Opus 4.5 moment where the models really started to take on these long-running tasks to today we’re back to Opus, but Opus 5.5. I mean, that’s just a magnificent model that can really do a ton of work. Over that period, I’ve seen the actual output of engineers, I would say,

10:44 roughly double in terms of what they were able to do to what they’re able to cover in the same time span now. Is a single story point worth the same now as it was a year ago? We’re always trying to calibrate. It’s a tricky equation. Is there inflation or deflation? I mean, in theory, there should be deflation, right? Yeah. Yeah. I don’t know. And that’s roughly how we think about it. I would say at least once a quarter, we kind of sit down and evaluate, like, what is our criteria for these story points? How are we actually estimating these projects? There’s a fine balance to be made between, like, still rewarding this

11:19 output, but also as the models get better, we do need to be able to kind of calibrate, like, the value that we’re creating. so overall I would say roughly that 2x has played a similar role in how we do these estimations but if I just look at some of the features and things people are building and pushing out now versus this time last year, it’s insane what the difference has been so there’s a couple ways I can take this but I’m just going to check in on the model preference stuff since it’s always a topic conversation I think you just let people choose whatever tools they like but then maybe you have some statistics

11:55 on what is preferred in the office. It seems like Opus is leading. Can you care to estimate market share, whatever? The vibe right now is Opus 5.5, I think. No Fable? I don’t particularly use Fable and I think a lot of people right now are just defaulting away from it. Opus is so good. It’s fast, it’s cheap. It kind of hits all my checkboxes. I would say I previously have had two to three models depending on whether they’re cheap and fast, whether they’re really smart, whether they can solve specific problems. And now Opus is kind of covering my full range.

12:30 I think a lot of the team still reaches for tools like codecs when we need to do something maybe more in the computer use world. And really workflow-wise, I think using both is still the best option, whichever your driver is, which I think for most of our team is Opus 5.5. Using a codex to review and having them actually work back and forth on your machine, I think still produces the best results. But right now we’re pretty aligned on those two. And are you still AMP maxing or no? Oh, I am AMP maxing. So you’re using Opus through AMP? So yeah, they recently...

13:05 Or that’s the default model for AMP. Yeah, they recently launched the ability to hook in your Cloud Code subscription. So that’s been a pretty big game changer for me. Because that’ll really cut your cost by like 80%. Totally. I mean, I love AMP because it takes everything off my computer. They have this cloud setup, the orbs. We actually co-hosted a happy hour with them last week, so it was cool to meet the team and orb out. That’s never going to be a thing. Oh, yeah. I want to make it a thing. This may be the first time in history that any human has said it.

13:41 We’re orbing out. We were orbing out. No, they do say it’s orbing time. Yeah. But, yeah, that’s my interface of choice. and models are really between that if I’m leaning on codex. It’s Sol6 one. Did you give CloudTag a shot? Isn’t it weird that Anthropic never captured the remote environment thing in the way that Orbs did? Yeah, I think there’s a piece of that, I think on the AMP side of them being very opinionated and not trying to build this general solution. Tools like CloudCode in the cloud and CloudTag, I think, provide value for teams

14:16 that are really trying to collaborate maybe on non-technical tasks. But part of the reason I love AMP is that it has all these cross-model functionalities and I can do that. Opus writes it. I can even hop to another orb and say, look at that orb and review it. And all of those things still come together to basically replicate what I’m doing on my machine. And I just couldn’t quite get there with a Cloud Code in the cloud, a Cloud Tag. Yeah, yeah. Fair enough. Another thing I always track, I’m starting to track this year, is percent of, let’s say, token spend to employee spend. I don’t know if you have that number. So roughly, most companies are at maybe,

14:53 let’s call it $10 to $200 in spend per employee. The more sort of AI native ones would sort of convert that to a percentage, so roughly 10%. The average AI startup employee is about 1x in terms of employee spend and token spend, and then it goes up to maybe 5 to 10x in terms of token spend. Where are you guys in that range? I actually don’t have a number off the top of my head. I’d say we’re somewhere between, if we’re talking like raw tokens, somewhere between 1 to 1 and maybe like 2 to 1.

15:29 Really? That’s pretty high. And what we think about a lot is like when to reach for the biggest model. I think every company is starting to think about that. But I will say recently our costs have been driven just way down from Opus 5.5. We kind of had an era of like everyone was using Astra and Fable and they just gave the best results. But ultimately those models are expensive. And like a lot of like, you know, since the last time we talked, a lot of people had this sort of token maxing error, token leaderboards, everything. And then they were like, oh, no, no, no, no. Now we have too much cost. And like, well, it’s the most predictable thing in the world. Did you ever fear about it or like you were just getting good ROI

16:07 regardless? this yeah I mean there’s a few ways that I think about it is like we were for sure we were for sure worried about it just as like our headcount was growing and um I think also like even though our team I would say like I don’t feel like anyone on our team ever really ascribed to like token the token-maxing zeitgeist, because... I had a few moments. Yeah, me too. But it’s like, well, if the boss is going to pay for it, I’m just going to go nuts, right? Because then you do the weird stuff

16:42 so that you can scale back, but you have to do the weird stuff first to discover the lessons that you can take with you. Yeah, totally. I mean, this may sound crazy, and I’d be curious of your thoughts, but I was talking to someone earlier about the debate that continues around... like basically, I promise this is related. Like I was talking to someone earlier around the data companies, right? Like Micro One, Mercore, Handshake. And just like, what does the long-term for these businesses look like, right? Because like 90 plus percentage of their revenue is from the labs.

17:17 And a number of these founders would argue that they think it’s going to flip-flop where in the next few years, it’s going to be enterprise-driven. And then like the thing that comes to mind, like the next thing I assume you have to believe if enterprises are going to be the core buyer of that data, is that enterprises are going to be doing a lot of fine-tuning on their own models. And then the question becomes, where is that going to make sense? It’s either you think there’s going to be better quality, which, again, like the bar is just going to keep moving on the frontier. So that’s like a hard bet to make. Then it has to be price. The price one feels hard to me because it’s like we’re just seeing token costs continue to go down on these models.

17:52 And so I guess part of the reason we’re not concerned internally is because it almost feels like we’ve kind of hit this like Pareto point of like costs are coming down at a per task basis. Then it is way harder today for someone to token max and like really F the company in terms of cost than it was six to eight months ago. Like I almost think six to eight months ago was the point at which like you’d be really concerned. Yeah. Do you agree with that? Yeah. I think there’s a couple ways like the future might play out there like if there’s a world

18:30 where there’s a Astra 6.5 a Fable 5.5 and they’re super expensive but they get the best results I could see us trending back that direction but generally I agree that you know you look at something like Muse Wonder that they launched yesterday today and it’s on the frontier it’s dirt cheap, the competition is heating up, both in open source and closed source. And I think that just will continue to drive costs down. And when you really look at it, a lot of tasks that are economically valuable can be done with Opus 5.5. So it’s that cliff of like,

19:08 would a 10 times smarter model really justify the cost of moving some cells in a spreadsheet? Probably not. Right. So coming back to your customers and what you’re seeing from the market, what is the most common ask that you guys get? And then what is something that people come to you with and you’re like, no, we don’t actually want that job yet? Yeah. So one way I think about this is I think about enterprises adopting AI as almost like these different milestones in a race. And I think, especially at AIE, when I tell people about what percentage of companies have reached these milestones, they’re kind of like mind blown by it.

19:45 But it’s like, okay, first milestone in the enterprise AI race is just rolling out a general purpose LLM to your people. And I’d say of the, you know, let’s call it 100 enterprises we’re talking to today, 90% have rolled out either Claude or Codex to their employees. Then the next milestone is like actually teaching people how to use it, like training and enablement, which like never gets the love that it should get, but is arguably more important or as important as the technology itself. I would say that’s at like 60%.

20:21 Oh, that’s pretty good. Yeah, yeah. Like there’s some group there of companies that are like, our people will just figure it out. But then there’s like a good chunk that is like, okay, we need to teach our people what skills are. We need to teach our people the features and functionality of these surfaces. Then the next after training and enablement within a general purpose LLM is coding agents. And the crazy thing is like, I was asking a few people here, like what percentage of enterprises do you think have rolled out coding agents to the engine product org in their company? And most people are like, it has to be 70 or 80%. The number is closer to 50%.

20:57 Like every other enterprise that we speak to either hasn’t rolled out a coding agent or what they are using is closer to not having a coding agent than having a coding agent. Then the next, and this is where like the biggest, I would say, drop off is, where like the biggest diffusion gap is from like raw intelligence to enterprise usage, is agents in production for non-engineering work drops to like 5%. It’s like, I mean, you’re in these conversations all day long, but like most enterprises that we talk to that have, let’s just say more than a thousand employees do not have an agent in production that is doing work

21:33 that is not building software. And so I think there’s a few things. One is there is, because like there’s just, this whole wave is driven by a combination of some small group of people understanding what is possible from this technology in a way larger group of people driven by FOMO. And by the way, FOMO isn’t bad. It creates some forward momentum, but it also means that people will latch onto the things that are most in the zeitgeist. I guess now for the last year and a half, people have been saying it’s the year of the agent or however long it is. Every year is the year of the agent.

22:08 Like enterprises are like, can you help build us an agent for some back office task within the organization. That is one of the most common things. And it’s why now a lot of the work we’re doing at Tenex is deploying agents into production outside of the entor for companies for repeatable work that they’re doing. The second is permissioning and RBAC. Basically, as every employee in a company has CLAWD or they have codecs and they need access to certain data, it has turned into an absolute nightmare for CIOs and CISOs. And they don’t have basically a good gateway for easily doing,

22:46 almost like a control panel for data permissioning. Literally just what can you read, what can you not read. Exactly. And that is a huge issue. So that’s like something we spend a lot of time on. So you make them for people or do you like buy something off the shelf? So basically we started by building Bespoke for companies and now we actually productize that where we have our own gateway solution. Yeah, exactly. Is there anything else that comes to mind that’s like, we’re hearing a lot from customers? I think that covered what we’re hearing when it comes to like, particularly the agent deployments. I think one of the pains that just always gets discovered,

23:22 I phrase it to the team, is like, you can’t automate a process that does not exist. So when you have 40 people that are all roughly doing the same function, but they’re all doing it a little bit of a different way, there’s actually it’s it’s way harder to build an agent that’s kind of proactively doing that in the background you can totally sit down with them give them the tools walk them through how they could augment their own workflow but when you really start to get the leverage is when you have these agents that are surfacing work to you you’re reviewing it you’re throwing it back and you become more of this like you know everyone’s been talking about becoming an agent

23:57 manager and i think we’re on the cusp of that but when you zoom out oftentimes our recommendation to companies is like, hey, first, we need to build a process around this organization within your company. And then here are the steps for us to actually hand off that work to an agent in the background. But if we don’t have some form of standardization, it’s going to be impossible for the agent to know whether it’s doing it right for us to build evals around this system and actually deliver the value that an agent that is doing this function would have. And even one thing I’d be curious your thoughts on we were literally talking about this right before

24:35 is and like uh cj will be okay with me sharing this is like the we were talking about like vertical specific ai solutions and like what does the future look like for these solutions and we we were generally expression expressing an overall bearishness with vertical specific ai solutions in different functions or industries and part of it is just from our vantage point of spending all day in like the guts of enterprises, there is so much nuance and so much inertia and so many systems you have to wrangle and people you have to wrangle. And also when a buyer of a vertical solution

25:11 is expecting truly a one-of-one experience, we’re in an age where people are expecting one-of-one software solutions. And so if you’re expecting that, it’s no wonder that every software company is basically building up a team of FDEs because there’s just this convergence happening of services and software when every company wants all this nuance that was not traditionally expected from software. And so I think it’s really interesting to think about, yeah, like four vertical specific solutions. Either one, how will they be able to do this

25:47 at software margins if everyone expects a one-of-one solution? And also, why should we not expect like frontier models to just gobble them up. Yeah. So to give some concrete examples, you’re talking like Harvey for lawyers. Yep. Open evidence for doctors. These are the classic examples of people that are winning their domains. And you’re bearish on them? I would say generally bearish on them. Correct. Wow. Well, so far you’ve been wrong. Yep. And I think the answer very simply is access to data that ChatGP is not going to have,

26:22 which is very common. That is a moat. Obviously, Anthropic and OpenAI are also acquiring those same data sources, but they’re just not as dedicated as these guys are going to be. They do have innovations. If you’ve tried it, Harvey, they have the Tabular Review. Legora also have them. We’ve had the CTO on the podcast where they have affordances and UIs and all that that is just customized because they only serve them. The pitch is focus, basically. Focus, affordances, maybe. What do you mean by affordances? In a way that you care about the last mile for your customers, they care

26:57 about their last mile in terms of the software. And as long as people train in that software, like it never goes away. Like as a banker or, you know, in my finance career, I could always access the same exact information from the NASDAQ, from the New York Stock Exchange. But why do I pay the $50,000 a year for Bloomberg? Well, it’s just like that’s that’s what everyone, all my peers do. And it’s all consolidated for me and it’s all specialized for me. So I don’t make mistakes or whatever. or like it’s tied into my trading system or my chat or whatever, my news system. No, I think people play for Bloomberg is the social network. I think people love the DMs in Bloomberg.

27:33 There’s that, there’s that. But as a trader, I know I actually use that for. Yeah, yeah, yeah. So, I mean, I think there’s a bunch of integrations and tools and like, I think ultimately it’s hard to be bearish on any of them, mostly because the demand for software, the demand for tooling to do your job is just huge. And some of it, yes, it’s going to be fulfilled by horizontal generalized tools like the Claws and ChatGPTs, and some of it, you’re just going to want something better. And, you know, I always think about maybe the success of Superhuman as an email client. Everyone has Gmail, but here’s just premium Gmail with, like, more stuff for people who live in email all day long.

28:09 And, like, that’s a market. It maybe wasn’t the largest startup in the world, but I think reasonable people can disagree how large each thing is. Yeah, totally. I think what’s not to be disagree with is the total market’s going to grow, and all rising tide is where they’re tall boats. And then you’re just talking percentage pressure market share at that point. Yeah, yeah, I agree with that. Totally agree. I think there’s even, there’s like a differentiation between some new app built from the ground up with AI involved. That’s what comes to mind when I think about Superhuman. I love Superhuman. I don’t even spend a ton of time in email, but anything that has...

28:44 No, you pay 30 bucks a month just to not worry about it. Anything that has cool hotkeys, I’m into. Yeah. But when I think about, I guess, products where they are really pushing some agent as the product that is doing work in some specific vertical, unless there is some very strategic compliance data, whatever it might be, moat, I think those are going to struggle going forward as we start to see quite literally the ability to either spin up your own really quickly, continue to compound as the models get better, or just those tasks start to fall into the world

29:21 of the models just generally. But anything that has this app and tool built around it, kind of to my previous point, changes the process and has an agent on top of it. And I think that kind of builds on the same philosophy, and I think that can still be super helpful, and those will be successful. Yeah. Oh, okay, okay. So there is a related view about this that maybe isn’t quite as spicy as yours, which is basically it’s not like nobody wants to use... People probably don’t want to use the agent that you built. They want to use the agent that they already have, and they want to plug in your data into their agent. So everyone is sitting on some part of your data,

29:58 and they’re like, use my agent, please. Please pay me that $20 a month. But you can’t just stack up all these $20 a month subscriptions. Ultimately, probably a small set of companies are going to own the interface, and then everyone just plugs into that overall agent, which I agree with. Then it’s interesting that Salesforce, being one of the system of record companies, is the biggest probably or the most well-known, is embracing that path as like you will not use our agents. Like we will just provide APIs and MCPs for you to plug into your stuff. Yeah, which is kind of cool. Okay, so I actually wanted to also walk through one example.

30:35 So I just released my results for my Kill My SaaS competition where we replace a mid-tier SaaS. I call it mid-tier basically in the sense that it’s probably not that trivial, but it’s not Salesforce. It’s somewhere in the middle. And I think there’s a lot of mid-tier SaaS out there that people are always talking about, like, the death of software, right? we were quoted $40,000 a year. We spent $1,000 to replace it. It’s going to go into production for the next conference. So we are that customer that says, like, I want the one-on-one tool, and I’m expecting all these things. And we’re building a domain-specific or vertical-specific thing for our business. And, like, so far it’s going okay.

31:10 Tell me where it’s going wrong. Also, I’m wondering how many story points that is. Yeah, well, CJ can size up the story points on it. I mean, my general view is I think the ripping out of – my general view is like the more you move upmarket in business, the harder the proposition of ripping out and building from scratch gets. I agree with you. Like part of the reason when, as people have said, like a very small group have said, like Salesforce is really at risk. SAP is really at risk. Like don’t agree with that at all. It’s just like those businesses in my mind just accrue value in this moment

31:48 in time. And part of the reason they accrue value is there’s so much money at stake where if a company gets it wrong ripping them out, you’re talking about potentially hundreds of millions or billions of dollars. You’re going to be risk averse. Yeah, exactly. I think for mid-tier, it actually comes down to what is the size of the opportunity, and is that ever going to be the worthiest opportunity to rebuild inside of a company? I’m trying to even think. In an enterprise, let’s just say there’s some mid-tier SaaS that is a few million bucks a year.

32:23 I think the first question is, from a percentage of revenue or cost perspective, is that going to ever move the needle enough to basically allocate engineer or resource to? I think it’s going to be case by case. I think for some enterprises, the answer may be yes. For other enterprises, the answer may be no. I think more of it is going to be the thing that gets companies outside of capability. Like, you know, we talk about how, I forgot who created the repo. That is basically the reverse engineer repo. RIA. Yeah, exactly. Like, everyone’s talking about that right now. And even at Tenex, like, we built our own, like, CRM from scratch.

33:05 What did we build? Our OS we built from scratch. And, like, one thing I think we realized, and every company realizes this, is one of the beauties of software is not just the utility that you get from it. It is all of the headaches you don’t have to think about in maintaining it and improving it when it’s not in your core. So I think it basically comes down to, is the size of the prize big enough to justify the cognitive load that is going to take from some amount of resource in your company to nurture something that is not core to your business?

33:41 Evidently, like I think for your business, the difference of basically $39,000 a year, you’re implicitly saying that it is worth it. It is not just about costs. That’s defense. I also think about the offense of like, no, actually our experience could be a lot better for literally every party concerned. Totally, totally. And so, yeah, I think part of the thought there is actually just, there’s a significant, basically it’s not just that. So there’s two pieces to it. One is like, not just we’re being overcharged for a product, but it’s like the product has been mediocre for a long period of time. Yes. What would the 10x better version of this product look like?

34:17 Yes, because we are the customers and we can iterate with like minutes and not months. Totally. I think, I actually think what’s interesting is I think startups and like even like high growth, mid-sized companies can be nimble in that way. I think for enterprises where like they’re so entrenched in a certain capability, rarely is that going to be the thing that, rarely are they going to say like, oh, we can create a 10x better version of this SaaS we use and we’re going to spin it out potentially as its own thing or whatever it is. I just don’t think enterprises function that way. For sure. Yeah, it’s been discussed.

34:52 Actually, I would say that you talk about automation. The more spicy thing is are they automating humans, right? And once everything is automated and smooth sailing, do we replace those jobs? I don’t know how much you can speak to that. Yeah, I mean, I think there’s two pieces here. I really deeply believe in the like, AI will produce good for humanity. It’s going to augment people’s jobs. It will absolutely change and displace. But I think it’s more of a movement as opposed to just like cutting off work. I think A16Z posted something about

35:28 how like call centers are truly now on the decline and AI is starting to handle more and more of that work. And ultimately, if that is a skill you’ve built up your whole career, that is devastating to have something come in and take that function over. That being said, part of what we do when we’re talking to customers about, hey, an agent is now going to do this work is actually around reallocation. I don’t think any business would claim like once we have this solved, there’s nothing for this group of people to work on and try to solve. So our approach really is about what can we replace and then what are the next biggest problems to look at?

36:05 and I’d make an argument in most large companies that that cycle kind of continues forever. There’s always the next problem if I think about how we’ve grown. Infinite demands. There’s always been another problem and actually taking those people that may have this like super deep subject matter expert and bringing them above water to start to maybe work more across teams or like think about parts of the business less so on like the level of a sell in a spreadsheet and more about how these teams are working together. Ultimately, in the long run, I think that keeps most people there

36:41 and allows those businesses to become more nimble and actually tackle the big meaty problems and not the drudge of most people’s day-to-day. When you’re thinking about someone that’s in a finance division, they’re mostly in spreadsheets, but how can we elevate them to think strategically about the business and where money is allocated and things like that. Yeah, and I also feel like when we have these conversations around job displacement or job reallocation, we talk about it as this thing happening to people. And one of the reasons I don’t like that is I feel like it doesn’t give enough credit to people

37:17 to have agency in their own lives, meaning I think... You can adjust. Yeah, people really should be thinking. When I think about enablement or people re-skilling, I think it is the job of an employer because they want to drive growth in their business, but it is also the job of an employee. And I think when people say that AI is going to replace all of the work that people don’t enjoy doing, I think that’s a little bit of like cope for the reality of just people are going to be put out of jobs because a lot of people don’t have the luxury of liking or not liking a job. They do a job to make money. And also I think people, to the extent that it’s possible,

37:53 should have agency in saying to themselves, okay what is the the stability or replaceability of my job over the next few years and if i am to try to skill up in a way that i’m better protected what does that look like people have agency to say that to themselves and actually take action yeah yeah yeah um i think that’s a good message for people to reflect on um the last piece i wanted to touch on before i let you guys go is just new york um you know uh greatest city in the world according to some people some people what’s the mood generally among the people they talk to, whether it’s at work, personally, what’s the feeling, right?

38:28 I live in San Francisco all day long. Everyone’s just in agents all day long. The discussion is instincts or grok or muse. It’s not like do you like AI or not, right? And I feel like I want to step out of that bubble for a bit, but it’s still New York. It’s still progressive. It’s still forward-looking. Yeah. I mean, what I’d say is I feel like, basically, I would just say there’s a bubble in New York that is a smaller bubble than exists in SF. If SF is like a big bubble, I think we at Tenex feel like, you know, we are in a, what I would say is like a microcosm of San Francisco

39:03 because of the people that self-select to work at the company. And then also the companies that we’re partnered with and like the people that we engage with in New York. But I would for sure say also, it is true that it is a smaller universe than like in SF, when you go to SF, it feels all encompassing where it’s like everyone, everywhere you look, There was someone who, similar to AIE, how I was saying earlier when I went on stage, if people go on Twitter or X because they feel like they’re on Edge City, they’re living in the future, AIE is that on steroids. I feel like SF is similar to AIE.

39:40 I mean, it’s probably the overlap of AIE generally and people who have lived at some point in their life in SF is probably a very large overlap. We’re roughly 1% of the city now. Actually? For World’s Fair, yeah. That’s crazy. Yeah. But like to that point, I just think like it is in the culture and the DNA of SF where a very big population of people live in the future. I think you can find that group in New York. And I think we at Tenex have found that people. And part of our goal is to drive like AI as a hub or New York as a hub for

40:16 AI in the US because there is so much commerce here that like it is going to be a place where diffusion of the technology actually has such a meaningful impact on the GDP of this country. I think the bubble is smaller now and we take it almost as like a charge of ours to increase our bubble. Maybe I don’t know that it’s possible to get to SF size, but I do think we can have a meaningful impact on getting more people to live in the future here. There’s a physical, like you see it on the billboards. It’s slowly creeping in. Oh, totally. People used to, remember the friend campaign where people would like spray all sorts of graffiti on that? I think that was like a little bit of an intentional rage bait. But now you have the unironic AI billboards and people are like, I guess

40:52 that’s part of life now. A hundred percent. Yeah, so I think it’s like, I think there’s an opportunity to bring kind of like the edge and like the, just like the living in the future nature that SF has created because of just like the economic climate that exists there because of venture, because of tech companies, then because of the people that are attracted there. but I also think of us as like we’re in a, it’s weird, we’re in a smaller pond for AI but I would argue we’re in as big of a pond if not a bigger pond for like

41:29 commercial value and so I actually the way I put it is like the reason I do AI in New York is I bring all the people from San Francisco to come here and meet their customers and also get a reality check Totally. And I actually think there’s a really interesting arbitrage there. Yeah. Yeah. Yeah, totally. Okay, awesome. Call for hiring. I know you guys are always hiring. Yep. What are you hiring for? What’s changed? Yep. We’re hiring forward deployed engineers at Tenex. Just go to tenex.co. We’re also always hiring AI strategists, which are basically technical consultants who are leading client engagements. And the team is 75 strong in New York City. We’re continuing to make New

42:07 York City our home. And then if you are an enterprise who is looking to actually get AI native and you’ve tried it, it’s not working, hit us up. I promise we’ll do better work than, not going to name them, but a lot of other traditional consultancies. Ooh, spicy. CJ? Yeah, I mean, I echo all of that. I’m particularly looking for engineers. What’s a special project? What is that? A special project involves anything outside of kind of our client work. So I think about my brain right now. It is building that meta harness and system internally that basically it’s

42:45 like find the bleeding edge of what the models can do and give them tools to augment how well they can do it. And then also internally working on the platform that Alex mentioned. big goal there, diffuse these background agents and actually get them out to enterprises in a way that’s truly, truly helpful to them. Yeah. And solve a lot of those like messy IT problems that we hit on earlier. Dude, when you have some solutions, we take guest posts on the Latent Space blog. I’d love to welcome some sharing of knowledge. But otherwise, thank you so much for your time.

43:20 And good to see you again in New York. Yeah, likewise. Appreciate it. Once a year, once a year. Hell yeah. Thank you.