Software Huddle

Today, we have Philipp Krenn on the show. He's the head of DevRel for Elastic, and we took a deep dive on all the Elasticsearch stuff like Indexes, Mappings, Shards and Replicas and how to think about performance and all that stuff.

We also discussed the Use Cases and applications where Elastic is not suitable to use. This episode is packed with fundamentals and we think you'd love it.

Timestamps

02:00 Introduction

04:13 What is Elasticsearch

05:33 Use Cases

11:25 Where not to use Elasticsearch

13:51 Index

16:44 Shards

23:29 Routing

33:57 Replicas

41:08 Bottlenecks

01:02:30 Upgrading an Elasticsearch Cluster

01:06:12 Rapid Fire

Zig is a new programming language with big ambitions: to be a better C.

Loris Cro is the VP of Community at the Zig Software Foundation, and he takes us through the ins and outs of Zig -- how was it created, what problems is it trying to solve, and where is it being used. We heard Joran Dirk Greef rave about Zig during our TigerBeetle episode, and there are a lot of passionate Zig fans out there. Zig has some really unique aspects, particularly the comptime keyword that allows for running arbitrary code at compile time.

We also talk about Loris's background and his rapid rise to lead marketing for a software foundation. Loris talks about how he got there, how Zig things about community, and how they're working to make Zig sustainable.

Today, we have Joe Reis on the show. Joe is the co author of the book, Fundamentals of Data Engineering, probably the best and most comprehensive book on data engineering you could think to read.

We talk about the culture of Data Engineering, Relationship with Data Science, the downside of chasing bleeding edge technology in approaches to Data Modeling. Joe's got lots to say, lots of opinions and is super knowledgeable.

So even if Data Engineering, Data Science isn't your thing. We think you're still going to really enjoy listening to the interview.

Our special episode is back! Join Sean, Alex & Vino in this fun conversation.

00:00 Introduction

10:08 Sora by OpenAi

16:11 Google Gemini 1.5

22:05 Mixture-of-Experts

38:02 Nvidia’s Valuation

40:19 Apple Vision Pro

49:05 Tech Layoffs

Today's episode is with Craig Kerstiens, Craig has been in the Postgres space for a long time. First at Heroku, doing Heroku Postgres. Then at Citus, doing Distributed Postgres. Now at Crunchy Data, he's Chief Product Officer there.

He's done a lot of Postgres advocacy and a lot of interesting stuff. In this episode we'll talk about the Postgres ecosystem, some of the Postgres features, some of the naysayers about Postgres, and just get Craig's thoughts on those.

Today on the show, we have the founder and CEO of Akita Software and now head of product at Postman, Dr. Professor Jean Yang. Jean has a super interesting background, a former computer science professor at Carnegie Mellon University with a focus on programming language research.

She then went on to found Akita Software, which was focused on solving hard problems around the API observability space. And last year, the company was acquired by Postman. And during the interview, we covered a lot of ground talking about Jean's academic experience, motivations for starting a company, and the problem Akita set out to work on.

01:05 Intro

06:40 Software as a Social Problem

12:10 Over engineering

20:44 Motivation

25:22 The problems

32:10 Existing methods to solve

36:21 Some other similar systems

36:21 Packet to Reconstruction

39:43 Aha moments for customers

41:33 Why sell to Postman

47:23 Would you do it again?

52:03 Rapid Fire

Today's guest is a legend in the distributed systems community. Stephan Ewan was one of the creators of Apache Flink, a stream processing engine that took off with the rise of Apache Kafka. Stephan is now working on core transactional problems by building a durable async/await system that integrates with any programming language. It's designed to help with a number of difficult problems in transactional processing, including idempotency, dual writes, distributed locks, and even simple retries and cancellation.

In this episode, we get into the details of how Restate works and what it does. We cover core use cases and how people are solving these problems today. Then, we dive into the core of the Restate engine to learn why they're building on a log-based system. Finally, we cover lessons learned from Stephan's time with Flink and what's next for Restate.

Today's guest is Bain Capital partner Rak Garg. Rak is a super smart guy that's worked as an ML researcher. Then he was in product at Atlassian before moving over to the venture capital side of the world.

In this episode, we talk about BCV Labs, an AI incubator and community for AI founders that Rak helped establish. Rak shares his thoughts on the big opportunities he sees in AI and how it's going to impact the world, both in the short and long term, and how BCV Labs is helping support AI founders bring these visions to reality.

There's a huge amount of opportunity to automate away a lot of manual tasks across industries like legal, insurance, and healthcare. But of course, there's a lot of complexity with actually bringing this technology to market.

In this episode, We spoke with the founders of WarpStream Labs, Richard Artoul and Ryan Worl.

WarpStream is a fascinating rethink of Kafka -- how could you simplify and improve the Kafka design by slightly tweaking your constraints? The result is very compelling -- a Kafka-compatible API that bypasses local disk by writing everything directly to S3. For the tradeoff of a slightly higher end-to-end latency, you can get a Kafka cluster that's much cheaper and way easier to operate.

Richie and Ryan have been working on high-scale data systems for years and were the engineers behind Husky, Datadog's custom-built database for logs and metrics. In this episode, they walk us through their experience building WarpStream. They touch on all the hard parts of building your own system (including why it's gotten easier!), as well as some of the difficult problems they had to solve for full compatibility with existing Kafka client libraries. They also touch on using FoundationDB, their thoughts on S3 Express One Zone, and whether AWS's cross-AZ network costs are a scam.

Lots of interesting thoughts here from a really sharp team.

Today, we have Cassidy Williams, CTO of Contenda. Contenda unbelievably started as a sticker distribution platform that pivoted into a product that converts podcasts and videos into various other forms of written content via AI. But in our conversation with Cassidy today, we talk about their latest pivot to a product called Brainstory, which is an AI based brainstorming application.

We talked through some of their product choices around focusing on speech as the main input mechanism, some of the technical challenges they've had to overcome, how they're using multiple AI models in the backend to make all this magic happen, and where they're seeing initial product traction. If you're a founder or thinking of starting a company, we think you'll find this conversation super interesting.

Check Out Brainstory: https://www.brainstory.ai/

Software Huddle: https://twitter.com/SoftwareHuddle

Cassidy: https://twitter.com/cassidoo

Sean: https://twitter.com/seanfalconer