On The Frontier

This episode is for builders experimenting in AI. We dive deep into technical aspects of agent based workflows, data models, and interesting new fundamental research.

Madhav Singhal is a lead AI researcher at Replit focused on the frontier of what AI can enable for Replit users. In this episode, we dive into specific applications of models across code generation, testing, programming languages and more. We get fairly technical into how data maps to ML models and the types of agent based workflows that work today or may work in the future. We cover why focusing time on new model architectures may not be as relevant for founders to spend time on and how everything with new technology always maps to thinking about what solves the core problem efficiently. Enjoy this deep dive into the technical aspects of agent based workflows.

Where to Find Madhav:

* Twitter: https://x.com/madhavsinghal_

* LinkedIn: https://www.linkedin.com/in/ms337/

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(00:40) – What does Replit do?

(02:40) – Tackling Data Bottlenecks and the Shape of the Data

(05:50) – Evolution of LLM Usage & Orchestration

(07:58) – Synthetic Data Use Cases

(11:25) – Background Behind Replit’s Program Repair Agent

(14:48) – Sampling Petabytes of Data a Day for ML

(16:06) – Handling Models Across Programming Languages

(17:12) – When to Pretrain Models and Use Smaller Models

(21:53) – Narrower Problems Means Less Hallucinations

(23:05) – Real-Time Inference

(25:51) – How Madhav Allocates His Research Time

(28:00) – Choosing Mixture of Experts or Model Merging Based on Product Use Case

(30:00) – Multi-Modal Applicability for Code

(33:25) – Why New Model Architectures Don’t Necessarily Benefit Users

(34:46) – Binary Embeddings Improving Memory & Performance for Agents

(36:18) – Agent Based vs CoPilot Approaches

(38:45) – Replit’s Culture & How They Ship So Fast

(41:57) – Simple vs Complex Agent Use Cases

(43:56) – New Paradigm: IDE Specific Workflow Models

(49:20) – Wrap Up

Show Notes:

IDE Specific Agent Workflows

This episode is perfect for those interested in tactical advice from seed stage to IPO around marketing tactics and sales compensation.

Meghan Gill is SVP Sales Operations & Sales Development at MongoDB. She previously joined as employee #8 and has scaled with the org over the past 15 years. In this episode, we cover tactical learnings about early and scaled marketing & sales compensation approaches. We tackle how to incentivize sales teams and how educational content can be effective lead gen for an early go-to-market motion. Meghan also shares many anecdotes from the scaling journey at MongoDB with advice for founders along the way.

Where to Find Meghan:

* Twitter: https://x.com/meghanpgill?lang=en

* LinkedIn: https://www.linkedin.com/in/meghanpgill/

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(00:50) – Joining MongoDB as Employee 8

(04:37) – Moving from Marketing into Sales Ops

(05:45) – Setting Appropriate Sales Compensation

(07:08) – What Early MongoDB Got Right

(08:30) – Early Criticism

(10:27) – Do Things That Don’t Scale Moment

(12:08) – Early Enterprise Customers

(13:42) – Early NoSQL Messaging & Positioning

(16:55) – What Meghan Would Tell Herself Back as Employee 8

(18:30) – Managing 3 Kids While Building Startups

(19:50) – How Meghan Scaled Across Stages

(24:10) – Incentivizing Reps with New Product Rollouts

(26:20) – Focus Drives Efficiency

(29:05) – Career Advice

(31:00) – Early Marketing

(33:13) – Early Pricing

(34:18) – When to Make First RevOps Hire

(36:42) – What Are Common Mistakes with Sales Quotas

(39:45) – When to Adjust Sales Compensation

(40:57) – Most Common Advice Meghan Gives Around Marketing

(42:37) – Education Content as a Marketing Engine

(44:00) – Wrap Up

For those curious about the $25B valued business of Canva, this episode is the deep dive you’ve been waiting for.

Sandy Kory is the Founding Partner of HorizonVC, an early stage VC fund, and an angel investor in Canva. In this episode, we cover everything about the company. How it scaled to $2B ARR growing 50-60% while being cash flow positive, it’s unique product and go-to-market motion, and how AI will continue to drive the next leg of future growth. Sit back and enjoy this masterclass on Canva.

Where to Find Sandy:

* Twitter: https://x.com/sandykory

* LinkedIn: https://www.linkedin.com/in/sandykory/

* Blog: https://sandykory.substack.com

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(00:37) – Canva’s Founding Story

(06:38) – How Sandy Invested in Canva

(08:50) – Canva’s Scale & Revenue Composition

(11:30) – Canva’s Early Land Use Case

(14:27) – Core Land Use Cases Today

(17:45) – Becoming a Platform

(19:47) – M&A Strategy

(21:50) – Personas Who Use the Product

(24:06) – TAM Estimate

(28:34) – Competitive Landscape

(31:15) – Pricing Model & User-Led Motion

(36:30) – Distribution Channels

(38:14) – Evaluating Canva’s Moats

(42:10) – Australia’s Startup Ecosystem

(44:25) – Future Product & Market Evolution for Canva

(46:50) – Potential Risks Ahead

(50:10) – Text to Video Product

(52:22) – Wrap Up

For those interested in learning about how tech powers capital markets and how enterprises approach digital transformation, this is the episode for you.

Sid Bala is the CIO of Data & Analytics at BNY Mellon. He is also Head of Public Cloud at the bank. In this episode, we chat about how to approach enterprise scale transformations & architecture and taking a portfolio approach to using new and existing tech. We also cover the use cases for Generative AI and how data sharing advancements across enterprises will continue to change the landscape in the future. This is a great peek into how an enterprise leader is thinking about future use cases that enterprise software could help with.

Where to Find Sid:

* LinkedIn: https://www.linkedin.com/in/coffeehead/

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(00:40) – Sid’s Background in Enterprise Scale Tech Transformations

(02:50) – “The Art of the Possible”: Shifting to the Cloud at Large Enterprises

(08:41) – Justifying Large Scale Tech Modernization & Migrations

(12:20) – Enterprise Architecture

(13:30) – Knowing Your Own Identity

(16:25) – BNY Mellon’s Data & Analytics Platform

(19:17) – How Enterprise Can Learn From Startups

(21:00) – When to Explore New Tech Vs Improve Existing Tech

(24:17) – How Data Sharing Across Enterprises is Unlocking New Opportunities

(25:54) – Shifting from the World of Math to World of Verbal with Generative AI

(30:52) – Taking a Portfolio Approach to GenAI vs Current ML Use Cases

(33:18) – Tying Innovation to Client Needs

(34:38) – GenAI Unlocking Knowledge in Codebases & Shortening Code Review Cycles

(38:32) – How Startups Can Effectively Sell to Enterprises

(42:12) – What’s Coming Up for BNY Mellon

Show Notes:

BNY Mellon & Microsoft Strategic Alliance for Data & Analytics Platform

BNY Mellon ML Trade Settlement Models

For those interested in learning about OneTrust (last valued at $4.5B), this is the episode for you.

Kumar Gautam is a Partner at Sands Capital, a multi-strategy asset manager where he focuses on the late stage venture practice. In this episode, we do a deep dive into OneTrust, the ~$500M ARR business that is scaling rapidly in the data privacy, trust & GRC space. We cover all the aspects of the business in depth including the platform, customer use cases & personas, competitive landscape, go-to-market motion, and the future ahead.

Where to Find Kumar:

* Twitter: https://twitter.com/kumargautam11

* LinkedIn: https://www.linkedin.com/in/kumar-gautam-8736996/

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, YouTube

In this episode, we cover:

(00:33) – Founding Story of OneTrust

(03:34) – OneTrust Scale (ARR, FCF, Customer Scale)

(05:06) – Why Cookie Collection is Only the Tip of the Iceberg

(09:24) – Overview of OneTrust Platform’s Components

(12:45) – Buyer & User Personas

(14:29) – Customer Pain Points & Case Study

(19:12) – OneTrust’s TAM

(20:58) – Competitive Landscape

(23:40) – How OneTrust Prices & Generates Revenue

(25:48) – Go-To-Market Motion Behind 14k+ Customers

(27:55) – Why First Party Data is So Valuable

(30:13) – OneTrust’s Moat

(33:42) – Indirect Network Effects Benefiting the Business

(34:58) – Regulations, Data Management, & Consumer Expectations

(37:15) – Future Risks

(39:12) – Wrap Up

Where to Listen:

* Apple Podcasts, Spotify, YouTube

This episode is a great balance of the high level impact of AI & Cloud on enterprise use cases and detailed tech deep dives behind how they’re powered. Whether you’re a CIO, Founder, or IC, there’s stuff in here for everyone.

James Barney is Director of Cloud Platform Engineering at Ally Financial. He previously was a data engineer at Accenture and Lowe’s. In this episode, we cover how Ally as a digital bank builds and utilizes cloud technology to deliver a great experience to customers. We talk about migrating through modern tech transitions, centralizing data for internal teams’ efficiency, and of course the impact of AI. James discusses where he sees the impact of AI on enterprise workflows in the future and how to manage the challenges around using LLMs.

Where to Find James:

·       LinkedIn: https://www.linkedin.com/in/james-barney/

Where to Find Shomik:

·       Twitter: https://twitter.com/shomikghosh21

·       LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

·       Podcast: Apple Podcasts, Spotify, YouTube.

In this episode, we cover:

(01:50) – Migrating to Hadoop & Centralizing Data at Lowe’s

(07:31) – Ally’s Early Cloud Journey

(10:50) – How Ally Innovates Using Modern Technology

(15:46) – Managing Downtime in Systems

(18:58) – Building a Cloud Workload, Security & FinOps Internal Developer Platform

(26:04) – Utilizing GenAI for Platform Eng Use Cases

(30:02) – How Good Are LLMs at Text to SQL

(34:50) - AI’s Impact of Access to Knowledge Across the Org

(38:08) - What’s Slowing Down Potential Enterprise Adoption of GenAI

(42:27) - Wrap Up

Show Notes:

* PII Masking Module for Langchain

* Early Efforts into Generative AI

* Ally Tech Blog

How to Subscribe:

Available on Apple Podcasts, Spotify, YouTube.

The most important trend powering AI is semiconductors and that trend is not slowing down anytime soon. We need to understand what is happening at the semiconductor level to be able to project forward what we think will occur in AI. This episode is the best way to learn what’s ahead of semis.

Doug O’Laughlin is the first 2nd time guest on Software Snack Bites! He is the Founder of Fabricated Knowledge which is one of the fastest growing and highest grossing newsletters out there. The reason for that is because each of his write-ups is packed with insights that even a layperson can understand. We did this podcast right before Nvidia GTC so you’ll have to read Fabricated Knowledge to get Doug’s takes on the new announcements. In this podcast we talk about the inference specific chips (Groq & Cerebras), why HBM (high bandwidth memory) is so important to the future semi story, Nvidia’s current dominant strategy and why they are well positioned to continue winning, where ASICs and Broadcom fit in, and some other trends outside of semis Doug is watching.

If you had listened to Doug’s episode last year, you would be up 250% on Nvidia. If you had read his writing, you would be up even more. Additionally, if you read his writing on Intel, you would have had a 65% gain in < 6 months.

Of course, this is not investment advice, do your own research, but Doug’s knowledge will at least help you figure out where to start your diligence. There are multiple links below with terminology discussed and Doug’s free articles on certain topics. The best way to support him is to subscribe to Fabricated Knowledge.

Where to Find Doug:

* LinkedIn: https://www.linkedin.com/in/dougolaughlin/

* Twitter: https://twitter.com/FoolAllTheTime

* Semiconductor Publication: https://www.fabricatedknowledge.com

Where to Find Shomik:

* Twitter: https://twitter.com/shomikghosh21

* LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

* Software Snack Bites Podcast: Apple Podcasts, Spotify, Google.

In this episode, we cover:

(02:06) – Why GPUs & CUDA are So Important to LLMs

(03:58) – Why GPUs Are More Specialized Than We Think

(06:47) – The Incentive for AI Teams to Stay with Nvidia

(08:07) – Why the Market is Excited About Groq

(09:17) – SRAM not Scaling and Why That Matters

(12:44) – Why HBM is So Important to Scaling

(15:05) – Why Massive Chips with SRAM Have Constraints (Cerebras, Groq)

(17:55) – The Challenge with Inference Specific Chips (Cerebras, Groq)

(19:27) – What is Infiniband and How Nvidia is Embracing Ethernet

(22:18) – Why Google is the Best Bet to Be Able to Take on Nvidia

(24:00) – AMD’s Challenges vs Nvidia’s Tick-Tock Cadence

(25:37) – Nvidia’s Strategy Around GPU Clouds

(27:44) – Why TSMC Isn’t As Big of a Bottleneck as You Might Think

(31:14) – 14 Nanometer Chips vs Leading Edge as a Strategy

(33:00) – VLIW Challenges to Work With Various Models (Cerebras, Groq)

(36:57) – Why SemiCap Companies May Not Be Overdone in Markets

(42:35) – Broadcom’s Place in This World Given ASICs

(47:15) – What is Under Hyped in Semis Still

(49:55) – Doug’s Call On Intel Outperforming in 2023 (Backside Power Delivery)

(53:10) – Why Yield is the Challenge in Building New Chips

(53:50) – Calling Our Shot on Google in March 2024

(55:34) – Other Trends That Doug is Watching Outside of Semis

Show Notes:

Datacenter is the New Compute Unit (Why Nvidia Will Continue to Dominate)

Why HBM is the Hottest Thing in Memory (and Semis)

Detailed Nvidia GTC Analysis

Is This The Intel Inflection (June 2023)

Terminology:

Backside Power Delivery

Hybrid Bonding

High Bandwidth Memory

Static Random Access Memory

Very Long Instruction Word Architecture

Nvidia SpectrumX Ethernet Networking

We hear about hackers all the time but very rarely get a glimpse into the world of what they actually do when exploiting applications. Douglas Day is a top grossing hacker on the HackerOne platform and a Senior Security Engineer at Elastic. In this episode, we dive into all things bug bounties and ethical hacking. How does Douglas find entry points, differences between defensive and offensive security, and escalating user permissions to find deep vulnerabilities are all covered. We also talk about common attack patterns for Douglas and other hackers and why WAFs are more annoying then useful.

Where to Find Douglas:

·       Twitter: https://twitter.com/ArchAngelDDay

·       LinkedIn: https://www.linkedin.com/in/douglas-day-39baa8108/

Where to Find Shomik:

·       Twitter: https://twitter.com/shomikghosh21

·       LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

·       Podcast: Apple Podcasts, Spotify, YouTube.

In this episode, we cover:

(00:40) – Douglas’ Journey into Ethical Hacking

(05:11) – Winning Most Value Hacker at HackerOne Event

(08:03) – Bug Bounties vs Pen-testing

(11:08) – Utilizing Hacking Exploits for Defensive Security

(12:34) – Proliferation of Open Source Attacking Tools

(14:44) – Flipping from Offensive to Defensive Security

(15:27) – Working with a Team of Hackers

(18:02) – Finding a Vulnerable Entry Point to an Application

(21:16) – Utilizing User Permissions to Hack an App

(25:48) – How Does Multi-Factor Auth Help Be More Secure

(27:45) – Leveraging an Entry Point into Escalations

(29:20) – Phishing As An Attack Vector (Red Teaming vs Bug Bounties)

(31:15) – A Hacker’s Spidey Sense for Common Vulnerabilities

(34:15) – Random Number Generators for Security

(36:07) – APIs as an Attack Vector

(37:32) – Why Exposed Secrets are a Common Entry Point

(41:20) – Why Web Application Firewalls are Not That Effective for Stopping Hackers

(43:30) – How Hackers are Using LLMs in Their Attack Workflows

(45:48) – Utilizing AI Agents in Hacking

(46:30) – Why Ethical Hackers are Assets to Security Teams

(50:30) – Wrap Up

How to Subscribe:

Available on Apple Podcasts, Spotify, YouTube.

Scaling companies is incredibly difficult but at each stage the building blocks need to be set in place properly. Charles Zedlewski has been COO, GM, and a product leader at companies like Temporal, Cloudera, and SAP. In this episode, we cover what the COO & GM roles do, stories from early Cloudera days especially around figuring out what products to sell and pricing and packaging. We cover hiring Player/Coaches, nailing positioning, and talk about countless anecdotes from both Temporal and Cloudera to learn how to build and scale a great org.

Where to Find Charles:

·       Twitter: https://twitter.com/zedlewski

·       LinkedIn: https://www.linkedin.com/in/charleszedlewski/

Where to Find Shomik:

·       Twitter: https://twitter.com/shomikghosh21

·       LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

·       Newsletter: https://www.shomik.substack.com

·       Podcast: Apple Podcasts, Spotify, YouTube.

In this episode, we cover:

(01:02) – Joining Cloudera Early from SAP

(04:14) – Fighting Company Politics in Culture

(07:38) – Figuring Out Commercial Products to Sell At Cloudera

(14:58) – How Customers Weight Support & Services Differently Over Time

(16:42) – Closing Early Customers at Cloudera

(19:14) – Embracing the GM Role at Cloudera

(23:41) – Reasons Cloudera Wasn’t Able to Realize Full Potential

(31:00) – Moving from Cloudera to Temporal

(34:34) – Differences Between COO and GM Roles

(38:29) – What a COO Does in the First Few Months

(44:02) – Nailing Positioning & Product Messaging

(50:00) – When’s the Right Time to Hire a COO

(54:35) – Hiring Player / Coaches

(59:27) – Common Failure Paths of Startups

How to Subscribe:

Available on Apple Podcasts, Spotify, YouTube.

For those interested in the future of autonomous software testing & code generation, this is the episode for you.

Itamar Friedman is Co-Founder & CEO of Codium AI. He previously was Head of AI at Alibaba Israel and a founder of multiple startups before that. In this episode, we cover why Prompt Engineering is shifting into Flow Engineering. We dive deep into the areas of software verification and testing. We talk about how to bring up a world of zero bugs in software development and what agent-assisted code generation, testing, & completion looks like. We also cover how future coding agents will function with Codium’s AlphaCodium which beat Deepmind and OpenAI in recent code competitions.

Where to Find Itamar:

·       Twitter: https://twitter.com/itamar_mar

·       LinkedIn: https://www.linkedin.com/in/itamarf/

Where to Find Shomik:

·       Twitter: https://twitter.com/shomikghosh21

·       LinkedIn: https://www.linkedin.com/in/shomik-ghosh-a5a71319/

·       Newsletter: https://www.shomik.substack.com

·       Podcast: Apple Podcasts, Spotify, YouTube.

In this episode, we cover:

(00:37) – What Codium AI is Building

(01:42) – Itamar’s Background

(04:21) – Evolution of Mobile Machine Vision & Learning

(06:55) – Alibaba’s Applied AI Use Cases

(10:45) – Why Software Verification is Important to Code Generation & Completion

(15:00) – Challenges with Agent-Assisted Software Verification

(19:05) – How AlphaCodium Won Against Deepmind and OpenAI in Competition

(26:04) – Building a “System 2” Brain for Code Generation

(33:54) – Multiple Agent-Assisted Developer Workflows

(36:45) – Design in AI Systems

(40:00) – Why Humans Will Continue to Write Code

(41:25) – Future Products for CodiumAI

Show Notes:

-       What is AlphaCodium

-       From Prompt Engineering to Flow Engineering

How to Subscribe:

Available on Apple Podcasts, Spotify, YouTube.