The Data Stack Show

The Data Stack Show

Download on the App Store

All The Data Stack Show episodes

  • 161: The Intersection of Generative AI and Data Infrastructure with Chang She of LanceDB

    Highlights from this week’s conversation include:

    • Chang’s background and journey with Pandas (6:26)
    • The persisting challenges in data collection and preparation (10:37)
    • The resistance to change in using Python for data workflows (13:05)
    • AI hype and its impact (14:09)
    • The success and evolution of Pandas as a data framework (20:04)
    • The vision for a next-generation data infrastructure (26:48]
    • LanceDB's file and table format (34:35)
    • Trade-Offs in Lance Format (42:45)
    • Introducing the Vector Database (46:30)
    • The split between production and serving databases (51:14)
    • The importance of unstructured data and multimodal use cases (57:01)
    • The potential of generative AI and the balance between value and hype (1:01:34)
    • Changing expectations of interacting with information systems (1:13:53)
    • Final thoughts and takeaways (1:15:32)

    The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

    RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.


    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

    1 hr 22 min
  • 160: Closing the Gap Between Dev Teams and Data Teams with Santona Tuli of Upsolver

    Highlights from this week’s conversation include:

    • Santona’s journey from nuclear physics to data science (4:59)
    • The appeal of startups and wearing multiple hats (8:12)
    • The challenge of pseudoscience in the news (10:24)
    • Approaching data with creativity and rigor (13:22)
    • Challenges and differences in data workflows (14:39)
    • Schema Evolution and Quality Problems (27:01)
    • Real-time Data Monitoring and Anomaly Detection (30:34)
    • The importance of data as a business differentiator (35:48)
    • The SQL job creation process (46:25)
    • Different options for creating solver jobs (47:20)
    • Adding column-level expectations (50:17)
    • Discussing the differences of working with data as a scientist and in a startup (1:00:18)
    • Final thoughts and takeaways (1:04:01)

    The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

    RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.


    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

    1 hr 6 min
  • 159: What Is a Vector Database? Featuring Bob van Luijt of Weaviate

    Highlights from this week’s conversation include:

    • How music impacted Bob’s data journey (3:16)
    • Music’s relationship with creativity and innovation (11:38)
    • The genesis of Weaviate and the idea of vector databases (14:09)
    • The joy of creation (19:02)
    • OLAP Databases (22:21)
    • The progression of complexity in databases (24:31)
    • Vector database (29:23)
    • Scaling suboptimal algorithms (34:34)
    • The future of vector space representation (35:51)
    • Databases role in different industries (39:14)
    • The brute force approach to discovery (45:57)
    • Retrieval augmented generation (51:26)
    • How generative model interacts with the database (57:55)
    • Final thoughts and takeaways (1:03:20)

    The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

    RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.


    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

    1 hr 9 min
  • 158: The Orchestration Layer as the Data Platform Control Plane With Nick Schrock of Dagster Labs

    Highlights from this week’s conversation include:

    • Nick’s background and journey in data (2:28)
    • Founding Dagster Labs (7:50)
    • The evolution of data engineering (12:32)
    • Fragmentation in data infrastructure (15:04)
    • The role of orchestration in data platforms (19:53)
    • The importance of operational tools for data pipelines (25:01)
    • Lessons learned from working with GraphQL (26:19)
    • The role of the orchestrator in data engineering (34:51)
    • The boundaries between data infrastructure and product engineering (37:33)
    • Different orchestrators in the data infrastructure landscape(42:03)
    • The role of MLOps in data engineering (46:04)
    • Data Quality and Orchestration (51:04)
    • Future of Data Teams and Orchestration (54:27)
    • Final thoughts and takeaways from (58:01)

    The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

    RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.


    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

    1 hr 3 min
  • 157: From Search Engine to Answer Engine Using Grounded Generative AI, Featuring Amr Awadallah of Vectara

    Highlights from this week’s conversation include:

    • Amr’s extensive background in data (3:23)
    • The evolution of neural networks (9:21)
    • The role of supervised learning in AI (11:17)
    • Explaining Vectara (13:07)
    • Papers that laid the foundation for AI (15:02)
    • Contextualized translation and personalization (20:07)
    • Ease of use and answer-based search (25:01)
    • AI and potential liabilities (35:54)
    • Minimizing difficulties in large language models (36:43)
    • The process of extracting documents in multidimensional space (44:47)
    • Summarization process (46:33)
    • The danger of humans misusing technology (54:59)
    • Final thoughts and takeaways (57:12)

    The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.

    RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.


    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

    1 hr 4 min