Data Brew by Databricks

Benchmarking Domain Intelligence | Data Brew | Episode 45


Listen Later

In this episode, Pallavi Koppol, Research Scientist at Databricks, explores the importance of domain-specific intelligence in large language models (LLMs). She discusses how enterprises need models tailored to their unique jargon, data, and tasks rather than relying solely on general benchmarks.

Highlights include:
- Why benchmarking LLMs for domain-specific tasks is critical for enterprise AI.
- An introduction to the Databricks Intelligence Benchmarking Suite (DIBS).
- Evaluating models on real-world applications like RAG, text-to-JSON, and function calling.
- The evolving landscape of open-source vs. closed-source LLMs.
- How industry and academia can collaborate to improve AI benchmarking.

...more
View all episodesView all episodes
Download on the App Store

Data Brew by DatabricksBy Databricks

  • 5
  • 5
  • 5
  • 5
  • 5

5

18 ratings


More shows like Data Brew by Databricks

View all
The Cloudcast by Massive Studios

The Cloudcast

152 Listeners

a16z Podcast by Andreessen Horowitz

a16z Podcast

1,030 Listeners

Thoughtworks Technology Podcast by Thoughtworks

Thoughtworks Technology Podcast

40 Listeners

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch by Harry Stebbings

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

517 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

621 Listeners

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

441 Listeners

Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

Super Data Science: ML & AI Podcast with Jon Krohn

297 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

322 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

140 Listeners

DataFramed by DataCamp

DataFramed

267 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

121 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

75 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

459 Listeners

Training Data by Sequoia Capital

Training Data

42 Listeners

The Pragmatic Engineer by Gergely Orosz

The Pragmatic Engineer

53 Listeners