Sign up to save your podcasts
Or
Share Ep 2: Renu Tewari – will Agentic AI unlock 100x data engineering value?
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Ep 2: Renu Tewari – will Agentic AI unlock 100x data engineering value?
Renu Tewari, former LinkedIn data platform leader, shares about her experience scaling the platform from 1 trillion to 35 trillion messages per day, and how Agentic AI is now reshaping the future of data engineering – and every infrastructure assumption we have.
Renu and Roy explore how hyperscaling teams address increasing platform bottlenecks, constant failures, and ongoing need for deep root-cause analysis; the explosion of open formats and Lakehouse architecture; and how active metadata is the foundation for AI agents to make autonomous in-flight decisions.
Chapters:
01:09 - Introduction and Renu's journey
03:16 - LinkedIn's data infrastructure
07:17 - Lakehouse evolution and open table formats
09:26 - Challenges of hyperscaling data platforms
12:36 - The 5-Whys approach to robust root-cause analysis
24:11 - AI agents and the future of data engineering
33:26 - Final thoughts on platform scaling strategy
Renu Tewari is former Senior Director of Data Platform at LinkedIn and currently an Entrepreneur in Residence at Bain Capital. At Linkedin, her team scaled infrastructure to 35 trillion daily messages and pioneered a lakehouse architecture combining streaming and batch analytics and robust governance.
Roy Daniel is the Co-Founder & CEO of definity – the optimization and observability platform for the Lakehouse and Spark ecosystem.
Subscribe to Data Innovators for more conversations with technical data leaders solving complex challenges across the data ecosystem.
#DataInnovators #Podcast #DataPlatforms #AI #DataGovernance #CostOptimization #definity #Linkedin
View all episodes
By definityRenu Tewari, former LinkedIn data platform leader, shares about her experience scaling the platform from 1 trillion to 35 trillion messages per day, and how Agentic AI is now reshaping the future of data engineering – and every infrastructure assumption we have.
Renu and Roy explore how hyperscaling teams address increasing platform bottlenecks, constant failures, and ongoing need for deep root-cause analysis; the explosion of open formats and Lakehouse architecture; and how active metadata is the foundation for AI agents to make autonomous in-flight decisions.
Chapters:
01:09 - Introduction and Renu's journey
03:16 - LinkedIn's data infrastructure
07:17 - Lakehouse evolution and open table formats
09:26 - Challenges of hyperscaling data platforms
12:36 - The 5-Whys approach to robust root-cause analysis
24:11 - AI agents and the future of data engineering
33:26 - Final thoughts on platform scaling strategy
Renu Tewari is former Senior Director of Data Platform at LinkedIn and currently an Entrepreneur in Residence at Bain Capital. At Linkedin, her team scaled infrastructure to 35 trillion daily messages and pioneered a lakehouse architecture combining streaming and batch analytics and robust governance.
Roy Daniel is the Co-Founder & CEO of definity – the optimization and observability platform for the Lakehouse and Spark ecosystem.
Subscribe to Data Innovators for more conversations with technical data leaders solving complex challenges across the data ecosystem.
#DataInnovators #Podcast #DataPlatforms #AI #DataGovernance #CostOptimization #definity #Linkedin