Weaviate Podcast

Ofir Press on AliBi and Self-Ask - Weaviate Podcast #65!


Listen Later

Hey everyone! Thank you so much for watching the Weaviate Podcast! I am SUPER excited to publish my conversation with Ofir Press! Ofir has done incredible work pioneering AliBi attention and Self-Ask prompting and I learned so much from speaking with him! As always we are more than happy to answer any questions or discuss any ideas you have about the content in the podcast!

+Huge Congratulations on your Ph.D. Ofir!
AliBi Attention: https://arxiv.org/abs/2108.12409
Self-Ask Prompting: https://arxiv.org/abs/2210.03350
Ofir Pres on YouTube: https://www.youtube.com/@ofirpress
Chapters
0:00 Welcome Ofir Press
0:41 Large Context LLMs
12:38 Quadratic Complexity of Attention
19:12 AliBi Attention, Visual Demo!
24:53 Recency Bias in LLMs
28:57 RAG in Long Context LLM Training
36:27 Self-Ask Prompting
46:07 Chain-of-Thought and Self-Ask
50:47 Gorilla LLMs
58:42 New Directions for New Training Data

...more
View all episodesView all episodes
Download on the App Store

Weaviate PodcastBy Weaviate

  • 4
  • 4
  • 4
  • 4
  • 4

4

4 ratings


More shows like Weaviate Podcast

View all
Practical AI by Practical AI LLC

Practical AI

204 Listeners

AWS Podcast by Amazon Web Services

AWS Podcast

205 Listeners

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

9,968 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

517 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

130 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

92 Listeners

Interconnects by Nathan Lambert

Interconnects

9 Listeners

OpenAI Podcast by OpenAI

OpenAI Podcast

52 Listeners