We were treated to technical marvels this week.

At Google, they announced Gemini Pro 1.5, with a million token context window within which it has excellent recall, using mixture of experts to get Gemini Advanced level performance (e.g. GPT-4 level) out of Gemini Pro levels of compute. This is a big deal, and I think people are sleeping on it. Also they released new small open weights models that look to be state of the art.

At OpenAI, they announced Sora, a new text-to-video model that is a large leap from the previous state of the art. I continue to be a skeptic on the mundane utility of video models relative to other AI use cases, and think they still have a long way to go, but this was both technically impressive and super cool.

Also, in both places, mistakes were made.

At OpenAI, ChatGPT [...]

---

Outline:

(01:48) Language Models Offer Mundane Utility

(05:08) Language Models Don’t Offer Mundane Utility

(10:17) Call Me Gemma Now

(11:35) Google Offerings Keep Coming and Changing Names

(13:08) GPT-4 Goes Crazy

(20:00) GPT-4 Real This Time

(22:11) Fun with Image Generation

(22:27) Deepfaketown and Botpocalypse Soon

(28:44) Selling Your Chatbot Data

(30:11) Selling Your Training Data

(32:18) They Took Our Jobs

(32:41) Get Involved

(32:55) Introducing

(35:13) In Other AI News

(36:16) Quiet Speculations

(40:27) The Quest for Sane Regulations

(43:43) The Week in Audio

(43:54) The Original Butlerian Jihad

(45:22) Rhetorical Innovation

(46:05) Public Service Announcement

(49:16) People Are Worried About AI Killing Everyone

(50:14) Other People Are Not As Worried About AI Killing Everyone

(52:57) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.