• Until now ChatGPT dealt with audio through a pipeline of 3 models: audio transcription, then GPT-4, then text-to-speech. GPT-4o is apparently trained on text, voice and vision so that everything is done natively. You can now interrupt it mid-sentence.
• It has GPT-4 level intelligence according to benchmarks. Somewhat better at transcription than Whisper, and considerably better at vision than previous models.
• It's also somehow been made significantly faster at inference time. Might be mainly driven by an improved tokenizer.
• It's confirmed it was the "gpt2" model found at LMSys arena these past weeks, a marketing move. It has the highest ELO as of now.
• They'll be gradually releasing it for everyone, even free users.
• No mention of safety that I've seen.
• The demos during the livestream still seemed a bit clanky in my opinion. Still far from naturally integrating in normal human conversation, which [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.