This document announces OpenAI's release of gpt-realtime, an advanced speech-to-speech model, and significant updates to its Realtime API for production-ready voice agents. The new gpt-realtime model boasts improvements in natural speech, intelligence, complex instruction following, and precise function calling, including the ability to interpret non-verbal cues and switch languages mid-sentence. Key Realtime API enhancements include support for remote MCP servers, enabling additional tool access, and the integration of image input, allowing voice agents to incorporate visual context into conversations. Furthermore, the API now supports Session Initiation Protocol (SIP) for phone network connectivity and offers reusable prompts, making development more efficient. These advancements, coupled with robust safety features and reduced pricing, aim to facilitate the creation of highly capable, reliable, and cost-effective AI voice applications.

==============

Code content percentage: 5.893%

Total text length: 11199 characters

🔗 Original article: https://openai.com/index/introducing-gpt-realtime/

📋 Monday item: https://omril321.monday.com/boards/3549832241/pulses/9944973346