Deep dive into DeepSeek v3's efficient compute usage, innovative attention mechanisms, and MoE improvements, plus Gemini 2.0's massive context window, native code execution, and benchmark results.

Sources:
[1] https://epoch.ai/gradient-updates/how-has-deepseek-improved-the-transformer-architecture
[2] https://9to5google.com/2025/01/21/gemini-2-0-flash-thinking-experimental-jan-2025/