The provided paper introduces Gemma, a family of lightweight, open-weights language models developed by Google DeepMind. Built upon the foundational research, architecture, and training methodologies of Google's Gemini models, Gemma is available in two sizes: 2 billion and 7 billion parameters.

Key highlights of the paper include:

• State-of-the-Art Performance: Gemma models outperform similarly sized open-source alternatives (such as LLaMA 2 and Mistral) across a wide range of academic benchmarks. They demonstrate particularly strong capabilities in mathematics, coding, and reasoning tasks.
• Model Variants: Google has released both raw, pretrained checkpoints and instruction-tuned versions that have been refined for dialogue, helpfulness, and safety.
• Advanced Architecture: The models are built on a transformer decoder architecture enhanced with improvements like Multi-Query Attention, RoPE Embeddings, and GeGLU Activations. They were trained on up to 6 trillion tokens of primarily English web documents, math, and code.
• Safety and Responsible Deployment: A major focus of the paper is minimizing AI risks. The developers extensively filtered the pre-training data to remove harmful content and personal information, resulting in no observed memorization of sensitive data. Furthermore, the fine-tuned models utilize supervised fine-tuning and reinforcement learning from human feedback (RLHF) to ensure safe and reliable outputs.

Ultimately, the release of Gemma aims to provide developers and researchers with equitable access to frontier AI technology, encouraging innovation while balancing the risks of open model deployment.