Pixtral 12B is a 12-billion parameter multimodal language model trained to understand both images and text. It uses a novel vision encoder trained from scratch which allows it to process images at their native resolution and aspect ratio. Pixtral outperforms comparable open-source models on multimodal benchmarks, including a new benchmark called MM-MT-Bench. This podcast also discusses the importance of having standardised evaluation protocols for multimodal language models. The pixtral paper authors highlight the problems with existing benchmarks and metrics, proposing solutions to improve the evaluation of these models.