Researchers at Peking University have developed a new benchmark called NumGLUE to evaluate numerical understanding and processing capabilities in large language models.

This benchmark addresses the need for comprehensive assessment of LLMs' ability to handle numerical data and perform mathematical reasoning. NumGLUE consists of 10 diverse tasks covering areas like arithmetic, algebra, statistics, and financial analysis. It aims to provide a standardized way to measure and compare numerical proficiency across different AI models.