Bloom is an open-source agentic framework designed to automate the development and execution of behavioral evaluations for frontier AI models. Unlike traditional static benchmarks, it utilizes a four-stage pipeline—Understanding, Ideation, Rollout, and Judgment—to generate diverse, targeted scenarios that quantify specific traits like sycophancy, sabotage, and bias. The tool is highly configurable, allowing researchers to adjust seed configurations, reasoning effort, and interaction lengths to produce reproducible and statistically significant metrics. Validation experiments show that Bloom effectively distinguishes between baseline models and those intentionally designed to be misaligned, while its automated scoring correlates strongly with human judgment. By providing a scalable alternative to high-effort manual auditing, it enables the rapid measurement of alignment-relevant behaviors across multiple model families. Ultimately, Bloom serves as a specialized instrument for precise behavioral measurement, complementing broader exploratory auditing tools in the AI safety landscape. Source: December 19, 2025 Bloom: an open source tool for automated behavioral evaluations Isha Gupta, Kai Fronsdal, Abhay Sheshadri, Jonathan Michala, Jacqueline Tay, Rowan Wang, Samuel R. Bowman, Sara Price https://alignment.anthropic.com/2025/bloom-auto-evals/ github.com/safety-research/bloom.