Many AI benchmarks reduce complex decision-making to simplified game mechanics, missing how real tactical reasoning juggles geometry, timing, resources, and interacting rules simultaneously. DungeonBench tackles this using Dungeons & Dragons combat, covering rich official ruleset content and testing agents on both single encounters and multi-encounter \"days\" requiring resource management across time. Evaluating frontier language models reveals they win individual fights but struggle with long-term resource budgeting and rest timing. This benchmark is valuable for developing AI agents capable of complex, rule-bound sequential planning, applicable to game AI, logistics, and multi-step strategic decision systems.

Authors: Ismayil Ismayilov, Atakan Kara, Kaan Oktay

Paper: https://arxiv.org/abs/2607.29577v1

Visual reasoning benchmarks like the Abstraction and Reasoning Corpus test whether models can infer abstract transformations from a few examples, but training typically only checks the final answer, ignoring how a model reasons through intermediate steps. TraceViT addresses this by training looped visual reasoners on step-by-step transformation chains derived from verified programs, grounding each iteration in the task\'s demonstrations. This produces stronger results on ARC-AGI benchmarks and shows that step supervision only helps when properly grounded. This approach could improve AI systems built for abstract pattern reasoning, program synthesis, and general visual problem-solving tasks.

Authors: Binnan Liu, Yechi Ma, Tian Xie, Wei Hua

Paper: https://arxiv.org/abs/2607.29586v1

Understanding social relationships from brief interactions is a subtle human skill; this paper asks whether AI models can match it. FriendBench tests whether models can tell if two people in a 20-second video clip are already friends or just meeting, using text, audio, and video across 26 models from seven companies. While top models match human accuracy, they show a different bias --- leaning toward guessing \"strangers.\" Applications include social robotics, video-conferencing analytics, and multimodal AI assistants that need to correctly infer relationship context from short behavioral cues rather than explicit words.

Authors: Jeffrey M. Girard, Jason Z. Zheng, Jacqueline R. Vertino,

Paper: https://arxiv.org/abs/2607.29602v1

As deep learning enters high-stakes domains like healthcare and finance, trustworthy explanations become essential --- yet measuring whether an explanation is actually good remains unresolved. This paper validates an EPC score that balances how sparse a feature explanation is against how much model performance it preserves. Tested across tabular, text, and image data, higher EPC scores align well with human judgments, including sentiment interpretation and visual annotation. This offers practitioners a model-agnostic, human-validated way to evaluate explainability tools before deploying AI systems where interpretability affects trust and accountability.

Authors: Christian Oliva, Luis F. Lago-Fernández

Paper: https://arxiv.org/abs/2607.29614v1

Imitation learning trains agents from expert demonstrations, powering robotics and language-model training, but standard behavior cloning suffers from compounding errors when the learner can\'t perfectly mimic the expert. This paper explains why querying the expert interactively helps: it lets the learner target the expert\'s value function rather than the harder task of replicating its exact policy. The authors introduce OVI, an algorithm exploiting this insight, and prove interaction is necessary for efficiency. This has practical implications for training robots or AI systems from limited or costly expert demonstrations.

Authors: Luca Viano, Antoine Moulin, Audrey Huang, Volkan Cevher, Philip

Paper: https://arxiv.org/abs/2607.29617v1

Neural networks used for time-series classification---like detecting mechanical faults from sensor data---are often black boxes, making it hard to trust their predictions in safety-critical settings. CENDRe improves interpretability by extracting \"concepts\" from a CNN\'s internal representations, automatically determining how many concepts exist and precisely localizing them in both time and frequency domains. Tested on bearing-fault data, it identifies exactly which frequency bands drive predictions, matching regions engineers already inspect. This has direct applications in industrial equipment monitoring, predictive maintenance, and any domain needing trustworthy explanations for CNN-based diagnostic models.

Authors: Antonia Holzapfel, Andres Felipe Posada Moreno, Sebastian

Paper: https://arxiv.org/abs/2607.29621v1

Traditional exams penalize ambition through subtractive grading, while oral exams introduce anxiety and power imbalances between examiner and student. This paper proposes a theoretical framework for \"Socratic Tests\" --- AI-mediated conversational exams that adaptively probe a student\'s understanding. By combining Dynamic Assessment, Bloom\'s and SOLO taxonomies, and graduated scaffolding, the framework maps students\' cognitive boundaries and quantifies their zone of proximal development. Potential applications include more equitable, diagnostic educational assessment tools that reward genuine understanding over rote penalty-avoidance, useful for online learning platforms and adaptive tutoring systems.

Authors: Ilya Mikhelson

Paper: https://arxiv.org/abs/2607.29624v1

As LLMs take on roles beyond code generation, acting as autonomous scientific agents, we need ways to test whether they can actually run experiments intelligently. AgentHPOBench evaluates this directly by having agents sequentially tune machine learning hyperparameters, observing results and logs before deciding on the next configuration. Testing 12 agents across 30 tasks, the benchmark exposes real limitations in iterative reasoning and log interpretation. This work matters for anyone building AI research assistants or AutoML systems, showing where current agents still fall short of matching human researchers\' experimental judgment and persistence.

Authors: Tianyu Huai, Tingshuo Fan, Xinchi Chen, Yining Zheng, Yuxin

Paper: https://arxiv.org/abs/2607.29626v1

Buildings lose efficiency and reliability when HVAC faults go undetected, but fault-detection systems are hampered by inconsistent, siloed data across equipment and vendors. FDD-ON addresses this by creating a formal ontology for variable air volume HVAC systems, defining faults, symptoms, and impacts through a shared vocabulary and explicit cause-effect relationships. This gives machines a common language for diagnostic reasoning. Applications include digital-twin building management, predictive maintenance platforms, and AI-driven facilities systems that need to interpret and share diagnostic data across different tools, ultimately supporting more scalable and interoperable smart-building infrastructure.

Authors: Yimin Chen, Brian Fricke, Bo Shen, Jamie Lian, Mingkan Zhang,

Paper: https://arxiv.org/abs/2607.29657v1

Enterprises increasingly deploy AI agents to pull structured data from documents like invoices, contracts, and forms --- but until now, no benchmark scored accuracy, completeness, grounding, and cost together. ExtractBench fills this gap with nearly 5,000 pages across 370 documents spanning 8 business domains and 67 document types. It reveals a key tradeoff: commercial vision-language models are fast but truncate long record lists, while coding agents are accurate but expensive. This benchmark is directly useful for companies choosing extraction tools for finance, legal, or logistics workflows, helping them balance accuracy against operational cost at scale.

Authors: Boyang Zhang, Adrian Lyjak, Eli Stewart, Zhaoqi Li, Simon Suo

Paper: https://arxiv.org/abs/2607.29677v1