Neural intel Pod

The provided text introduces VCR-Bench, a novel benchmark designed to evaluate the Chain-of-Thought (CoT) reasoning capabilities of large vision-language models (LVLMs) in video understanding. Current benchmarks for video understanding often fall short by not thoroughly assessing the reasoning process, focusing mainly on final answer accuracy and struggling to differentiate between perception and reasoning abilities. To address these limitations, VCR-Bench offers a multi-dimensional evaluation framework with detailed annotations of reasoning steps across diverse video types and tasks. Evaluations using VCR-Bench reveal that current LVLMs still have significant shortcomings in video reasoning, particularly in extracting and understanding temporal-spatial information, despite a strong correlation between CoT quality and answer accuracy.

This Amazon Science blog post introduces HalluMeasure, a novel approach for automatically detecting hallucinationsin large language model outputs. The method employs a three-pronged strategy combining claim-level evaluations, chain-of-thought reasoning, and the classification of hallucination error types. By decomposing LLM responses into individual claims and comparing them to relevant context, HalluMeasure aims to improve the accuracy of identifying incorrect assertions. Furthermore, it categorizes these errors into specific linguistic types, offering valuable insights for developers to refine their models. The described research presented at EMNLP 2024 demonstrates enhanced performance in hallucination detection compared to existing solutions.

Meta AI has introduced its Llama 4 family of large language models, highlighting two new open-weight models: Llama 4 Scout and Llama 4 Maverick, along with a powerful, still-training model called Llama 4 Behemoth. These multimodal models feature innovations like Mixture-of-Experts architecture and significantly expanded context windows, outperforming previous Llama iterations and competing with or exceeding the capabilities of other leading models on various benchmarks. The release emphasizes open access, with the new models available for download and integration, aiming to foster innovation and the development of personalized AI experiences. Meta also details their approach to pre-training, post-training, and safety measures implemented in the Llama 4 series, underscoring their commitment to responsible AI development.

This research introduces DISCIPL, a novel framework where language models dictate their own reasoning process for complex tasks. By having a Planner LM generate inference programs, DISCIPL orchestrates Follower LMs to solve problems, offering a more efficient and automated approach compared to existing methods like chain-of-thought or structured inference. The framework supports various inference patterns, including constrained generation and self-correction, and achieves significant performance gains on challenging tasks requiring structured reasoning and constraint satisfaction, as demonstrated through evaluations on diverse datasets. This work presents a new way to integrate code generation and probabilistic inference, enabling improved test-time adaptability for language models.

The provided text introduces Amazon Q Developer in SageMaker Canvas, a new AI tool that allows users to build and deploy machine learning models using natural language. This tool, powered by generative AI, simplifies the ML development process by automating tasks like data preparation, model selection, and training, thus requiring no specialized expertise. The article details the functionality of this agentic system, highlighting its chatbot interface and ability to guide users through the entire ML workflow, from data input to model deployment and explanation. Additionally, the source includes various job postings for Applied Scientists across different Amazon divisions, indicating active research and development in areas like machine learning, agentic AI, and personalization within the company.

This research addresses the challenge of catastrophic forgetting in large language models during continual learning, where adapting to new tasks degrades performance on old ones. To overcome this, the authors introduce a novel approach that utilizes adaptive singular value decomposition (SVD) to identify and preserve important knowledge while allowing flexible learning of new information. Their method dynamically determines task-specific low-rank parameter subspaces for updates, ensuring these updates remain orthogonal to critical directions learned from prior tasks. This constrained full fine-tuning technique achieves state-of-the-art results on various benchmarks, demonstrating effective knowledge retention and adaptation without increasing the model's parameter count or requiring storage of past gradients.

Meta AI has introduced its Llama 4 family of large language models, highlighting two new open-weight models: Llama 4 Scout and Llama 4 Maverick, along with a powerful, still-training model called Llama 4 Behemoth. These multimodal models feature innovations like Mixture-of-Experts architecture and significantly expanded context windows, outperforming previous Llama iterations and competing with or exceeding the capabilities of other leading models on various benchmarks. The release emphasizes open access, with the new models available for download and integration, aiming to foster innovation and the development of personalized AI experiences. Meta also details their approach to pre-training, post-training, and safety measures implemented in the Llama 4 series, underscoring their commitment to responsible AI development.

This paper introduces UniOcc, a new benchmark for 3D occupancy forecasting and prediction in autonomous driving. It unifies data from real-world datasets like nuScenes and Waymo with synthetic data from CARLA and OpenCOOD, providing 2D/3D occupancy labels and voxel-level flow. UniOcc also offers novel evaluation metrics that don't rely solely on imperfect ground truth and includes a toolkit for object segmentation and tracking within the voxel space, aiming to standardize and advance research in this critical area.

This research introduces a novel method for generating counterfactual explanations for graph-based predictions.The approach utilizes a permutation equivariant graph variational autoencoder (PEGVAE) to create a meaningful latent space representation of graphs. By traversing this latent space guided by the classifier, the method identifies minimally altered graphs with a different classification. This technique addresses the challenges posed by the discrete nature and permutation invariance of graphs. Experiments on molecular datasets demonstrate the method's ability to produce robust and high-performing counterfactual explanations compared to baseline approaches. The work offers a way to understand how to change a graph to alter its predicted outcome without explicitly defining a graph distance metric.

The provided research paper explores the novel problem of continual forgetting in pre-trained vision models, where the goal is to sequentially remove specific unwanted knowledge while retaining the model's performance on other tasks. To address this, the authors introduce Group Sparse LoRA (GS-LoRA++), a parameter-efficient fine-tuning approach that uses low-rank adaptation with a group sparsity regularization to selectively modify the model's feed-forward networks. Furthermore, to handle practical scenarios like scarce forgetting data, they incorporate prototype regularization to guide the forgetting process. Extensive experiments across various vision tasks demonstrate the effectiveness and efficiency of their proposed method in achieving selective and continual knowledge erasure with minimal impact on remaining knowledge.