Seventy3

Seventy3：借助NotebookLM的能力进行论文解读，专注人工智能、大模型、机器人算法方向，让大家跟着AI一起进步。

进群添加小助手微信：seventy3_podcast

备注：小宇宙

今天的主题是：Diffusion-Based Planning for Autonomous Driving with Flexible Guidance

Summary

The provided research paper introduces Diffusion Planner, a novel method for autonomous driving that utilizes diffusion models to achieve human-like planning in complex environments. This approach jointly models motion prediction and planning without relying on traditional rule-based refinements, addressing limitations of imitation learning. By learning the gradient of a trajectory score function and using a flexible classifier guidance mechanism, Diffusion Planner can adapt its driving behavior for safety and other preferences. Evaluations on public and newly collected datasets demonstrate that this method achieves state-of-the-art closed-loop performance with strong transferability across different driving styles.

该研究提出了 Diffusion Planner，一种新型自动驾驶规划方法，利用扩散模型（diffusion models）在复杂环境中实现类人规划。该方法联合建模运动预测与规划，无需依赖传统的基于规则的优化，从而克服了模仿学习的局限性。通过学习轨迹评分函数的梯度，并引入灵活的分类器引导机制，Diffusion Planner 能够根据安全性及其他偏好自适应调整驾驶行为。实验结果表明，该方法在公开数据集和新采集数据集上的闭环性能达到最先进水平，并展现出跨不同驾驶风格的强泛化能力。

原文链接：https://arxiv.org/abs/2501.15564

前往小宇宙评论区与主播互动

Seventy3：借助NotebookLM的能力进行论文解读，专注人工智能、大模型、机器人算法方向，让大家跟着AI一起进步。

进群添加小助手微信：seventy3_podcast

备注：小宇宙

今天的主题是：Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning

Summary

This paper examines "slow-thinking" in large language models (LLMs), where increased computation time enhances reasoning. It theoretically analyzes how errors accumulate during LLM reasoning, termed "snowball errors," linking them to the decreasing probability of correct reasoning through information theory. The research proposes that external slow-thinking methods, which involve expanding the search for solutions, primarily work by mitigating these error probabilities. A comparative analysis of various slow-thinking approaches, including Best-of-N and Monte Carlo Tree Search, suggests their effectiveness hinges more on the reliability of evaluation mechanisms and overall computational cost than the specific algorithmic framework. Ultimately, the study advocates for focusing on improving reward functions and core reasoning capabilities for better slow-thinking strategies.

本文研究了大型语言模型（LLMs）中的**“慢思考”（slow-thinking）现象，即增加计算时间如何提升推理能力**。作者从理论角度分析了 LLM 推理过程中错误的积累机制，并将其定义为**“滚雪球错误”（snowball errors），借助信息论揭示了正确推理概率随推理深度降低的趋势**。研究提出，外部慢思考方法（如扩展解空间搜索）主要通过降低错误概率来提升推理质量。对比分析了多种慢思考方法，包括Best-of-N 和 蒙特卡洛树搜索（MCTS），结果表明，其有效性更多取决于评估机制的可靠性和计算成本，而非具体算法框架。最终，研究强调优化奖励函数和核心推理能力的重要性，以改进慢思考策略。

原文链接：https://arxiv.org/abs/2501.15602

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：Demystifying Long Chain-of-Thought Reasoning in LLMs

Summary

This paper investigates how large language models (LLMs) achieve long chain-of-thought (CoT) reasoning, which involves extended, step-by-step thought processes for complex tasks. The authors explore the roles of supervised fine-tuning (SFT) and reinforcement learning (RL) in enabling this capability. Key findings highlight that while SFT on long CoT data improves performance and facilitates better RL, carefully designed reward functions are crucial for stable CoT length and enhanced reasoning. The study also examines the use of noisy web data for training and nuances in analyzing emergent reasoning behaviors during RL from base models. Ultimately, the research offers practical insights for optimizing training strategies to bolster sophisticated reasoning in LLMs.

本文探讨了大型语言模型（LLMs）如何实现长链式思维（CoT）推理，即在复杂任务中执行逐步、扩展的思考过程。作者研究了监督微调（SFT）和强化学习（RL）在提升这一能力中的作用。关键发现包括：虽然在长 CoT 数据上进行 SFT 可提高性能并优化 RL 训练，但精心设计的奖励函数对于稳定 CoT 长度和增强推理能力至关重要。此外，研究还分析了带噪声的网页数据用于训练的影响，以及在 RL 过程中基于基础模型解析涌现推理行为的细微差别。最终，该研究提供了优化训练策略的实用见解，以提升 LLMs 的高级推理能力。

原文链接：https://arxiv.org/abs/2502.03373

####🥚####彩####蛋####🥚####

本博客从24年10月2日开启，已更新半年，借助NotebookLM的能力进行论文解读，专注人工智能、大模型、机器人算法的论文，也有几十个听友。本人是一介书生，现计划建立微信群，平时可以聊聊技术，聊聊生活。希望博客可以继续更新下去！

进群添加微信小助手：seventy3_podcast

备注：小宇宙

####🥚####彩####蛋####🥚####

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills

Summary

The provided research paper introduces ASAP, a novel two-stage framework designed to bridge the gap between simulated and real-world physics for humanoid robots, enabling them to perform complex, agile movements. The first stage involves pre-training control policies in simulation using human motion data. The second stage deploys these policies in the real world to collect data and train a "delta action model" that learns to compensate for discrepancies in dynamics. This model is then integrated back into the simulator to fine-tune the control policies, allowing for more accurate and agile real-world execution. Experiments demonstrate that ASAP significantly improves the ability of humanoid robots to perform challenging tasks, outperforming existing methods in both simulated and real environments. The work highlights a promising direction for transferring skills learned in simulation to physical robots, ultimately leading to more versatile and capable humanoids.

该研究提出了 ASAP，一种创新的两阶段框架，旨在弥合仿真与现实物理之间的差距，使人形机器人能够执行复杂且灵活的运动。第一阶段在仿真环境中使用人类运动数据进行控制策略的预训练。第二阶段将这些策略部署到现实环境，采集数据并训练一个**“增量动作模型”（delta action model），用于补偿动力学差异。随后，该模型被集成回仿真环境，以微调控制策略**，从而实现更精准、灵活的现实世界执行。实验结果表明，ASAP 显著提升了人形机器人完成高难度任务的能力，无论在仿真还是现实环境中均优于现有方法。本研究为仿真训练迁移至现实机器人提供了一条有效途径，推动人形机器人向更多功能、更智能的方向发展。

原文链接：https://arxiv.org/abs/2502.01143

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：LLM-AutoDiff: Auto-Differentiate Any LLM Workflow

Summary

The provided research introduces LLM-AutoDiff, a novel framework for automating prompt engineering for complex Large Language Model workflows. This system extends gradient-based optimization to multi-step and cyclic LLM applications by treating textual inputs as trainable parameters. LLM-AutoDiff constructs a graph representing the workflow, enabling a "backward engine" LLM to generate feedback that guides iterative prompt improvements, even across functional nodes and repeated calls. The framework incorporates techniques like selective gradient computation and two-stage validation to enhance efficiency. Experimental results demonstrate that LLM-AutoDiff outperforms existing methods in accuracy and training cost across various tasks, offering a new paradigm for scaling and automating LLM deployments.

该研究提出了 LLM-AutoDiff，一个自动化提示工程（prompt engineering）的新框架，旨在优化复杂的大型语言模型（LLM）工作流。该系统通过将文本输入视为可训练参数，将基于梯度的优化方法扩展到多步和循环 LLM 应用。LLM-AutoDiff 构建了一个表示工作流的计算图，并利用**“反向引擎” LLM** 生成反馈，指导跨功能节点和重复调用的迭代提示优化。该框架还引入了选择性梯度计算和双阶段验证等技术，以提高优化效率。实验结果表明，LLM-AutoDiff 在多个任务上的准确性和训练成本方面均优于现有方法，为 LLM 部署的自动化和规模化提供了一种新范式。

原文链接：https://arxiv.org/abs/2501.16673

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：s1: Simple test-time scaling

Summary

This research explores improving language model reasoning through a technique called test-time scaling, where extra computation during inference enhances performance. The authors introduce s1K, a small, high-quality dataset of reasoning problems, and budget forcing, a method to control the model's computational effort at test time. By finetuning a language model on s1K and using budget forcing, they achieve strong results on math reasoning benchmarks, even surpassing previously reported methods while using significantly less training data. The work also analyzes different approaches to test-time scaling, finding sequential methods like budget forcing more effective than parallel ones like majority voting. Ultimately, this study demonstrates a sample-efficient way to boost reasoning through strategic test-time computation.

本研究探讨了通过测试时扩展（test-time scaling）提升语言模型推理能力的方法，即在推理阶段增加计算量以增强性能。作者提出了s1K——一个小型高质量的推理问题数据集，并引入了预算强制（budget forcing），一种在测试时控制模型计算资源的方法。通过在 s1K 上微调语言模型并应用预算强制，研究在数学推理基准上取得了优异成绩，甚至在训练数据大幅减少的情况下超越了此前的方法。此外，研究分析了不同的测试时扩展策略，发现顺序方法（如预算强制）比并行方法（如多数投票）更有效。最终，该研究证明了一种数据高效的方式，即通过策略性测试时计算来提升推理能力。

原文链接：https://arxiv.org/abs/2501.19393

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：Spurious Forgetting in Continual Learning of Language Models

Summary

This paper introduces the concept of spurious forgetting in large language models during continual learning, distinguishing it from actual knowledge loss and attributing it to the disruption of task alignment. The authors demonstrate through experiments and theoretical analysis that early training on new tasks can misalign the model, particularly in the bottom layers. To address this, they propose a Freezing strategy that keeps the initial layers unchanged, significantly improving performance in various continual learning scenarios like safety alignment and instruction tuning. Their findings highlight the importance of task alignment over pure knowledge retention and offer a practical method to mitigate performance degradation.

本文引入了大型语言模型在持续学习过程中出现的虚假遗忘概念，将其与实际的知识丧失区分开来，并将其归因于任务对齐的破坏。作者通过实验和理论分析表明，在新任务的早期训练阶段，模型（尤其是底层）可能会发生错位。为此，他们提出了一种冻结策略，即保持初始层不变，从而在安全对齐、指令微调等多种持续学习场景下显著提升模型性能。研究结果强调了任务对齐的重要性，相较于单纯的知识保留更为关键，并提供了一种实用的方法来缓解性能下降问题。

原文链接：https://arxiv.org/abs/2501.13453

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training

Summary

This paper explores the surprising parallels between learning-rate schedules used in large model training and theoretical performance bounds from convex optimization. It demonstrates that a simple learning-rate schedule with a constant phase followed by a linear cooldown mirrors the behavior predicted by theory, even for non-convex deep learning problems. Furthermore, the research shows how this theoretical understanding can be practically applied to improve learning-rate tuning for continued training and transfer optimal rates across different schedules, leading to tangible gains in model performance. The work provides theoretical justification for empirically successful scheduling techniques and suggests that principles from convex optimization offer valuable insights into the training of complex neural networks.

本文探讨了大型模型训练中使用的学习率调度与凸优化理论性能界限之间的惊人相似性。研究表明，一个简单的学习率调度方案——先保持恒定，然后线性降温——即使在非凸深度学习问题中，也能呈现出与理论预测相符的行为。此外，研究还展示了如何将这一理论理解实际应用于改进持续训练的学习率调优，并在不同的调度方案之间转移最优学习率，从而显著提升模型性能。本研究为经验上成功的调度技术提供了理论依据，并表明凸优化的原理可以为复杂神经网络的训练提供有价值的见解。

原文链接：https://arxiv.org/abs/2501.18965

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space

Summary

TokenVerse introduces a new method for multi-concept personalization in text-to-image generation. The technique extracts visual elements and attributes from single or multiple images using only text captions and a pre-trained diffusion model. By leveraging the modulation space within Diffusion Transformers, TokenVerse disentangles complex concepts like objects, poses, and lighting. This enables users to combine these learned concepts in novel ways to create customized images without needing additional supervision like masks. TokenVerse shows significant advantages over existing personalization techniques, providing greater flexibility and control for personalized content creation and storytelling. The paper presents quantitative and qualitative results demonstrating the effectiveness of the TokenVerse framework.

TokenVerse 提出了一个用于 文本到图像生成 的新方法，旨在实现多概念个性化。该技术通过仅使用文本描述和预训练的扩散模型，从单一或多个图像中提取视觉元素和属性。通过利用扩散变换器（Diffusion Transformers）中的调制空间，TokenVerse 解构了诸如物体、姿势和光照等复杂概念。

这种方法使用户能够以创新的方式将这些学习到的概念进行组合，从而创建个性化图像，而无需像遮罩（masks）之类的额外监督。与现有的个性化技术相比，TokenVerse 展现了显著的优势，提供了更大的灵活性和控制力，促进了个性化内容创作和叙事的实现。

论文通过定量和定性结果展示了 TokenVerse 框架的有效性，证明了其在个性化生成和故事创作中的潜力。

原文链接：https://arxiv.org/abs/2501.12224

前往小宇宙评论区与主播互动

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：TensorLLM: Tensorising Multi-Head Attention for Enhanced Reasoning and Compression in LLMs

Summary

This research introduces TensorLLM, a novel framework for improving the reasoning abilities and compression of Large Language Models (LLMs) by focusing on the Multi-Head Attention (MHA) block. The method employs multi-head tensorisation and Tucker decomposition to denoise and compress MHA weights by enforcing a shared higher-dimensional subspace across multiple attention heads. Experiments demonstrate that TensorLLM enhances LLM reasoning capabilities across various benchmark datasets and architectures without requiring additional training. The framework can also be combined with existing techniques that denoise the feed-forward network (FFN) layers for further performance gains. The study validates the approach through ablation experiments and comparisons with other compression techniques, showing consistent improvements in accuracy and compression rates. The paper concludes by emphasizing the potential of TensorLLM as a versatile module for improving LLMs and suggesting future work on finding generalizable hyperparameter settings.

本研究提出了 TensorLLM，一种新颖的框架，通过聚焦于多头自注意力（MHA）块来提升大型语言模型（LLM）的推理能力和压缩效率。该方法采用多头张量化和Tucker 分解，通过在多个注意力头之间强制共享一个更高维度的子空间，来去噪和压缩 MHA 权重。

实验表明，TensorLLM 在不同的基准数据集和架构上提升了 LLM 的推理能力，而无需额外的训练。该框架还可以与现有的去噪前馈网络（FFN）层的技术结合，进一步提升性能。通过消融实验和与其他压缩技术的比较，研究验证了该方法的有效性，显示出在准确性和压缩率方面的持续改进。

论文最后强调了 TensorLLM 作为一种多功能模块，具有提升 LLM 性能的潜力，并提出了未来研究的方向，即寻找可以广泛应用的超参数设置。

原文链接：https://arxiv.org/abs/2501.15674

前往小宇宙评论区与主播互动