This document details research into improving Federated Learning (FL) efficiency in autonomous mobile networks by incorporating tiny language models (TLMs) for predicting network performance features. It focuses on the challenge of communication overhead in FL due to frequent neural network data exchanges. The paper proposes and evaluates the use of NNCodec, an implementation of the ISO/IEC Neural Network Coding (NNC) standard, to compress the data shared between participating network cells (clients) and a central server. Experimental results using the Berlin V2X dataset demonstrate that NNCodec can significantly reduce communication data while maintaining the prediction performance of the TLMs.

This technical report introduces Mobile-MMLU, a new benchmark designed to evaluate large language models (LLMs) specifically for mobile devices, addressing the limitations of existing benchmarks which focus on desktop or server environments. Mobile-MMLU and its challenging subset, Mobile-MMLU-Pro, consist of thousands of multiple-choice questions across 80 mobile-relevant domains, emphasizing practical daily tasks and on-device AI constraints like efficiency and privacy. The creation process involved AI and human collaboration to generate and refine questions, ensuring relevance and mitigating biases. Evaluation results show that Mobile-MMLU effectively differentiates the performance of LLMs in mobile contexts, revealing that strong performance on traditional benchmarks doesn't guarantee success on mobile tasks.

This document presents AI-RAN, a paradigm shift integrating Radio Access Network (RAN) and Artificial Intelligence (AI) workloads onto a unified platform. It outlines the evolution of RAN and categorizes AI-RAN into three forms: AI-for-RAN, AI-on-RAN, and AI-and-RAN. The paper identifies key requirements and enablers for this convergence, including accelerated computing and cloud-native design. A proposed reference architecture is provided, and a proof-of-concept utilizing NVIDIA Grace-Hopper servers demonstrates the potential for concurrent workload processing and improved asset utilization. Finally, the document concludes with future work directions for AI-RAN.

These sources collectively explain that fine-tuning is a process of retraining a pre-trained Large Language Model on a specialized dataset to enhance its performance on particular tasks or domains. While it can significantly improve a model's responses, it's not ideal for injecting entirely new factual knowledge. Several methods exist for evaluating fine-tuned models, including general knowledge tests, human preference comparisons, and using another larger model to score responses. Due to the significant computational resources needed for full fine-tuning, Parameter-Efficient Fine-Tuning (PEFT) techniques are increasingly popular, offering similar performance with considerably reduced computational costs and memory requirements by only training a small subset of parameters. Ollama, a local application, facilitates the use and evaluation of these models.

This document describes the development of a Large Language Model (LLM) specifically tailored for explaining VHDL code within a high-performance processor design environment. Recognizing the unique requirements of such settings, including data security and leveraging existing design knowledge, the researchers employed extended pretraining (EPT) and instruction tuning on a base LLM using proprietary data. They created specialized test sets and utilized an LLM-as-a-judge approach to efficiently evaluate model performance, finding significant improvements in explanation accuracy compared to the original model. The work highlights the potential of customized LLMs to enhance productivity and facilitate knowledge transfer in complex hardware design workflows.

This document introduces and analyzes AWNN (Adaptively Weighted Nearest Neighbors), a novel matrix completion method. Traditional Nearest Neighbor (NN) methods struggle with selecting the appropriate number of neighbors and their weights, often relying on computationally expensive techniques like cross-validation. AWNN addresses this by formulating weight selection as a convex optimization problem, balancing bias-variance tradeoffs and providing a data-driven approach for tuning without manual intervention. The authors present theoretical guarantees for AWNN, demonstrating its performance advantages over unweighted NN methods, particularly in handling missing data, and support these claims with empirical results from synthetic experiments. AWNN offers a principled and efficient alternative for matrix completion in various applications.

This academic paper explores the training dynamics of neural networks, specifically focusing on gradient flow for fully connected feedforward networks with various smooth activation functions. The authors establish a dichotomy, showing that gradient flow either converges to a critical point or diverges to infinity while the loss approaches a generalized critical value. Utilizing the mathematical framework of o-minimal structures, they prove that for certain nonlinear polynomial target functions, sufficiently large networks and datasets lead to loss values approaching zero only asymptotically, causing the gradient flow to diverge when initialized well. The paper supports these theoretical findings with numerical experiments on polynomial regression and real-world tasks, observing the parameter norm increasing as the loss decreases.

This academic paper introduces WavReward, a novel evaluation system for end-to-end spoken dialogue models, which process speech input and output directly, unlike older systems that rely on text. Recognizing the limitations of existing evaluations that primarily focus on text, WavReward leverages audio language models to assess both the content and acoustic aspects of spoken interactions, including factors like emotion and tone. The authors also present ChatReward-30K, a new dataset containing speech-to-speech dialogues with human-assigned scores, designed specifically for training and testing spoken dialogue evaluators, thus addressing a significant gap in available resources for this field. Experiments show that WavReward, utilizing techniques like reinforcement learning and reasoning, surpasses current evaluation methods and aligns better with human judgment.

This academic paper introduces BLIP3-o, a suite of cutting-edge multimodal models designed for both understanding and generating images. The research investigates various architectural choices and training techniques, finding that CLIP image features and flow matching are effective for image generation, while a sequential training strategy—starting with understanding before generation—yields the best overall performance. The authors also present BLIP3o-60k, a new dataset created with GPT-4o, to improve the models' ability to follow instructions and produce aesthetically pleasing images. The paper includes performance benchmarks and a human study demonstrating BLIP3-o's superior capabilities and offers its components as open-source resources to encourage further advancements in unified multimodal AI.

This document introduces CodePDE, a new framework for using large language models (LLMs) to generate code that solves partial differential equations (PDEs). The authors frame PDE solving as a code generation problem and demonstrate that, with techniques like debugging and refinement, LLMs can create solvers that are competitive with, and sometimes surpass, human-written solvers on various PDE families like Burgers, Advection, and Darcy flow. They highlight the ability of LLMs to reason, debug, and improve code through feedback, suggesting a promising future for LLMs in scientific computing, despite challenges with certain PDE types like Reaction-Diffusion.