AI Today

AI Today

By AI Today Tech TalkTechnology
Download on the App Store

AI Today episodes

  • Veo 2, Imagen 3, and Whisk: State-of-the-Art AI Image and Video Generation | #ai #2024 #genai

    Blog: https://blog.google/technology/google...

    Google announced updates to its AI video and image generation models, Veo 2 and Imagen 3, boasting state-of-the-art capabilities in realism and style diversity. These improvements are integrated into existing Google Labs tools, VideoFX and ImageFX, and a new tool, Whisk, which allows image-based prompting and remixing using Imagen 3 and Gemini's visual understanding. Veo 2 excels in cinematic video generation, while Imagen 3 produces high-quality images across various artistic styles. The updates emphasize responsible AI development, including SynthID watermarks to combat misinformation. Whisk is currently available in the US, with Veo 2 expanding to YouTube and other platforms in the future.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science, apple, samsung, anthropic, turing

    20 min
  • Allegro: Open the Black Box of Commercial-Level Video Generation Model | #ai #2024 #genai

    Paper: https://arxiv.org/pdf/2411.01747

    This research report introduces Allegro, a novel, open-source text-to-video generation model that surpasses existing open-source and many commercial models in quality and temporal consistency. The authors detail Allegro's architecture, a multi-stage training process leveraging a custom-designed Video Variational Autoencoder (VideoVAE) and Video Diffusion Transformer (VideoDiT), and a rigorous data curation pipeline resulting in a dataset of 106 million images and 48 million videos. Extensive evaluations, including user studies, demonstrate Allegro's superior performance across various metrics, though some limitations remain, particularly regarding large-scale motion. The authors also provide insights into future improvements, including expanding model capabilities and enhancing data diversity. Finally, the complete Allegro model and code are released under the Apache 2.0 license.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science, apple, samsung, anthropic, turing

    20 min
  • DynaSaur : Large Language Agents Beyond Predefined Actions | #ai #2024 #genai

    Paper: https://arxiv.org/pdf/2411.01747

    The paper "DynaSaur: Large Language Agents Beyond Predefined Actions" introduces a novel large language model (LLM) agent framework that dynamically generates and executes actions using a general-purpose programming language, overcoming limitations of existing systems restricted to predefined action sets. This approach enhances the LLM agent's flexibility and planning capabilities, significantly improving performance as demonstrated by its top ranking on the GAIA benchmark. The framework allows for action reuse and recovery from unexpected situations. The authors provide code and a preprint of their research.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science, apple, samsung, anthropic, turing

    20 min
  • STAR ATTENTION: EFFICIENT LLM INFERENCE OVER LONG SEQUENCES | #ai #2024 #genai

    Paper: https://arxiv.org/pdf/2411.17116

    The paper introduces Star Attention, a novel two-phase attention mechanism for efficient Large Language Model (LLM) inference on long sequences. It improves computational efficiency by sharding attention across multiple hosts, using blockwise-local attention in the first phase and sequence-global attention in the second. This approach achieves up to an 11x speedup in inference time while maintaining high accuracy (95-100%). The effectiveness of Star Attention is demonstrated through experiments on various LLMs and benchmarks, exploring the trade-off between speed and accuracy based on block size and anchor block design. The research also analyzes the algorithm's performance across different task categories.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science, apple, samsung, anthropic, turing

    17 min
  • FERRET-UI 2: MASTERING UNIVERSAL USER INTERFACE UNDERSTANDING ACROSS PLATFORMS | #ai #2024 #genai

    Paper: https://arxiv.org/pdf/2410.18967

    The paper introduces Ferret-UI 2, a multimodal large language model (MLLM) that significantly improves upon its predecessor, Ferret-UI, by enabling universal user interface (UI) understanding across diverse platforms (iPhone, Android, iPad, webpages, and AppleTV). Key improvements include multi-platform support, high-resolution perception through adaptive scaling, and advanced task training data generation using GPT-4o with visual prompting. Ferret-UI 2 demonstrates superior performance on various benchmarks, showcasing strong cross-platform transfer capabilities and surpassing existing models in UI referring, grounding, and user-centric advanced tasks. The enhanced model architecture and higher-quality training data contribute to these advancements. The authors conclude by outlining future work focusing on broader platform coverage and the development of a truly generalist UI navigation agent.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science, apple, samsung

    15 min
  • Adapting While Learning: Grounding LLMs for Scientific Problems I-Tool Usage Adaptation | #ai #2024

    Paper: https://arxiv.org/abs/2411.00412

    This research introduces a novel two-stage training method to improve Large Language Models' (LLMs) ability to solve complex scientific problems. The method, called Adapting While Learning (AWL), first distills world knowledge into the LLM via supervised fine-tuning. Then, it adapts tool usage by classifying problems as easy or hard, using direct reasoning for easy problems and tools for hard ones. Experiments across various scientific datasets show significant improvements in both answer accuracy and tool usage precision, surpassing several state-of-the-art LLMs. The study also explores extensions to open-ended questions and robustness to noisy data.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science

    15 min
  • Mixtures of In-Context Learners | #ai #genai #llm #2024 #ml

    Paper: https://arxiv.org/pdf/2411.02830

    This research introduces Mixtures of In-Context Learners (MOICL), a novel approach to improve in-context learning (ICL) in large language models (LLMs). MOICL addresses ICL's limitations by partitioning demonstrations into expert subsets and learning a weighting function to combine their predictions. Experiments demonstrate MOICL's superior performance across various classification datasets, enhanced efficiency, and robustness to noisy or imbalanced data. The method dynamically identifies helpful and unhelpful demonstration subsets, improving accuracy and reducing computational costs. A key advantage is MOICL's ability to handle more demonstrations than standard ICL by mitigating the quadratic complexity of attention mechanisms.
    ai , artificial intelligence , arxiv , research , paper , publication , llm, genai, generative ai , large visual models, large language models, large multi modal models, nlp, text, machine learning, ml, nividia, openai, anthropic, microsoft, google, technology, cutting-edge, meta, llama, chatgpt, gpt, elon musk, sam altman, deployment, engineering, scholar, science

    15 min
  • LLM2CLIP: POWERFUL LM UNLOCKS RICHER VISUAL REPRESENTATION | #ai #genai #lvm #llm #mmm #cv #ms #2024

    Paper: https://arxiv.org/pdf/2411.04997

    Github: https://github.com/microsoft/LLM2CLIP
    The paper introduces LLM2CLIP, a method to improve the visual representation learning capabilities of CLIP by integrating large language models (LLMs). LLM2CLIP addresses CLIP's limitations with long and complex text by fine-tuning the LLM to enhance its textual discriminability, effectively using the LLM's knowledge to guide CLIP's visual encoder. Experiments demonstrate significant performance improvements across various image-text retrieval tasks and benchmarks, including cross-lingual retrieval. The approach is efficient, requiring minimal additional computational cost compared to training the original CLIP model. The improved model shows enhanced understanding of long and complex text semantics, exceeding the performance of state-of-the-art CLIP models.
    ai , computer vision , cv , peking university , artificial intelligence , arxiv , research , paper , publication , lvm , large visual models

    15 min
  • OPENSCHOLAR: SYNTHESIZING SCIENTIFICLITERATURE WITH RETRIEVAL-AUGMENTED LMS | #ai #genai #llm #2024

    Paper: https://arxiv.org/pdf/2411.14199

    Github: https://github.com/AkariAsai/OpenScholar
    The research introduces OpenScholar, a retrieval-augmented large language model (LLM) designed for synthesizing scientific literature. OpenScholar uses a large datastore of open-access papers and iterative self-feedback to generate high-quality responses to scientific questions, including accurate citations. A new benchmark, ScholarQABench, is introduced for evaluating open-ended scientific question answering, incorporating both automatic and human evaluations. Experiments demonstrate OpenScholar's superior performance compared to other LLMs and even human experts in certain aspects, particularly in terms of information coverage. Limitations of OpenScholar and ScholarQABench are discussed, alongside plans for open-sourcing the model and benchmark.
    ai , llm , retrieval augmented, rag , artificial intelligence , arxiv , research , paper , publication , genai , generativeai, agentic

    15 min
  • Bilateral Reference for High-Resolution Dichotomous Image Segmentation | #ai #genai #llm #cv #2024

    Paper: https://arxiv.org/pdf/2401.03407

    Github: https://github.com/ZhengPeng7/BiRefNet
    This research introduces BiRefNet, a novel deep learning framework for high-resolution dichotomous image segmentation. BiRefNet uses a bilateral reference mechanism, incorporating both original image patches and gradient maps, to improve the accuracy of segmenting fine details. The framework is composed of localization and reconstruction modules, enhancing performance through multi-stage supervision and other training strategies. Extensive experiments demonstrate BiRefNet's superior performance across several image segmentation tasks, outperforming existing state-of-the-art methods. The authors also highlight the model's potential applications and its adoption by the community for various third-party projects.
    ai , computer vision , cv , nankai university , artificial intelligence , arxiv , research , paper , publication , lvm , large visual models, llm

    15 min

About AI Today

From the publisher's feed

Welcome to AI Today TechTalk – where we geek out about the coolest, craziest, and most mind-blowing stuff happening in the world of Artificial Intelligence! 🚀