GPT Reviews

Tesla and OpenAI are in a talent war for AI experts, with OpenAI offering salaries of up to $925,000.

YouTube has warned that OpenAI's training of its text-to-video AI model using YouTube videos would violate its policies.

Three AI research papers are discussed, including a new approach to unsupervised domain adaptation for ranking, a more efficient method for dense retrieval using bit vectors, and a new approach to representation finetuning for language models.

The episode includes humorous banter and a quirky sponsor segment for a mosquito repellent that doesn't work.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:26 Tesla vs OpenAI Talent Wars

02:51 YouTube Says OpenAI Training Sora With Its Videos Would Break Rules

04:25 Your guide to AI: April 2024

06:26 Fake sponsor

08:34 ReFT: Representation Finetuning for Language Models

09:58 Efficient Multi-Vector Dense Retrieval Using Bit Vectors

11:54 DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation

13:51 Outro

Command R+ is a new language model designed for enterprise-grade workloads that outperforms similar models in the scalable market category and offers multilingual coverage in 10 key languages to support global business operations.

JetMoE-8B is a new model that was trained with less than $0.1 million cost and outperformed LLaMA2-7B from Meta AI, who has multi-billion-dollar training resources.

Mixture-of-Depths is a new method proposed for transformer-based language models that dynamically allocates compute to specific positions in a sequence, optimizing the allocation along the sequence for different layers across the model depth.

Think-and-Execute is a new framework that aims to improve algorithmic reasoning in large language models by decomposing the reasoning process into two steps: discovering task-level logic that is shared across all instances for solving a given task and expressing it with pseudocode, and simulating the generated pseudocode to execute the code. 

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:42 Introducing Command R+: A Scalable LLM Built for Business

03:38 JetMoE: Reaching LLaMA2 Performance with 0.1M Dollars

05:08 AI & the Web: Understanding and managing the impact of Machine Learning models on the Web

06:37 Fake sponsor

08:44 Do language models plan ahead for future tokens?

10:04 Mixture-of-Depths: Dynamically allocating compute in transformer-based language models

11:33 Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models

13:40 Outro

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:52 ‘Lavender’: The AI machine directing Israel’s bombing spree in Gaza

04:01 Open Source SWE Agent

05:17 IPEX-LLM

06:58 Fake sponsor

08:39 Measuring Style Similarity in Diffusion Models

10:11 Advancing LLM Reasoning Generalists with Preference Trees

12:11 Octopus v2: On-device language model for super agent

14:05 Outro

The potential risks associated with false claims and grifting in the AI industry.

The resurgence of Intel in the foundry and product space, with a goal to become the second-largest foundry by 2030.

The discovery of a new technique called "Many-shot jailbreaking" that can be used to evade the safety guardrails of large language models.

The evaluation of the performance of large language models in long in-context learning scenarios, highlighting a notable gap in current LLM capabilities for processing and understanding long, context-rich sequences.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:47 Google's DeepMind CEO says the massive funds flowing into AI bring with it loads of hype and a fair share of grifting

03:37 Many-shot jailbreaking

05:07 Is Intel Back? Foundry & Product Resurgence Measured

06:21 Fake sponsor

08:03 Logits of API-Protected LLMs Leak Proprietary Information

09:48 ViTamin: Designing Scalable Vision Models in the Vision-Language Era

11:24 Long-context LLMs Struggle with Long In-context Learning

13:09 Outro

OpenAI's voice cloning AI model can create a synthetic voice based on just a 15-second clip of someone's voice, while Microsoft and OpenAI are reportedly working on a data center project that could cost up to $100 billion and include an AI supercomputer called "Stargate". "On the Impossibility of Supersized Machines" presents seven distinct arguments that show why it is not only implausible but impossible for machines to ever exceed human size. Finally, "DiJiang: Efficient Large Language Models through Compact Kernelization" proposes a novel frequency domain kernelization approach that can transform a pre-trained vanilla transformer into a linear complexity model, with little training costs.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:00 OpenAI’s voice cloning AI model only needs a 15-second sample to work

03:38 Microsoft, OpenAI plan $100 billion data-center project, media report says

05:25 RagFlow

06:42 Fake sponsor

08:39 On the Impossibility of Supersized Machines

09:56 Jamba: A Hybrid Transformer-Mamba Language Model

11:52 DiJiang: Efficient Large Language Models through Compact Kernelization

13:29 Outro

Amazon's $2.75 billion investment in Anthropic, a top player in AI research and development, is the largest outside investment in history and could have significant implications for the AI arms race.

The potential economic explosion caused by AI is explored in a Vox article, discussing how AI could cause economic growth at a scale never seen before.

Anthropic's Claude 3 Opus language model has unseated OpenAI's GPT-4 on Chatbot Arena, marking a notable moment in the relatively short history of AI language models.

Three impressive AI research papers are discussed, including a reproduction of OpenAI's TL;DR summarization work using Reinforcement Learning from Human Feedback, a joint model for list-aware retrieval that achieved state-of-the-art performance, and ViTAR, a cost-effective solution for enhancing the resolution scalability of Vision Transformers.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:50 Amazon Invests $2.75B in Anthropic

03:29 How AI could explode the economy and how it could fizzle

04:39 “The king is dead”—Claude 3 surpasses GPT-4 on Chatbot Arena for the first time

06:17 Fake sponsor

08:02 The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization

09:24 List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation

10:53 ViTAR: Vision Transformer with Any Resolution

12:46 Outro

DataBricks introduces DBRX, a new state-of-the-art open LLM that surpasses GPT-3.5 and even competes with Gemini 1.0 Pro.

Apple's upcoming AI strategy for iOS 18 is expected to include a revamped version of Siri, AI integrated into iMessage, auto-generated playlists, and even an AI health coach for watches.

BinaryVectorDB offers lightning-fast search capabilities on large datasets with an embedding model that uses native int8 and binary support.

DINO-Tracker is a new framework for long-term dense tracking in videos that achieves state-of-the-art results on known benchmarks and significantly outperforms self-supervised methods.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:07 Introducing DBRX: A New State-of-the-Art Open LLM

03:48 Apple Presenting AI Strategy on June 10th for iOS 18

05:22 BinaryVectorDB - Efficient Search on Large Datasets

06:35 Fake sponsor

08:14 Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs

10:00 DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video

11:17 The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarization

12:37 Outro

A group of tech giants, including Google and Intel, are teaming up to challenge Nvidia's monopoly on the AI chip market with an open-source software suite called the UXL Foundation.

Baidu has partnered with Apple to provide the AI backend for the Chinese versions of the iPhone 16, macOS, and the upcoming iOS 18, beating out rival Alibaba and others to secure the role.

The Elasticsearch open inference API has added support for Cohere Embeddings, allowing developers to build semantic search use cases.

AIOS is an LLM agent operating system that embeds large language models into operating systems as the brain of the OS, enabling an operating system "with soul." It optimizes resource allocation, facilitates context switch across agents, enables concurrent execution of agents, provides tool service for agents, and maintains access control for agents.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:44 Behind the plot to break Nvidia's grip on AI by targeting software

02:47 Baidu Shares Rise After Reports That Apple Will Use Its AI Services in China Products

04:33 Elasticsearch open inference API adds support for Cohere Embeddings

05:47 Fake sponsor

07:26 Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance

08:57 AIOS: LLM Agent Operating System

10:38 MasonTigers at SemEval-2024 Task 9: Solving Puzzles with an Ensemble of Chain-of-Thoughts

12:25 Outro

Stability AI founder Emad Mostaque plans to resign as CEO amid investor pressure and financial struggles.

OpenAI is courting Hollywood studios and directors with its unreleased AI video generation tool, Sora.

Researchers are exploring ways to teach information retrieval models to follow complex instructions and enhance LLMs with iterative data augmentation strategies.

A study investigates the ability of large language models to engage in exploration and highlights the need for algorithmic interventions in complex decision-making settings.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:49 Stability AI Founder Emad Mostaque Plans To Resign As CEO

03:05 OpenAI Courts Hollywood in Meetings With Film Studios, Directors

05:03 Nvidia’s Optical Boogeyman – NVL72, Infiniband Scale Out, 800G & 1.6T Ramp

06:03 Fake sponsor

08:18 FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions

09:52 LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement

11:59 Can large language models explore in-context?

13:57 Outro

OpenAI is set to release GPT-5, a language model that could revolutionize the AI world by calling on other AI agents to help with tasks.

The European Conference on Information Retrieval is taking place in Glasgow, with a focus on ethical issues in information retrieval technologies.

Three AI research papers were discussed, including a new approach to generating high-quality videos more efficiently, a framework for simplifying video editing, and a novel mesh-based representation for real-time rendering and editing of complex 3D effects.

The episode also included some fun banter and jokes, making for an entertaining and informative listen.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:25 OpenAI Might Launch GPT-5 Soon

03:00 European Conference on Information Retrieval This Week in Glasgow

05:03 Scaling vector search using Cohere binary embeddings and Vespa

06:20 Fake sponsor

08:42 Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition

10:14 AnyV2V: A Plug-and-Play Framework For Any Video-to-Video Editing Tasks

11:49 Gaussian Frosting: Editable Complex Radiance Fields with Real-Time Rendering

13:28 Outro