GPT Reviews

OpenAI's ChatGPT has released a new update to give users more control over how it responds. A.I. companies have agreed to voluntary safeguards to manage the risks associated with their technology. "Secrets of RLHF in Large Language Models Part I: PPO" introduces a new approach to reinforcement learning with human feedback, which is important for large language models. Finally, "SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval" introduces a new paradigm in retrieval called neural sparse retrieval, and a toolkit called SPRINT to evaluate and compare different models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:48 Custom instructions for ChatGPT

03:20 Pressured by Biden, A.I. Companies Agree to Guardrails on New Tools

05:17 llama2.c Repository by Andrej Karpathy

06:31 Fake sponsor

08:27 Secrets of RLHF in Large Language Models Part I: PPO

10:19 Provably Faster Gradient Descent via Long Steps

11:42 SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval

13:49 Outro

Cerebras has sold a $100 million AI supercomputer and is planning eight more, challenging the market for AI hardware and validating the market for specialized AI hardware outside of GPUs. The NYC subway system is using AI to track fare evasion, raising concerns about privacy and surveillance. LLM Evaluation research...

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:39 Cerebras Sells $100 Million AI Supercomputer, Plans Eight More

03:14 NYC subway using AI to track fare evasion

04:50 AI Safety and the Age of Dislightenment

06:23 Fake sponsor

08:12 Towards A Unified Agent with Foundation Models

09:40 L-Eval: Instituting Standardized Evaluation for Long Context Language Models

11:48 SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models

13:34 Outro

LLAMA 2, the new open-source conversational language model from Meta, has been released, with Microsoft as the preferred partner. Apple has created its own AI-based chatbot called "Apple GPT" to compete with Google and Open AI, but had to halt the rollout due to security concerns around generative AI. "Precise Zero-Shot Dense Retrieval without Relevance Labels" proposes a new approach called Hypothetical Document Embeddings (HyDE) to address the challenge of creating fully zero-shot dense retrieval systems without any relevance labels. "To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations" explores the use of large language models and multi-agent simulations to generate high-quality episodic content for intellectual properties.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:59 LLAMA 2 is here, the new open-source conversational Language Model from Meta

03:28 Apple is testing a ChatGPT-like AI chatbot

05:12 The evolution of GPT-4's capabilities

06:13 Fake sponsor

07:59 Precise Zero-Shot Dense Retrieval without Relevance Labels

09:30 Diffusion Models Beat GANs on Image Classification

11:32 To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

13:43 Outro

A comprehensive look at Wix's AI journey, from its current AI-powered features to its ambitious future plans. It also delves into the SEC's warning about AI's potential risks to financial stability, including its use in financial fraud and conflicts of interest. The episode also features a deep dive into three intriguing research papers, exploring the NaViT Vision Transformer, the Retentive Network as a potential successor to Transformers, and a theory on Adam Instability in large-scale machine learning. Lastly, the episode includes a segment on Random Reads, where potential inaccuracies in the "gzip beats BERT" paper are discussed.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:14 To our users, my thoughts on AI: past, present and future

03:46 SEC warns AI risks financial stability

05:32 Bad numbers in the "gzip beats BERT" paper?

06:57 Fake sponsor

09:02 Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

10:45 Retentive Network: A Successor to Transformer for Large Language Models

12:29 A Theory on Adam Instability in Large-Scale Machine Learning

14:29 Outro

CM3leon, a new generative model for text and images that is more efficient and state-of-the-art. OpenAI researcher Jason Wei is also featured, offering an "Ask Me Anything" document on AI research. Additionally, Sumformer, a linear-complexity alternative to self-attention for speech recognition, and DreamTeacher, a self-supervised feature representation learning framework that uses generative networks for pre-training downstream image backbones, are discussed.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:16 Introducing CM3leon, a more efficient, state-of-the-art generative model for text and images

03:40 OpenAI product leader denies claims GPT-4 has gotten ‘lazier and dumber’

05:12 Jason Wei (OpenAI Researcher) tweets

06:04 Fake sponsor

07:54 Sumformer: A Linear-Complexity Alternative to Self-Attention for Speech Recognition

09:05 NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis

10:19 DreamTeacher: Pretraining Image Backbones with Deep Generative Models

12:15 Outro

AP's partnership with OpenAI, Google's language model Bard, and Meta's release of a commercial version of LLaMA. Additionally, two AI research papers are discussed, one about using LLMs to help robots with complex tasks and another about a hypernetwork called HyperDreamBooth that can efficiently generate personalized weights from a single image of a person.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:30 AP strikes news-sharing and tech deal with OpenAI

02:37 July Bard Update from Google

04:01 Meta to release open-source commercial AI model to compete with OpenAI and Google

06:07 Twitter Thread on scaling LLAMA to 8k context

07:18 Fake sponsor

09:14 Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners

10:55 HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models

12:48 Outro

Elon Musk's new AI company, xAI, aims to understand the true nature of the universe and has a team of heavy hitters from AI powerhouses.

Anthropic's new model, Claude 2, has made significant improvements in coding, math, and reasoning, and is being used by businesses for a wide variety of use cases.

Google's new AI-backed tool, NotebookLM, is a note-taking tool that uses AI to help users with research and document review, and is part of Google's push to integrate AI into every aspect of our lives.

The challenges of regulating advanced AI models with potentially dangerous capabilities are discussed in a paper titled "Frontier AI Regulation: Managing Emerging Risks to Public Safety", which proposes safety standards to address the risks of frontier AI models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:35 Elon Musk’s new xAI company launches to ‘understand the true nature of the universe’

02:59 Anthropic Releases Claude 2

04:54 Google Introduces NotebookLM

06:19 Fake sponsor

07:50 Instruction Mining: High-Quality Instruction Data Selection for Large Language Models

09:35 Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives

11:12 Frontier AI Regulation: Managing Emerging Risks to Public Safety

13:27 Outro

Google's efforts to combat political misinformation, Volkswagen's partnership with Mobileye to launch autonomous vehicles, and two AI research papers on video understanding and efficient text generation. The episode highlights the need for improved video-focused foundation models and the potential for more efficient inference in large language models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:27 Google working on tech to discern AI-made content: Company executive

03:19 Volkswagen to start testing self-driving ID Buzz vans in Austin

04:55 Yao Fu Tweets about Hallucination in Language Models

06:11 Fake sponsor

08:31 VideoGLUE: Video General Understanding Evaluation of Foundation Models

10:11 Lost in the Middle: How Language Models Use Long Contexts

11:25 SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference

13:16 Outro

From the concerning flood of AI-generated books on Amazon's Kindle Program to OpenAI's Superalignment team dedicated to addressing the superintelligence alignment problem. The team also discusses the temporary shutdown of the web browsing feature for ChatGPT Plus subscribers and three research papers, including LongNet, SDXL, and KnowNo. These papers cover topics such as scaling sequence length, text-to-image synthesis, and aligning the uncertainty of LLM-based planners.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:34 Amazon has a big problem as AI-generated books flood Kindle Unlimited

02:54 OpenAI disables ChatGPT Web Browsing

04:30 Introducing Superalignment

06:27 Fake sponsor

08:14 LongNet: Scaling Transformers to 1,000,000,000 Tokens

09:28 SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

11:04 Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners

12:50 Outro

The EU and Japan's potential partnership on AI and chips to reduce reliance on China, the UN Security Council's first-ever meeting on the potential threats of AI to global peace and security, a paper challenging the belief that pre-trained models always outperform meta-learning algorithms in few-shot learning, and Microsoft's ZeRO++ introducing communication volume reduction techniques to improve the efficiency of training large language models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:42 EU and Japan look to partner on A.I. and chips as China ‘de-risking’ strategy continues

02:58 UN council to hold first meeting on potential threats of artificial intelligence to global peace

04:53 June 2023, A Stage Review of Instruction Tuning

05:48 Fake sponsor

07:36 Is Pre-training Truly Better Than Meta-Learning?

09:37 ZeRO++: Extremely Efficient Collective Communication for Giant Model Training

11:26 Understanding Parameter Sharing in Transformers

13:46 Outro