GPT Reviews

Microsoft's £2.5 billion investment in AI infrastructure and skills in the UK, iA Writer's new authorship feature, and research papers on Universal Self-Consistency for Large Language Model Generation, One-step Diffusion with Distribution Matching Distillation, and QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:59 Boost for UK AI as Microsoft unveils £2.5 billion investment

03:30 iA Writer can now track what you or ChatGPT wrote

05:35 Extracting Training Data from ChatGPT

06:53 Fake sponsor

08:56 Universal Self-Consistency for Large Language Model Generation

10:44 One-step Diffusion with Distribution Matching Distillation

12:42 QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models

15:09 Outro

OpenAI's Q* model, which is viewed as a potential breakthrough in the quest for Artificial General Intelligence. They also cover Andrej Karpathy's Intro to Large Language Models, DuckTrack, a new multimodal computer interaction data collector, and GAIA, a benchmark for General AI Assistants that proposes real-world questions that require a set of fundamental abilities.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:57 OpenAI’s Q* model: Was an AGI breakthrough the impetus for the management crisis?

03:38 Andrej Karpathy's Intro to Large Language Models

04:54 DuckTrack: Accurate Computer Activity Tracking

05:52 Fake sponsor

07:37 White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?

09:15 GAIA: a benchmark for General AI Assistants

10:47 Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

12:20 Outro

Sam Altman's return as CEO of OpenAI after a power struggle, the release of a new voice feature for OpenAI's ChatGPT AI language model, the release of Inflection-2, and the Orca 2 project teaching small language models how to reason effectively.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:50 Sam Altman Returns as the CEO of OpenAI

03:21 OpenAI Voice for all users

04:52 Inflection-2 Released

06:52 Twitter Thread on the usage of Claude's 200k context size

08:23 Fake sponsor

10:29 ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

12:01 Orca 2: Teaching Small Language Models How to Reason

14:10 Outro

Generative video models, language model finetuning, and generative retrieval. Claude 2.1 boasts a larger context window and decreased false statements, while Stable Video Diffusion can be applied in various sectors. LQ-LoRA offers a promising approach for more efficient language model finetuning and compression, and RIPOR surpasses state-of-the-art generative retrieval models by a large margin.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:31 Anthropic Introduces Claude 2.1

03:14 Introducing Stable Video Diffusion

05:16 Cryptic Note to OpenAI Board Surfaces

06:37 Fake sponsor

09:20 System 2 Attention (is something you might need too)

10:45 LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning

12:52 Scalable and Effective Generative Information Retrieval

15:04 Outro

The drama at OpenAI with Sam Altman trying to return as CEO and staff threatening to quit unless the board resigns. We also explore the potential of using shallow neural networks as an alternative to attention layers in transformers, and a paper that proposes a method called SelfEval for evaluating generative models. Additionally, we discuss a paper that explores the effectiveness of using shallow feed-forward networks as an alternative to the attention mechanism in the Transformer model.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:39 Sam Altman is still trying to return as OpenAI CEO

02:52 OpenAI Staff Threaten to Quit Unless Board Resigns

04:34 Large Language Models and Lost in the Middle

06:09 Fake sponsor

07:33 LLMs cannot find reasoning errors, but can correct them!

09:00 Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers

10:41 SelfEval: Leveraging the discriminative nature of generative models for evaluation

12:28 Outro

the fallout from Sam Altman's firing from OpenAI and Meta's decision to disband its Responsible AI team. Our collaborators also break down some cutting-edge research in the AI field, including contrastive chain-of-thought prompting, multi-agent reinforcement learning, and language models. Finally, we discuss ML-Bench, a new evaluation setup for large language models that leverages open-source libraries for machine learning tasks. Tune in to stay up-to-date on the latest developments in the world of AI.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:31 A timeline of Sam Altman’s firing from OpenAI — and the fallout

02:50 Meta disbanded its Responsible AI team

04:40 Beyond Singular Intelligence: Exploring Multi-Agent Systems and Multi-LoRA in the Quest for AGI

06:16 Fake sponsor

08:11 Contrastive Chain-of-Thought Prompting

09:47 JaxMARL: Multi-Agent RL Environments in JAX

11:33 ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks

13:25 Outro

From Google's new inaudible watermarks in AI-generated music to Microsoft's new AI chips and NVIDIA's breakthrough in chip design using generative AI, there's plenty to get excited about. DeepMind's new music generation model and experiments in music AI tools are also explored. Additionally, research papers on evaluating Large Language Models on non-English languages and confidence estimation are discussed, highlighting the importance of developing NLP technologies for non-English speakers and maintaining user trust in large language models.

Contact:  [email protected]

Timestamps:

00:34 Introduction

01:29 Google is embedding inaudible watermarks right into its AI generated music

03:30 Microsoft Launches New AI Chips

04:28 Silicon Volley: Designers Tap Generative AI for a Chip Assist

05:46 DeepMind's New Music Generation Model

07:15 Fake sponsor

09:05 MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks

10:47 Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation

12:28 Outro

Airbnb's acquisition of a secretive AI startup launched by Siri's co-founder, Notion's new Q&A feature that allows users to ask an AI about their notes, and Microsoft's infrastructure developments in AI and custom silicon. The team also explores research papers on improving the factuality of language models, refining large language models' self-judgment, and the vulnerability of language models to adversarial arithmetic attacks.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:46 Airbnb acquires secretive firm launched by Siri co-founder

04:03 Notion’s new Q&A feature lets you ask an AI about your notes

06:04 Microsoft Infrastructure - AI & CPU Custom Silicon Maia 100, Athena, Cobalt 100

08:14 Fake sponsor

10:53 Fine-tuning Language Models for Factuality

12:15 The ART of LLM Refinement: Ask, Refine, and Trust

13:44 Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?

15:34 Outro

NVIDIA's supercharged AI computing platform, Hopper, and YouTube's new labeling system for AI-generated videos. The team also explores groundbreaking AI research papers, including the impact of large language models on scientific discovery, the capabilities of large multimodal models, and the development of a universal navigation system for mobile robots called GOAT.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:22 NVIDIA Supercharges Hopper, the World’s Leading AI Computing Platform

03:53 YouTube will show labels on videos that use AI

05:44 GraphCast: AI model for faster and more accurate global weather forecasting

07:57 Fake sponsor

10:15 The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4

11:50 The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

13:33 GOAT: GO to Any Thing

15:46 Outro

The fierce battle for talent between OpenAI and Google, as well as Microsoft's development of AI chips to rival Nvidia. The team also explores the impact of the Hollywood Actors Strike on AI and streaming. Additionally, they review three cutting-edge research papers on topics such as a unified vision model, fast text-to-3D generation, and efficient convolutions.

Contact:  [email protected]

Timestamps:

00:34 Introduction

02:30 OpenAI lures Google's top AI researchers with multimillion-dollar offers

03:59 Hollywood Actors Strike Ends With a Deal That Will Impact AI and Streaming for Decades

05:47 Jason Wei (OpenAI researcher) Tweets

07:58 Fake sponsor

10:45 Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

12:11 Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model

13:27 FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

15:43 Outro