Quantization is a powerful technique for reducing memory usage and speeding up AI applications built with LLMs, diffusion models, CNNs, and other architectures. In fact, quantization is fundamental to all data compression—from JPEG and GIF to MP3 and MP4 (HEVC)! In this episode, we'll cover the basics of neural network quantization, laying the groundwork for future episodes where we'll dive into specific quantization algorithms.
The AI Podcast is hosted by Kirill, CEO of TheStage AI. With his team's deep scientific and industrial expertise in neural network acceleration and deployment, they'll show you how to run AI anywhere and everywhere!
01:24 - Structure of Podcast
01:46 - When and How to Use Quantization?
03:11 - Speedup or reduce memory? Or Both?
04:18 - Hardware with quantization support
05:28 - DNN compilers to run quantized networks
06:01 - What is quantization mathematically?
07:22 - Fake Quantized Tensors
08:43 - Symmetric, asymmetric, per-tensor, per-channel, per-group
09:43 - Quantized matrix multiplication
11:31 - Quantization algorithms
13:23 - Examples of PTQ and QAT
16:11 - Quantized parameters exists not in discrete space! Is it manifold?
18:08 - Details of the next episode!
