Papers Read on AI

We show that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models. We achieve this on unconditional image synthesis by finding a better architecture through a series of ablations. For conditional image synthesis, we further improve sample quality with classifier guidance: a simple, compute-efficient method for trading off diversity for sample quality using gradients from a classifier. We match BigGAN-deep even with as few as 25 forward passes per sample, all while maintaining better coverage of the distribution. Finally, we find that classifier guidance combines well with upsampling diffusion models. We release our code at https://github.com/ openai/guided-diffusion.

2021: Prafulla Dhariwal, Alex Nichol

Ranked #1 on Image Generation on ImageNet 64x64 (FID metric)

https://arxiv.org/pdf/2105.05233v4.pdf

SciPy is an open-source scientific computing library for the Python programming language. Since its initial release in 2001, SciPy has become a de facto standard for leveraging scientific algorithms in Python, with over 600 unique code contributors, thousands of dependent packages, over 100,000 dependent repositories and millions of downloads per year. In this work, we provide an overview of the capabilities and development practices of SciPy 1.0 and highlight some recent technical developments. This Perspective describes the development and capabilities of SciPy 1.0, an open source scientific computing library for the Python programming language.

2020: Pauli Virtanen, R. Gommers, T. Oliphant, Matt Haberland, Tyler Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, Jonathan Bright, Stéfan J. van der Walt, M. Brett, Joshua Wilson, K. Millman, Nikolay Mayorov, Andrew R. J. Nelson, E. Jones, Robert Kern, Eric Larson, C. J. Carey, Ilhan Polat, Y. Feng, Eric W. Moore, J. Vanderplas, D. Laxalde, Josef Perktold, R. Cimrman, I. Henriksen, E. Quintero, Charles R. Harris, A. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, P. van Mulbregt, Aditya Alessandro Pietro Alex Andreas Andreas Anthony Ant Vijaykumar Bardelli Rothberg Hilboll Kloeckner Sco, A. Vijaykumar, Alessandro Pietro Bardelli, Alex Rothberg, A. Hilboll, Andre Kloeckner, A. Scopatz, Antony Lee, A. Rokem, C. N. Woods, Chad Fulton, C. Masson, C. Häggström, Clark Fitzgerald, David A. Nicholson, David R. Hagen, D. Pasechnik, E. Olivetti, E. Martin, E. Wieser, Fabrice Silva, F. Lenders, Florian Wilhelm, G. Young, Gavin A. Price, G. Ingold, Gregory E. Allen, Gregory R. Lee, H. Audren, I. Probst, J. Dietrich, J. Silterra, James T. Webber, J. Slavic, J. Nothman, J. Buchner, Johannes Kulick, Johannes L. Schönberger, J. V. de Miranda Cardoso, J. Reimer, J. Harrington, J. Rodríguez, Juan Nunez-Iglesias, Justin Kuczynski, K. Tritz, M. Thoma, M. Newville, Matthias Kümmerer, Maximilian Bolingbroke, Michael Tartre, M. Pak, Nathaniel J. Smith, N. Nowaczyk, Nikolay Shebanov, O. Pavlyk, P. A. Brodtkorb, Perry Lee, R. McGibbon, Roman Feldbauer, Sam Lewis, S. Tygier, Scott Sievert, S. Vigna, Stefan Peterson, S. More, Tadeusz Pudlik, T. Oshima, T. Pingel, T. Robitaille, Thomas Spura, T. Jones, T. Cera, Tim Leslie, Tiziano Zito, Tom Krauss, U. Upadhyay, Y. Halchenko, Y. Vázquez-Baeza

Keywords: SciPy, Python, Sparse matrix, Computational science, Algorithm, Linear algebra, Computational geometry, Interpolation, Signal processing, Image processing, Cluster analysis, Programming language, Library (computing), Machine learning, Input/output, Black Hole, Open-source software, File spanning, Computation

https://arxiv.org/pdf/1907.10121v1.pdf

Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language.

2011: Fabian Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, Mathieu Blondel, Gilles Louppe, P. Prettenhofer, Ron Weiss, Ron J. Weiss, J. Vanderplas, Alexandre Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay

Keywords: Machine learning, Documentation, Python, scikit-learn, Unsupervised learning, High-level programming language, High- and low-level, Usability, BSD, Binary file, Hypertext Transfer Protocol, Consistency model, Application programming interface, Algorithm, General-purpose markup language

https://arxiv.org/pdf/1201.0490v4.pdf

We present Brax, an open source library for rigid body simulation with a focus on performance and parallelism on accelerators. Brax reaches millions of simulation steps per second on environments like OpenAI Gym’s MuJoCo

2021: C. Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, Olivier Bachem

Keywords: Simulation

https://arxiv.org/pdf/2106.13281v1.pdf

We find that existing language modeling datasets contain many near-duplicate examples and long repetitive substrings. Deduplication allows us to train models that emit memorised text ten times less frequently and require fewer train steps to achieve the same or better accuracy. We release code for reproducing our work and performing dataset deduplication at https://github.com/google-research/ deduplicate-text-datasets.

2021: Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, D. Eck, Chris Callison-Burch, Nicholas Carlini

Keywords: NLP, Language

https://arxiv.org/pdf/2107.06499v1.pdf

Attention-based neural networks such as the Vision Transformer (ViT) have recently attained state-of-the-art results on many computer vision benchmarks. We successfully train a ViT model with two billion parameters, which attains a new state-of-the-art on ImageNet of 90.45% top-1 accuracy. The model also performs well on few-shot learning, for example, reaching 84.86% top-1 accuracy on ImageNet with only 10 examples per class.

2021: Xiaohua Zhai, Alexander Kolesnikov, N. Houlsby, L. Beyer

https://arxiv.org/pdf/2106.04560v1.pdf

Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. This package focuses on bringing machine learning to non-specialists using a general-purpose high-level language.

2011: Fabian Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, Mathieu Blondel, Gilles Louppe, P. Prettenhofer, Ron Weiss, Ron J. Weiss, J. Vanderplas, Alexandre Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay

Keywords: Machine learning, Documentation, Python, scikit-learn, Unsupervised learning, High-level programming language, High- and low-level, Usability, BSD, Binary file, Hypertext Transfer Protocol, Consistency model, Application programming interface, Algorithm, General-purpose markup language

https://arxiv.org/pdf/1201.0490v4.pdf

We propose NeuMIP, a neural method for representing and rendering a variety of material appearances at different scales. In this work, we generalize traditional mipmap pyramids to pyramids of neural textures, combined with a fully connected network. We also introduce neural offsets, a novel method which allows rendering materials with intricate parallax effects without any tessellation. Neural materials within our system support a 7-dimensional query, including position, incoming and outgoing direction, and the desired filter kernel size. The materials have small storage, and can be integrated within common Monte-Carlo path tracing systems.

2021: Alexandr Kuznetsov, Krishna Mullia, Zexiang Xu, Milos Hasan, R. Ramamoorthi

Keywords: Graphics, Machine Learning, Image and Video Processing

https://arxiv.org/pdf/2104.02789.pdf

TensorFlow enables developers to experiment with novel optimizations and training algorithms. TensorFlow supports a variety of applications, with a focus on training and inference on deep neural networks. Several Google services use TensorFlow in production, we have released it as an open-source project, and it has become widely used for machine learning research. In this paper, we describe the TensorFlow dataflow model and demonstrate the compelling performance that TensorFlow achieves for several real-world applications.

2016: Martín Abadi, P. Barham, Jianmin Chen, Z. Chen, Andy Davis, J. Dean, M. Devin, S. Ghemawat, Geoffrey Irving, M. Isard, M. Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, D. Murray, Benoit Steiner, P. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, Xiaoqian Zhang

TensorFlow, Machine learning, Dataflow, Algorithm, Reinforcement learning, Programming model, Data-flow analysis, Fault tolerance, Scheduling (computing), Memory management, Open-source software, Computation, Server (computing), Weak consistency, Distributed computing, Deep learning, Repository (version control), Experiment, Requirement, Mathematical optimization, User space, Multi-core processor, Scalability, Immutable object, Application-specific integrated circuit, Graphics processing unit, Central processing unit, Map, Artificial neural network, Processor register

https://arxiv.org/pdf/1605.08695v2.pdf

We propose StarGAN v2, a single framework that tackles both and shows significantly improved results over the baselines. Experiments on CelebA-HQ and a new animal faces dataset (AFHQ) validate our superiority in terms of visual quality, diversity, and scalability. To better assess image-to-image translation models, we release AFHQ, high-quality animal faces with large inter- and intra-domain differences.

2020: Yunjey Choi, Youngjung Uh, Jaejun Yoo, Jung-Woo Ha

Keywords: Scalability, Experiment

https://arxiv.org/pdf/1912.01865v2.pdf