Papers Read on AI

In this paper we introduce BEVFusion, an efficient and generic multi-task multi-sensor fusion framework. It unifies multimodal features in the shared bird’s-eye view (BEV) representation space, which nicely preserves both geometric and semantic information.

2022: Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, D. Rus, Song Han

Ranked #1 on 3D Object Detection on nuScenes

https://arxiv.org/pdf/2205.13542v1.pdf

Annotated datasets have become one of the most crucial preconditions for the development and evaluation of machine learning-based methods designed for the automated interpretation of remote sensing data. In this paper, we review the historic development of such datasets, discuss their features based on a few selected examples, and address open issues for future developments.

2021: M. Schmitt, S. A. Ahmadi, R. Hänsch

https://arxiv.org/pdf/2105.11726v2.pdf

We suggest “Hopular”, a novel Deep Learning architecture for medium and small sized datasets, where each layer is equipped with continuous modern Hopfield networks. The modern Hopfield networks use stored data to identify feature-feature, feature-target, and sample-sample dependencies. Hopular’s novelty is that every layer can directly access the original input as well as the whole training set via stored data in the Hopfield networks. Hopular outperforms XGBoost, CatBoost, LightGBM and a state-of-the art Deep Learning method designed for tabular data. Thus, Hopular is a strong alternative to these methods on tabular data.

2022: Bernhard Schafl, Lukas Gruber, Angela Bitto-Nemling, S. Hochreiter

Ranked #1 on General Classification on Shrutime

https://arxiv.org/pdf/2206.00664v1.pdf

We propose to use pretraining to boost general image-to-image translation. Prior image-to-image translation methods usually need dedicated architectural design and train individual translation models from scratch, struggling for high-quality generation of complex scenes, especially when paired training data are not abundant. In this paper, we regard each image-to-image translation problem as a downstream task and introduce a simple and generic framework that adapts a pretrained diffusion model to accommodate various kinds of image-to-image translation. We also propose adversarial training to enhance the texture synthesis in the diffusion model training, in conjunction with normalized guidance sampling to improve the generation quality.

2022: Tengfei Wang, Ting Zhang, Bo Zhang, Hao Ouyang, Dong Chen, Qifeng Chen, Fang Wen

https://arxiv.org/pdf/2205.12952v1.pdf

We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large transformer language models in understanding text and hinges on the strength of diffusion models in high-fidelity image generation. Our key discovery is that generic large language models (e.g. T5), pretrained on text-only corpora, are surprisingly effective at encoding text for image synthesis: increasing the size of the language model in Imagen boosts both sample fidelity and image-text alignment much more than increasing the size of the image diffusion model.

2022: Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. S. Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, D. Fleet, Mohammad Norouzi

Ranked #1 on Text-to-Image Generation on COCO (using extra training data)

https://arxiv.org/pdf/2205.11487v1.pdf

We propose a recipe on how to build a general, powerful, scalable (GPS) graph Transformer with linear complexity and state-of-the-art results on a diverse set of benchmarks. Graph Transformers (GTs) have gained popularity in the field of graph representation learning with a variety of recent publications but they lack a common foundation about what constitutes a good positional or structural encoding, and what differentiates them.

2022: Ladislav Rampášek, Mikhail Galkin, Vijay Prakash Dwivedi, A. Luu, Guy Wolf, D.Beaini

https://arxiv.org/pdf/2205.12454v1.pdf

Conditional Generative Adversarial Networks (cGAN) generate realistic images by incorporating class information into GAN. While one of the most popular cGANs is an auxiliary classifier GAN with softmax cross-entropy loss (ACGAN), it is widely known that training ACGAN is challenging as the number of classes in the dataset increases. ACGAN also tends to generate easily classifiable samples with a lack of diversity. In this paper, we introduce two cures for ACGAN. First, we identify that gradient exploding in the classifier can cause an undesirable collapse in early training, and projecting input vectors onto a unit hypersphere can resolve the problem. Second, we propose the Data-to-Data Cross-Entropy loss (D2D-CE) to exploit relational information in the class-labeled dataset.

2021: Minguk Kang, Woohyeon Shim, Minsu Cho, Jaesik Park

Ranked #1 on Conditional Image Generation on CIFAR-10

https://arxiv.org/pdf/2111.01118v1.pdf

In this work, we present a symbolic symphony music generation solution, SymphonyNet, based on a permutation invariant language model. To bridge the gap between text generation and symphony generation task, we propose a novel Multi-track Multi-instrument Repeatable (MMR) representation with particular 3-D positional embedding and a modified Byte Pair Encoding algorithm (Music BPE) for music tokens. A novel linear transformer decoder architecture is introduced as a backbone for modeling extra-long sequences of symphony tokens.

2022: Jiafeng Liu, Yuanliang Dong, Zehua Cheng, Xinran Zhang, Xiaobing Li, Feng Yu, Maosong Sun

Ranked #1 on Audio Generation on Symphony music

https://arxiv.org/pdf/2205.05448v1.pdf

In this paper, we propose an End-to-End framework for Flow-Guided Video Inpainting (E 2 FGVI) through elaborately designed three trainable modules, namely, flow completion, feature propagation, and content hallucination modules.

2022: Z. Li, Cheng Lu, Jia Qin, Chunle Guo, Mingg-Ming Cheng

Ranked #1 on Video Inpainting on YouTube-VOS 2018 val

https://arxiv.org/pdf/2204.02663v2.pdf

We introduce Ivy, a templated Deep Learning (DL) framework which abstracts existing DL frameworks such that their core functions all exhibit consistent call signatures, syntax and input-output behaviour. Ivy allows high-level framework-agnostic functions to be implemented through the use of framework templates.

2021: Daniel Lenton, Fabio Pardo, Fabian Falck, Stephen James, R. Clark

https://arxiv.org/pdf/2102.02886v3.pdf