Generally Intelligent

Dylan Hadfield-Menell (Google Scholar) (Website) recently finished his PhD at UC Berkeley and is starting as an assistant professor at MIT. He works on the problem of designing AI algorithms that pursue the intended goal of their users, designers, and society in general.  This is known as the value alignment problem.

Highlights from our conversation:

👨‍👩‍👧‍👦 How to align AI to human values

📉 Consequences of misaligned AI -> bias & misdirected optimization

📱 Better AI recommender systems

Drew Linsley (Google Scholar) is a Paul J. Salem senior research associate at Brown, advised by Thomas Serre. He is working on building computational models of the visual system that serve the dual purpose of (1) explaining biological function and (2) extending artificial vision.

Highlights from our conversation:

🧠 Building neural-inspired inductive biases into computer vision

🖼 A learning algorithm to improve recurrent vision models (C-RBP)

🤖 Creating new benchmarks to move towards generalization

Giancarlo Kerg (Google Scholar) is a PhD student at Mila, supervised by Yoshua Bengio and Guillaume Lajoie.  He is working on out-of-distribution generalization and modularity in memory-augmented neural networks. 

Highlights from our conversation:

🧮 Pure math foundations as an approach to progress and structural understanding in deep learning research

🧠 How a formal proof on the way self-attention mitigates gradient vanishing when capturing long-term dependencies in RNNs led to a relevancy screening mechanism resembling human memory consolidation

🎯 Out-of-distribution generalization through modularity and inductive biases

Yujia Huang (Website) is a PhD student at Caltech, working at the intersection of deep learning and neuroscience.  She worked on optics and biophotonics before venturing into machine learning. Now, she hopes to design “less artificial” artificial intelligence.

Highlights from our conversation:

🏗 How recurrent generative feedback, a neuro-inspired design, improves adversarial robustness and and can be more efficient (less labels)

🧠 Adapting theories from neuroscience and classical research for machine learning

📊 What a new Turing test for “less artificial” or generalized AI could look like

💡 Tips for new machine learning researchers!

Julian Chibane (Google Scholar) is a PhD student at the Real Virtual Humans group at the Max Planck Institute for Informatics in Germany.  His recent work centers around intrinsic functions for 3D reconstruction.

Highlights from our conversation:

🖼 How, surprisingly, the IF-Net architecture learned reasonable representations of humans & objects without priors

🔢 A simple observation that led to Neural Unsigned Distance Fields, which handle 3D scenes without a clear inside vs. outside (most scenes!)

📚 Navigating open questions in 3D representation, and the importance of focusing on what's working

Katja Schwartz came to machine learning from physics, and is now working on 3D geometric scene understanding at the Max Planck Institute for Intelligent Systems. Her most recent work, “Generative Radiance Fields for 3D-Aware Image Synthesis,” revealed that radiance fields are a powerful representation for generative image synthesis, leading to 3D consistent models that render with high fidelity.

We discuss the ideas in Katja’s work and more:

🥦 the role 3D generation plays in conceptual understanding

📝 tons of practical tips on GAN training

〰 continuous functions as representations for 3D objects

Joel Lehman was previously a founding member at Uber AI Labs and assistant professor at the IT University of Copenhagen. He's now a research scientist at OpenAI, where he focuses on open-endedness, reinforcement learning, and AI safety.

Joel’s PhD dissertation introduced the novelty search algorithm. That work inspired him to write the popular science book, “Why Greatness Cannot Be Planned”, with his PhD advisor Ken Stanley, which discusses what evolutionary algorithms imply for how individuals and society should think about objectives.

We discuss this and much more:

- How discovering novelty search totally changed Joel’s philosophy of life

- Sometimes, can you reach your objective more quickly by not trying to reach it?

- How one might evolve intelligence

- Why reinforcement learning is a natural framework for open-endedness

Cinjon Resnick was formerly from Google Brain and now is doing his PhD at NYU. We talk about why he believes scene understanding is critical to out of distribution generalization, and how his theses have evolved since he started his PhD.

Some topics we over:

• How Cinjon started his research by trying to grow a baby through language and games, before running into a wall with this approach
• How spending time at circuses 🎪 and with gymnasts 🤸🏽‍♂️ re-invigorated his research, and convinced him to focus on video, motion, and activity recognition
• Why MetaSIM and MetaSIM II are underrated papers
• Two research ideas Cinjon would like to see others work on

Sarah Jane Hong is the co-founder of Latent Space, a startup building the first fully AI-rendered 3D engine in order to democratize creativity.

We touch on what it was like taking classes under Geoff Hinton in 2013, the trouble with using natural language prompts to render a scene, why a model’s ability to scale is more important than getting state-of-the-art results, and more.

We interview Kelvin Guu, a researcher at Google AI and the creator of REALM. 

The conversation is a wide-ranging tour of language models, how computers interact with world knowledge, and much more.