CS224U

CS224U

By Chris PottsTechnology
Download on the App Store

CS224U episodes

  • Ellie Pavlick on true language understanding

    Grounding through pure language modeling objectives, the origins or probing, the nature of understanding, the future of system assessment, signs of meaningful progress in the field, and having faith in yourself.

    Transcript: https://web.stanford.edu/class/cs224u/podcast/pavlick/

    • Ellie's website
    • The LUNAR Lab
    • MIT Scientist Captures 90,000 Hours of Video of His Son’s First Words, Graphs It
    • Michael Frank
    • Spot robots
    • Dylan Ebert
    • Ian Tenney
    • What do you learn from context? Probing for sentence structure in contextualized word representations
    • BERT Rediscovers the Classical NLP Pipeline
    • JSALT: General-Purpose Sentence Representation Learning
    • Sam Bowman
    • Skip thought vectors
    • What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties
    • Hex
    • Charlie Lovering
    • Designing and interpreting probes with control tasks
    • Jerry Fodor
    • Been Kim
    • Mycal Tucker
    • What if this modified that? Syntactic interventions via counterfactual embeddings
    • Yonatan Belinkov
    • HANS: Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
    • Conceptual pacts and lexical choice in conversation
    • Locating and editing factual knowledge in GPT
    • Could a purely self-supervised language model achieve grounded language understanding?
    • Dartmouth Summer Research Project on Artificial Intelligence (1956)
    • Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain
      • 1 hr 24 min
    • Richard Socher on conviction in research

      The early days of the rise of deep learning in NLP, conviction, the importance of applied work in the current moment, start-up risks, the state of Web search, paramotoring, and over-looked gems in the U.S. National Park system.

      Transcript: https://web.stanford.edu/class/cs224u/podcast/socher/

      • Richard's website
      • Richard's Twitter
      • Richard's paramotoring
      • Salesforce Research
      • you.com
      • @YouSearchEngine
      • Nate Chambers
      • Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems
      • Parsing Natural Scenes and Natural Language with Recursive Neural Networks
      • CS224n
      • Collobert and Weston 2008: A unified architecture for natural language processing: deep neural networks with multitask learning
      • Stephen Merity
      • CoVe: Learned in Translation: Contextualized Word Vectors
      • https://believermag.com/ghosts/
      • Frances Arnold
      • ProGen: Language Modeling for Protein Generation
      • Lav Varshney
      • decaNLP
      • Chris Ré
      • Loebner Prize
      • Eugene Goostman
      • Richard's castle on airbnb
      • American Samoa National Park
      • Great Sand Dunes National Park
      • White Sands National Park
      • Zion National Park
        • 1 hr 35 min
      • Omar Khattab on neural information retrieval

        Pronouncing "ColBERT", the origins of ColBERT, doing NLP from an IR perspective, how getting "scooped" can be productive, OpenQA and related tasks, PhD journeys, why even retrieval plus attention is not all you need, multilingual knowledge-intensive NLP, and aiming high in research projects.

        Transcript: https://web.stanford.edu/class/cs224u/podcast/khattab/

        • Omar's website
        • Matei Zaharia
        • Keshav Santhanam
        • Steven Colbert thowing paper with Obama
        • The ColBERT paper and the ColBERTv2 paper
        • DeepImpact: Learning passage impacts for inverted indexes
        • DPR: Dense passage retrieval for open-domain question answering
        • Incorporating query term independence assumption for efficient retrieval and ranking using deep neural networks
        • DeepCT: Context-aware sentence/passage term importance estimation for first stage retrieval
        • Reading Wikipedia to answer open-domain questions
        • ORQA: Latent retrieval for weakly supervised open domain question answering
        • QRECC
        • ColBERT-QA: Relevance-guided Supervision for OpenQA with ColBERT
        • Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval
        • Passage reranking with BERT
        • UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering
        • Self-driving search engines: The neural hype and comparisons against weak baselines
        • Mohammad Hammoud
        • RAG: Retrieval-augmented generation for knowledge-intensive NLP tasks
        • Hindsight: Posterior-guided training of retrievers for improved open-ended generation
        • Learning Cross-Lingual IR from an English Retriever
        • Blog post: A moderate proposal for radically better AI-powered Web search
        • Blog post: Building scalable, explainable, and adaptive NLP models with retrieval
        • XOR-TyDi
          • 1 hr 26 min
        • Adina Williams on deep linguistic analysis in NLP

          Neuroscience and neural networks, being a linguist in the world of NLP, evaluation methods, fine-grained NLI questions, the pace of research, and the vexing fact that, on the internet, people = men.

          Transcript: https://web.stanford.edu/class/cs224u/podcast/williams/

        • Adina's website
        • Adina on Twitter
        • Based on billions of words on the internet, people = men
        • April Bailey
        • Andrei Cimpian
        • Androcentrism
        • GloVe
        • fastText
        • Preregistration
        • P-hacking
        • Rishi Bommasani
        • Interpreting pretrained contextualized representations via reductions to static embeddings
        • Common Crawl
        • Battlestar Galactica
        • MultiNLI
        • ANLI
        • DynaBench
        • Breaching experiment
        • Liina Pylkkänen
        • Sam Bowman
        • Nikita Nangia
        • Ludwig Wittgenstein
        • Ido Dagan
        • Sebastian Riedel
        • SNLI
        • Yixin Nie
        • Douwe Kiela
        • Jason Weston
        • Emily Dinan
        • Build it break it fix it for dialogue safety: Robustness from adversarial human attack
        • Allyson Ettinger
        • GLUE
        • DynaSent
        • DeBERTa
        • RoBERTa
        • Breaking NLI systems with sentences that require simple lexical inferences
        • HANS: Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference
        • Targeted syntactic evaluation of language models
        • Max Bartolo
        • Magnetoencephalography
        • Marr's Levels
        • Marco Baroni
        • Richard Futrell
        • Ryan Cotterell
        • SIGMORPHON 2022
        • Alexis Conneau
        • FLORES
        • XNLI
        • OCNLI: Original Chinese natural language inference
        • Yann LeCun
        • Novel Ideas in Learning-to-Learn through Interaction
        • Grounding semantics in olfactory perception
        • Brain in a vat
        • Ellie Pavlick
        • Tom Kwiatkowski
        • Mohit Bansal
        • Identifying inherent disagreement in natural language inference
        • DALL-E 2
        • Winoground
        • Gary Marcus on Winoground
          • 1 hr 28 min
        • Douwe Kiela on research at Hugging Face

          Hugging Face, multimodality, data and model auditing, ethics review, adversarial testing, attention as more and less than you ever needed, neural information retrieval, philosophy of mind and consciousness, augmenting human creativity, openness in science, and a defininitive guide to pronouncing Douwe.

          Transcript: https://web.stanford.edu/class/cs224u/podcast/kiela/

          • Douwe's website
          • Hugging Face
          • Grounding semantics in olfactory perception
          • Model Cards for Model reporting
          • Datasheets for datasets
          • Dynabench
          • Hugging Face Spaces
          • http://www.isattentionallyouneed.com
          • The Annotated S4
          • Retrieval-Augmented Generation for knowledge-intensive NLP tasks
          • Language models as slightly consciousness
          • Fields of wheat as slightly pasta
          • True few-shot learning with language models
          • https://believermag.com/ghosts/
          • I Am A Strange Loop
          • AI Dungeon
          • LIGHT
          • Good first issue
            • 1 hr 22 min
          • Rishi Bommasani on Foundation Models

            Deriving static representations from contextual ones, interdisciplinary research, training large models, the Foundation Models paper and CRFM, being an academic on Twitter, and progress in NLP.

            Transcript: https://web.stanford.edu/class/cs224u/podcast/bommasani/

            • Rishi's website
            • Rishi on Twitter
            • Bommasani et al 2020
            • On the opportunities and risks of foundation models
            • Reflections on foundation models
            • EleutherAI
            • Chinchilla
            • http://www.isattentionallyouneed.com
            • Rishi on The Gradient podcast
              • 1 hr 30 min

            About CS224U

            From the publisher's feed

            Conversations about Natural Language Processing