Journal Club

George dives into his blog post experimenting with Scott Lundberg's SHAP library. By training an XGBoost model on a dataset about academic attainment and alcohol consumption can we develop a global interpretation of the underlying relationships? Lan leads the discussion of the paper Adversarial Examples Are Not Bugs, They Are Features by Ilyas and colleagues. This papers proposes a new perspective on adversarial susceptibility of machine learning models by teasing apart the 'robust' and the 'non-robust' features in a dataset. The authors summarizes the key take away message as "Adversarial vulnerability is a direct result of the models’ sensitivity to well-generalizing, ‘non-robust’ features in the data." Last  but not least, Kyle discusses Alphafold!

Today on the show, George leads a discussion about the Giant Language Test Room.  Lan presents a news item about Setting Fairness Goals with TensorFlow Constrained Optimization Library. This library lets users configure and train machine learning problems based on multiple different metrics, making it easy to formulate and solve many problems of interest to the fairness community. Last but not least, Kyle discusses ML Unfairness, Juvenile Recidivism in Catalonia.

Today on the show, Lan presents a blog post revealing the Dark secrets of BERT. This work uses telling visualizations of self-attention patterns before and after fine-tuning to probe: what happens in the fine-tuned BERT?  George brings a novel technique to the show, "radioactive data" - a marriage of data and steganography. This work from Facebook AI Research gives us the ability to know exactly who's been training models on our data. Last but not least, Kyle discusses the work "Learning Important Features Through Propagating Activation Differences."

Today on the show, Lan presents a blog post from Google Deepmind about Dopamine and temporal difference learning. This is the story of a fruitful collaboration between Neuroscience and AI researchers that found the activity of dopamine neurons in the mouse ventral tegmental area during a learnt probabilistic reward task was consistent with distributional temporal-difference reinforcement learning. That's a mouthful, go read it yourself! George presents his first attempts at designing an Auto-Trading Agent with Deep Q Networks. Last but not least, Kyle says "Hey Alexa! Sorry I fooled you ..."

George led a discussion about AlphaGo - The Movie | Full Documentary.

Lan informed us about the COVID-19 Open Research Dataset.

Kyle shared some thoughts about the paper Beyond R_0: the importance of contact tracing when predicting epidemics.

George discusses Google's Dataset Search leaving its closed beta program, and what potential applications it will have for businesses, scholars, and hobbyists.

Alex brings an article about Activation Atlases and we discusses the applicability to machine learning interpretability.

Lan leads a discussion about the paper Attention is not Explanation from Sarthak Jain and Byron C. Wallace. It explores the relationship between attention weights and feature importance scores (spoilers in the title).

Kyle shamelessly promotes his blog post using LIME to explain a simple prediction model trained on Wikipedia data.

Kyle discusses Google's recent open sourcing of ALBERT, a variant of the famous BERT model for natural language processing. ALBERT is more compact and uses fewer parameters.  George leads a discussion about the paper Explainable Artificial Intelligence: Understanding, visualizing, and interpreting deep learning models by Samek, Wiegand, and Muller. This work introduces two tools for generating local interpretability and a novel metric to objectively compare the quality of explanations. Last but not least, Lan talks about her experience generating new Seinfeld scripts using GPT-2.

Welcome to a brand new show from Data Skeptic entitled "Journal Club".

Each episode will feature a regular panel and one revolving guest seat. The group will discuss a few topics related to data science and focus on one featured scholarly paper which is discussed in detail.

Lan tells the story of a transformer learning to play chess. The experiment was to fine-tune a GPT-2 transformer model using a 2.4M corpus of chess games in standard notation, then to see if it can 'play chess' by generating the next move. This is a thought-provoking way to take advantage of the advances in NLP by 'transforming' a game into the 'language' of written text. This was work done by Shawn Presser.

George gives a breakdown of a Kaggle Cheating Scandal where a Grandmaster was caught training on the test set. The story follows Benjamin Minixhofer and his capable detective work to discover an obfuscation that artificially improved the winning team's accuracy.

Kyle leads a discussion on the paper Towards A Rigorous Science of Interpretable Machine Learning from Finale Doshi-Velez and Been Kim. The paper is a great survey of the spectrum of interpretability techniques and also contains suggestions for how we describe the "taxonomy" of various methodologies.