Sign up to save your podcasts
Or
Share Shaping capabilities with token-level data filtering
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Shaping capabilities with token-level data filtering
In this episode:
• Welcome and the Post-hoc Problem: Linda introduces the paper and the hosts discuss why post-hoc unlearning methods fall short against adversarial attacks.
• Token vs. Document Filtering: An exploration of why token-level filtering acts as a scalpel compared to the blunt instrument of document filtering.
• Scaling Labels with SAEs: The hosts discuss how the authors use Sparse Autoencoders to label a subset of data and distill that into a highly efficient biLM classifier.
• Scaling Laws and Adversarial Robustness: Linda reveals the massive 7000x compute slowdown on the forget domain and how it compares to state-of-the-art unlearning.
• The Alignment Twist: A surprising finding that token-level filtering actually improves a model's ability to be trained to refuse dangerous prompts.
• Welcome and the Post-hoc Problem: Linda introduces the paper and the hosts discuss why post-hoc unlearning methods fall short against adversarial attacks.
• Token vs. Document Filtering: An exploration of why token-level filtering acts as a scalpel compared to the blunt instrument of document filtering.
• Scaling Labels with SAEs: The hosts discuss how the authors use Sparse Autoencoders to label a subset of data and distill that into a highly efficient biLM classifier.
• Scaling Laws and Adversarial Robustness: Linda reveals the massive 7000x compute slowdown on the forget domain and how it compares to state-of-the-art unlearning.
• The Alignment Twist: A surprising finding that token-level filtering actually improves a model's ability to be trained to refuse dangerous prompts.
View all episodes
By Mechanical DirkIn this episode:
• Welcome and the Post-hoc Problem: Linda introduces the paper and the hosts discuss why post-hoc unlearning methods fall short against adversarial attacks.
• Token vs. Document Filtering: An exploration of why token-level filtering acts as a scalpel compared to the blunt instrument of document filtering.
• Scaling Labels with SAEs: The hosts discuss how the authors use Sparse Autoencoders to label a subset of data and distill that into a highly efficient biLM classifier.
• Scaling Laws and Adversarial Robustness: Linda reveals the massive 7000x compute slowdown on the forget domain and how it compares to state-of-the-art unlearning.
• The Alignment Twist: A surprising finding that token-level filtering actually improves a model's ability to be trained to refuse dangerous prompts.
• Welcome and the Post-hoc Problem: Linda introduces the paper and the hosts discuss why post-hoc unlearning methods fall short against adversarial attacks.
• Token vs. Document Filtering: An exploration of why token-level filtering acts as a scalpel compared to the blunt instrument of document filtering.
• Scaling Labels with SAEs: The hosts discuss how the authors use Sparse Autoencoders to label a subset of data and distill that into a highly efficient biLM classifier.
• Scaling Laws and Adversarial Robustness: Linda reveals the massive 7000x compute slowdown on the forget domain and how it compares to state-of-the-art unlearning.
• The Alignment Twist: A surprising finding that token-level filtering actually improves a model's ability to be trained to refuse dangerous prompts.