This episode explores the UIST 2025 paper "Creating General User Models from Computer Use," which proposes building a persistent user model from raw computer traces such as screenshots, UI text, message context, and app switching. It explains how the system stores confidence-weighted natural-language propositions about a person’s preferences, knowledge, goals, and current situation, aiming to support cross-application assistants that can help proactively rather than waiting for explicit requests. The discussion situates the idea against earlier recommender systems, Bayesian user modeling, and newer LLM memory architectures, arguing that the paper is most interesting as a synthesis of HCI user modeling and retrieval-and-revision style AI memory. Listeners would find it compelling because it gets concrete about both the upside of more context-aware assistants and the hard problems underneath them, including noisy behavioral data, narrow evidence relative to broad claims, and the privacy risks of inferring things users never said aloud.

Sources:

1. Creating General User Models from Computer Use — Omar Shaikh, Shardul Sapkota, Shan Rizvi, Eric Horvitz, Joon Sung Park, Diyi Yang, Michael S. Bernstein, 2025

http://arxiv.org/abs/2505.10831

2. User Modeling via Stereotypes — Elaine Rich, 1979

https://scholar.google.com/scholar?q=User+Modeling+via+Stereotypes

3. The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users — Eric J. Horvitz, John S. Breese, David Heckerman, David Hovel, Koos Rommelse, 1998

https://scholar.google.com/scholar?q=The+Lumiere+Project%3A+Bayesian+User+Modeling+for+Inferring+the+Goals+and+Needs+of+Software+Users

4. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions — Gediminas Adomavicius, Alexander Tuzhilin, 2005

https://scholar.google.com/scholar?q=Toward+the+Next+Generation+of+Recommender+Systems%3A+A+Survey+of+the+State-of-the-Art+and+Possible+Extensions

5. User Modeling and User Profiling: A Comprehensive Survey — Erasmo Purificato, Ludovico Boratto, Ernesto William De Luca, 2024

https://scholar.google.com/scholar?q=User+Modeling+and+User+Profiling%3A+A+Comprehensive+Survey

6. Generative Agents: Interactive Simulacra of Human Behavior — Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein, 2023

https://scholar.google.com/scholar?q=Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior

7. MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez, 2023

https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems

8. Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration — Yijia Shao, Vinay Samuel, Yucheng Jiang, John Yang, Diyi Yang, 2024

https://scholar.google.com/scholar?q=Collaborative+Gym%3A+A+Framework+for+Enabling+and+Evaluating+Human-Agent+Collaboration

9. Supporting Physical Activity Behavior Change with LLM-Based Conversational Agents — Matthew Jörke, Shardul Sapkota, Lyndsea Warkenthien, Niklas Vainio, Paul Schmiedmayer, Emma Brunskill, James Landay, 2024

https://scholar.google.com/scholar?q=Supporting+Physical+Activity+Behavior+Change+with+LLM-Based+Conversational+Agents

10. TaskTracer: a desktop environment to support multi-tasking knowledge workers — Anton N. Dragunov, Thomas G. Dietterich, Kevin Johnsrude, Matthew McLaughlin, Lida Li, Jonathan L. Herlocker, 2005

https://scholar.google.com/scholar?q=TaskTracer%3A+a+desktop+environment+to+support+multi-tasking+knowledge+workers

11. UI-TARS: Pioneering Automated GUI Interaction with Native Agents — Yujia Qin et al., 2025

https://arxiv.org/abs/2501.12326

12. Need Help? Designing Proactive AI Assistants for Programming — Valerie Chen et al., 2024/2025

https://arxiv.org/abs/2410.04596

13. Proactive Agent Research Environment: Simulating Active Users to Evaluate Proactive Assistants — Deepak Nathani et al., 2026

https://arxiv.org/abs/2604.00842

14. ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation — Jiho Kim et al., 2025/2026

https://arxiv.org/abs/2509.21730

15. AI Post Transformers: Learning Latent Action World Models from Video — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-09-learning-latent-action-world-models-from-1570a4.mp3

16. AI Post Transformers: PaperBench: Can AI Replicate AI Research? — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-17-paperbench-can-ai-replicate-ai-research-862944.mp3

17. AI Post Transformers: Reasoning Theater and Unfaithful Chain-of-Thought — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-05-reasoning-theater-and-unfaithful-chain-o-a4507e.mp3

18. AI Post Transformers: Neural Computers as Learned Latent Runtimes — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-11-neural-computers-as-learned-latent-runti-9fa282.mp3

Interactive Visualization: Building General User Models from Computer Use