In this episode we explore new research on bias in AI writing feedback and what it means for teachers, school leaders, and anyone evaluating AI-powered writing tools in K–12 education. We look at a Stanford preprint on how large language models responded differently to the same student essay when only the attached demographic profile changed, and we connect that to earlier MIT research showing that AI systems can shift tone and quality based on who they think they are talking to. The bigger question: when AI claims to “personalize” feedback, is it actually supporting equity, or quietly automating lower expectations?

Topics covered:

• How a Stanford research team tested AI writing feedback using 600 real eighth-grade persuasive essays
• Why changing only demographic labels changed the feedback students received
• What “positive feedback bias” and “feedback withholding bias” can look like in classroom practice
• How AI can give more praise but less useful critique to some students
• What the earlier MIT chatbot study revealed about tone, condescension, and perceived vulnerability
• Why AI “personalization” can slip into profiling
• What teachers should ask before trusting AI-generated writing feedback
• How students can be taught to question, audit, and respond critically to AI feedback
• What curriculum leaders and district leaders should demand from vendors about inputs, transparency, and equity testing

Source:

https://arxiv.org/pdf/2603.12471