Women gained seven grade points with Mainstay, making chatbot student performance impossible to ignore in AI college courses.

In this episode:

• Women in an undergraduate Microeconomics course saw their final grades increase by seven percentage points due to course-specific chatbot messages, demonstrating how AI improved grades.
• The Mainstay chatbot improved chatbot student performance by providing timely administrative nudges and support links, rather than generating academic content.
• Research at Georgia State University showed that chatbot student engagement was most effective for the middle 80 percent of students, offering vital scaffolding without necessarily building transferable study habits.
• Implementing AI in higher ed for course support requires significant human design, oversight, and existing infrastructure for it to be effective, as evidenced by the NBER working paper.
• To successfully adopt AI college courses, educators should first audit specific student friction points and measure real outcomes and staff workload, rather than focusing solely on claims of AI improved grades.

Chapters:

• 00:00 — Cold open & welcome
• 00:30 — Women gained seven points with chatbot student performance
• 01:00 — Mainstay chatbot: Nudging, not generating
• 01:45 — The invisible administration and chatbot student engagement
• 02:45 — Strongest effects for the middle 80 percent
• 03:45 — Gender effects and the Microeconomics course
• 04:30 — Performance vs. capability: No transfer of skills
• 05:30 — Implementation details for successful AI college courses
• 06:15 — Research credibility and limitations of the NBER paper
• 07:00 — Advice for leaders: Audit the problem first

How can teachers use AI marking safely?
The study suggests AI is best used for administrative nudges like reminders and connecting students to resources, rather than for academic decision-making or grading content directly.

What specific benefits did the Mainstay chatbot provide in AI college courses?
The Mainstay chatbot improved chatbot student performance by sending personalized text messages for deadlines, missing work alerts, encouragement, and links to tutoring, leading to higher grades and engagement, particularly for women.

Did AI improved grades for all students?
While students assigned to the chatbot were generally more likely to earn an A or B, the most significant gain was observed in women in the Microeconomics course, who earned grades seven points higher; men showed no comparable effect in this study.

Featuring: Dan Fitzpatrick, NBER, Georgia State University, Mainstay, Registry of Efficacy and Effectiveness Studies, National Institute for Student Success, Katharine E. Meyer, Lindsay C. Page, Catherine Mata.

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Randi Weingarten AI policy pairs an elementary ban with training 100,000 teachers as Los Angeles Unified School District cuts screens.

In this episode:

• Randi Weingarten, president of the American Federation of Teachers, proposes banning student-facing AI in elementary schools to protect young children's foundational thinking skills.
• The American Federation of Teachers plans to train approximately 100,000 teachers in AI this year through the National Academy for AI Instruction, prioritizing adult readiness.
• The episode highlights a critical distinction: AI in elementary schools should be carefully considered, distinguishing between tools that offload cognition and those that remove barriers for students.
• Developing AI skills for students means fostering critical thinking, context evaluation, and fact-checking, rather than just showing them how to type prompts.
• Effective AI in education policy needs to move beyond blanket bans, focusing instead on developmentally informed, tightly controlled applications and teacher involvement from the outset.

Chapters:

• 00:00 — Cold open & welcome
• 00:15 — Randi Weingarten AI policy: Ban for young, train for adults
• 00:45 — The concern: AI and cognitive debt in elementary schools
• 01:15 — Defining 'student-facing AI' and policy nuances
• 01:45 — Beyond bans: AI for accessibility vs. offloading cognition
• 02:15 — Essential AI skills for students in secondary education
• 02:45 — Teacher training as a key part of AI in education policy
• 03:15 — Unions' role in ethical AI deployment and professional development
• 03:45 — Balancing protection and progress in AI policy
• 04:00 — Conclusion: Protected thinking vs. challenged thinking with AI

What is Randi Weingarten's stance on AI in elementary schools?
Randi Weingarten advocates for banning student-facing AI tools in elementary schools to prevent children from offloading foundational thinking skills they need to develop.

How is the American Federation of Teachers addressing AI skills for teachers?
The American Federation of Teachers, led by Randi Weingarten, aims to train approximately 100,000 teachers this year through the National Academy for AI Instruction, using a peer-led model.

What challenges does AI in education policy face regarding student use?
AI in education policy must navigate the tension between preventing cognitive offloading in young learners and leveraging AI for accessibility or developing critical AI skills for older students.

Featuring: Dan Fitzpatrick, Randi Weingarten, American Federation of Teachers, National Press Club, Charter, ChatGPT, Los Angeles Unified School District, National Academy for AI Instruction, AFL-CIO.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Retention jumped from 67% to 83% in Ayisha Irfan's AI for student thinking pilot at Forging Opportunities for Refugees in America.

In this episode:

• An AI Socratic method increased learning retention from 67% to 83% for students at Forging Opportunities for Refugees in America (FORA) compared to AI summarization.
• Student confidence in using AI tools like Gemini or ChatGPT showed almost no relationship with their actual learning, emphasizing the need for robust AI literacy in schools.
• The real value of AI for student thinking lies in how students respond to questions, retrieve information, and explain concepts, not just in the AI's output.
• For professional development, educators should compare an AI's instant summary with a carefully designed questioning dialogue to understand its impact on delayed learning retention.
• Ensuring AI in refugee education or any diverse context means demanding accessibility as a foundational design element, including language support and suitable reading levels, not an optional setting.

Chapters:

• 00:00 — Cold open & welcome
• 00:15 — FORA pilot: AI for student thinking improves retention to 83%
• 00:45 — Ayisha Irfan's research at Forging Opportunities for Refugees in America
• 01:15 — Gemini's Socratic method vs. ChatGPT's direct answers
• 01:45 — Why productive struggle is key for AI learning retention
• 02:30 — Rethinking AI literacy in schools: beyond confidence and usage
• 03:15 — Addressing equity and accessibility for AI in refugee education
• 03:45 — Essential questions for school leaders on AI engagement and data privacy
• 04:30 — Measuring what matters: delayed learning retention over activity

How can teachers use AI marking safely?
Teachers should prioritize AI tools that prompt students to explain and retrieve information over those that simply summarize, and always ensure robust data governance regarding identifiable student information.

What is the best way to measure AI literacy in schools?
Measuring AI literacy should go beyond self-reported confidence or usage figures; instead, focus on students' ability to reason, explain, and retain information without the AI tool, and how they understand AI's influence on their thinking.

How can AI help students in refugee education?
AI in refugee education can narrow learning gaps if systems provide usable instructions, suitable reading levels, language support, and careful teacher oversight, ensuring accessibility is a foundation, not an afterthought.

Featuring: Dan Fitzpatrick, Gemini, ChatGPT, Forging Opportunities for Refugees in America, FORA, The 74, Augmented Intelligence Advisory, Ayisha Irfan, Google Privacy Policy.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

ChatGPT makes a silent character speak, while AI in literature education shows why Emile Bernard’s Madeleine in the Bois d'Amour defeats pattern matching.

In this episode:

• AI in literature education should focus on critical thinking AI, enabling students to directly confront the limitations of tools like ChatGPT in literary analysis.
• ChatGPT made specific errors when analyzing Carson McCullers' *The Heart is a Lonely Hunter*, including changing a bus to a train and giving dialogue to the silent character, John Singer.
• Southern Gothic AI analysis reveals that AI struggles with the nuanced contradictions and 'grotesque' elements in authors like Flannery O'Connor, highlighting AI limitations in humanities.
• Effective teaching AI literary analysis involves students annotating AI responses for textual support and then revising them, fostering a 'productive struggle' that deepens understanding.
• Rather than simply dismissing AI for lack of 'feelings,' educators should guide students to critique AI interpretations based on textual evidence, coherence, and insight.

Chapters:

• 00:00 — Cold open & welcome
• 00:30 — Exploring AI in literature education with ChatGPT
• 00:52 — ChatGPT’s errors in *The Heart is a Lonely Hunter*
• 01:15 — Why Southern Gothic AI analysis challenges pattern matching
• 01:52 — Critiquing AI interpretations: beyond 'no feelings'
• 02:30 — ChatGPT ignores John Singer’s silence
• 03:00 — Practical teaching AI literary analysis: annotation and revision
• 03:45 — Rethinking assessment: product, process, and live understanding
• 04:15 — Professional development for critical thinking AI
• 04:45 — AI in literature education makes close reading essential

How can teachers use AI in literature education without students misusing it?
Teachers can foster critical thinking AI by allowing students to directly identify AI limitations in literary analysis, such as ChatGPT's errors when analyzing *The Heart is a Lonely Hunter*.

What are the specific AI limitations in humanities subjects, especially literature?
AI's linguistic fluency can create the appearance of understanding without genuine insight, missing nuanced textual details, contradictions, and character specificities, as seen in Southern Gothic AI analysis.

How can teachers develop critical thinking skills using AI tools for literary analysis?
Teachers can have students annotate AI-generated literary analyses, identifying supported claims, areas needing more evidence, and overlooked details, then requiring them to rewrite sections with their own defended interpretations.

Featuring: Dan Fitzpatrick, Emile Bernard, Madeleine in the Bois d'Amour, Carson McCullers, The Heart is a Lonely Hunter, Flannery O'Connor, Spiros Antonapoulos, John Singer, ChatGPT.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Screen time is a poor proxy for learning. AI in education policy should judge thinking, access, ethics and student data.

In this episode:

• The United States Department of Education outlines five core principles for AI in education policy: technology must be educator-led, ethical, accessible, transparent, and protective of student data.
• Evaluating education technology guidelines means shifting focus from mere screen time in schools to the quality of student thinking and the learning outcomes produced.
• Responsible AI education emphasizes rigorous edtech procurement, requiring independent evaluations and a focus on evidence of impact, not just vendor popularity or brand recognition.
• Accessibility features like text-to-speech and captioning are critical equity components of effective edtech procurement, ensuring all students can access grade-level content.
• Effective AI in education policy balances evidence, professional judgment, and local context to ensure technology genuinely enhances learning rather than becoming an expensive, unproven addition.

Chapters:

• 00:00 — Cold open & welcome
• 00:25 — United States Department of Education's 5 principles for AI in education policy
• 01:00 — Why screen time in schools is a poor metric for learning
• 01:50 — Balancing duration with educational value in education technology guidelines
• 02:35 — The critical difference between passive consumption and active thinking with an AI chatbot
• 03:15 — Raising standards for edtech procurement: evidence and independent evaluation
• 04:15 — Leadership responsibility in implementing new education technology guidelines
• 05:05 — Equity and accessibility as a foundation for responsible AI education
• 06:00 — Balancing evidence, professional judgment, and local context in AI in education policy

What are the United States Department of Education's five principles for AI in education policy?
The five principles are that technology should be educator-led, ethical, accessible, transparent, and protective of student data.

How should schools evaluate education technology guidelines beyond just screen time in schools?
Schools should focus on the quality of student thinking, the learning outcomes produced, and the cognitive tasks students are engaging in, rather than simply measuring screen exposure.

What evidence should districts look for during edtech procurement to ensure responsible AI education?
Districts should seek independent evaluations, randomized controlled trials, and evidence that considers the specific conditions under which the technology proved effective, not just brand popularity or basic usage numbers.

Featuring: Dan Fitzpatrick, United States Department of Education, Elementary and Secondary Education Act, Every Student Succeeds Act, Apple Podcasts, Spotify, Google, AI chatbot, Linda McMahon.

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Nine neighbouring districts set different rules for the same technology, showing why AI policy school districts adopt must be clearer.

In this episode:

• Nine Central Florida school districts demonstrate varied AI policy school districts are adopting, from outright prohibition to specific allowances for tools like Google Gemini and ChatGPT.
• Student AI use policy must clearly define 'permission' to avoid six teachers setting six different boundaries for the same student, ensuring consistent instructional guidance.
• Orange County Public Schools and Brevard Public Schools correctly avoid relying on AI detection software as definitive proof of cheating, requiring supporting evidence like writing samples or student conversations.
• Effective AI guidelines for teachers should integrate AI tools for education into learning design, emphasizing human judgment and student cognitive engagement over simple machine production.
• Districts like Flagler Schools offering enterprise access to AI tools for education can provide stronger privacy controls, but policy language needs technical precision to avoid vague terminology.

Chapters:

• 00:00 — Cold open & welcome
• 00:30 — AI policy in Central Florida schools: Nine districts, different rules
• 01:00 — Foundational AI guidelines for teachers
• 01:30 — Variations in student AI use policy
• 02:45 — Why AI detection software isn't definitive proof of cheating
• 03:45 — Procurement and governance of AI tools for education
• 04:30 — The problem of access and equitable provision
• 05:15 — Beyond training: measuring impact and designing professional development
• 06:15 — Characteristics of good AI policy in school districts

How can teachers use AI marking safely?
Teachers should use AI tools for education with permission, protect sensitive student data, disclose AI involvement, check outputs for errors or bias, and rely on human judgment, especially for grading and high-stakes decisions.

What are common challenges for AI policy in school districts?
Challenges include varied student AI use policies across classrooms, over-reliance on AI detection software for cheating, and the need for technically precise language in procurement to ensure privacy and security with tools like Google Gemini and ChatGPT.

How can schools ensure equitable access to AI tools for education?
Schools must design AI provision to be accessible for all students, addressing needs related to homes, disabilities, and languages, rather than treating accessibility as an afterthought to purchasing decisions.

Featuring: Dan Fitzpatrick, Maria Salamanca, Orange County School Board, Brevard Public Schools, Katye Campbell, Flagler Schools, Don Foley, Google Gemini, ChatGPT.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

A school AI policy reversed an AI ban two days before term, leaving teachers to define supervised student AI use.

In this episode:

• The Shawnee Mission School Board's last-minute reversal of an AI ban, two days before term, created immediate uncertainty for educators defining student AI use.
• Effective district AI guidelines must clarify 'teacher-guided access' for students, distinguishing between productive struggle and outsourcing thinking to AI tools.
• The PICRAT framework is a useful tool for teachers to consider student engagement with AI, but it doesn't replace the need for clear school AI policy and operational guidance.
• Assessment strategies for teaching with AI should prioritize student process, explanation, and live performance over relying on AI detection software to gauge understanding.
• A credible school AI policy requires genuine community workgroups, cross-departmental collaboration, and funding for professional development to support consistent student AI use.

Chapters:

• 00:00 — Cold open & welcome
• 00:15 — Shawnee Mission School Board reverses AI ban two days pre-term
• 00:45 — Challenges of 'teacher-guided access' for student AI use
• 01:15 — Defining acceptable student AI use vs. cheating
• 01:45 — PICRAT framework for teaching with AI
• 02:15 — Superintendent Schumacher's call for balance in AI in schools
• 02:45 — Parent concerns and AI policy governance
• 03:15 — Community workgroup and measures of AI success
• 03:45 — Rethinking assessment in the age of AI
• 04:15 — Funding the reality of school AI policy

What are the immediate challenges when a school AI policy changes right before term starts?
When a school AI policy changes last-minute, teachers face significant challenges in interpreting new rules, preparing for classroom scenarios, and communicating effectively with families due to a lack of time and consistent district AI guidelines.

How can schools define 'teacher-guided access' for student AI use effectively?
Schools can define 'teacher-guided access' by clarifying what counts as direct supervision, specifying approved tools and contexts (e.g., brainstorming vs. drafting), and distinguishing between AI reducing friction and removing productive struggle for students.

How can educators adapt assessment when students are using AI?
Educators can adapt assessment by focusing on the student's process, live performance, and ability to explain decisions or defend sources, rather than relying solely on the final product or unreliable AI detection software.

Featuring: Dan Fitzpatrick, Shawnee Mission School Board, PICRAT, Dr. Mike Schumacher, Center for Academic Achievement, KCTV.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

73 percent of faculty faced AI integrity cases, making AI assessment redesign safer than relying on unreliable detectors.

In this episode:

• A striking 73 percent of faculty have already faced academic integrity issues related to AI, according to a national survey highlighted by Inside Higher Ed.
• Major AI detectors such as OpenAI, Writer, Copyleaks, GPTZero, and CrossPlag are widely unreliable, prone to false positives, and disproportionately flag non-native English writers, making AI detectors in education a risky strategy.
• Instead of an endless 'cat-and-mouse' game with detection, a better approach is AI assessment redesign, focusing on 'AI-resilient' assignments that make it harder to outsource critical thinking.
• The 'Three Ps model' (product, process, and performance) offers a practical framework for teaching with AI, enabling educators to gather richer evidence by observing how students interact with and transform AI output.
• Successful academic integrity AI strategies require systemic support, not just individual teacher efforts, prioritizing curriculum reform over the purchase of unreliable AI detection software.

Chapters:

• 00:00 — Cold open & welcome
• 00:30 — 73% of faculty face AI academic integrity cases
• 00:55 — The unreliability of AI detectors in education
• 01:30 — Why the 'cat-and-mouse' game with AI detection fails
• 01:55 — Moving to AI-resilient assignments and AI assessment redesign
• 02:25 — Context matters: Scaling AI-proofing assignments for large classes
• 03:00 — The Three Ps model: product, process, and performance in teaching with AI
• 03:45 — Systemic support for AI assessment redesign, not just individual effort
• 04:30 — Balancing 'protected' and 'supported' AI use moments
• 05:00 — Rethinking academic integrity AI: revealing minds, not catching machines

What percentage of faculty are dealing with AI academic integrity issues?
A national survey cited by Inside Higher Ed indicates that 73 percent of faculty have personally dealt with academic integrity issues involving AI.

Are AI detectors in education reliable for identifying AI-generated text?
No, studies show AI detectors from companies like OpenAI, Writer, Copyleaks, GPTZero, and CrossPlag are deeply unreliable, producing inconsistent results and falsely flagging human writing, especially from non-native English speakers.

How can teachers implement AI assessment redesign to make assignments more 'AI-resilient'?
Educators can implement AI assessment redesign by making tasks require visible processes, real-world application, and live performance, such as photographing local features for a geography project or challenging AI claims, embodying the 'Three Ps model' of product, process, and performance.

Featuring: Dan Fitzpatrick, Inside Higher Ed, Brown University, Alcorn State University, OpenAI, Writer, Copyleaks, GPTZero, CrossPlag.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Half of each HSC result comes from school-based work, putting the AI impact on assessment and authentic student work under scrutiny.

In this episode:

• The Minns Government is exploring an HSC AI policy, including a potential moratorium on unsupervised take-home assessments to address the AI impact on assessment in NSW schools.
• Deputy Premier Prue Car has tasked the NSW Education Standards Authority (NESA) with an urgent review into AI and student learning, with changes potentially impacting the Class of 2027.
• Educators must distinguish between AI use that bypasses student thinking and that which provokes it, as blanket policies may miss opportunities to foster authentic student work.
• Effective assessment redesign for the Higher School Certificate should consider the student's process, product, and live performance to create a more robust picture of learning and mitigate AI's influence.
• A fair common approach for identifying inappropriate AI use, as requested by NESA, should rely on human judgment and professional processes rather than unreliable automated detection tools.

Chapters:

• 00:00 — Cold open & welcome
• 00:20 — Minns Government & Prue Car's urgent NESA review of AI impact on assessment
• 00:45 — Proposed moratorium on take-home assessments for HSC AI policy
• 01:00 — Legitimate concerns: AI outsourcing thinking and cognitive debt
• 01:30 — Distinguishing harmful AI use from productive AI prompts for student learning
• 02:20 — Trade-offs and equity issues of a supervised assessment approach
• 03:15 — Long-term solutions: Assessment redesign for authentic student work
• 03:45 — NESA's role in a common approach for identifying AI use, avoiding AI detection tools
• 04:30 — Workload implications and professional development for AI in NSW schools
• 05:00 — Balancing speed and certainty in government policy for AI and student learning

What is the Minns Government's current HSC AI policy regarding take-home assessments?
The Minns Government is considering a moratorium on unsupervised take-home assessments for the Higher School Certificate while the NSW Education Standards Authority (NESA) conducts an urgent review into AI and student learning.

How can teachers identify authentic student work when students use AI?
Teachers can focus on assessment redesign that includes examining student process (drafts, planning), the final product, and live performance (oral defence) to create a richer picture of understanding, rather than solely relying on AI detection tools.

What are the equity concerns of moving all assessments into supervised settings in NSW schools?
While supervised settings may reduce disadvantages for students lacking home support, they could disadvantage students needing extra processing time, experiencing assessment anxiety, or requiring specific adjustments.

Featuring: Dan Fitzpatrick, NSW Education Standards Authority, NESA, Higher School Certificate, HSC, Prue Car, Minns Government.

Read the original source

Follow AI in Education with Dan Fitzpatrick for more on AI in education.

Homework rose 18% while exams fell 20%. The Generative AI Learning Penalty Evidence from Chinese Secondary Education tracked 26,811 students.

In this episode:

• A study of 26,811 students in China revealed an "AI learning penalty": an 18% rise in homework scores coincided with a 20% fall in closed-book exam performance.
• Students completing AI-assisted homework in under 50 minutes showed strong homework scores but extremely weak examination results, indicating a significant AI homework impact on AI student performance.
• High-attaining and younger students were more susceptible to the generative AI education learning penalty, with estimated Zhongkao and Gaokao results 5-7% lower overall for AI adopters.
• The research by David Strömberg, Victor Lei, and Yanhui Wu highlights that improved homework performance could actively conceal declining understanding and poor AI study habits.
• Effective generative AI education strategies must focus on tasks that reward critical thinking, like challenging AI responses, rather than simply fast completion, to avoid the learning penalty.

Chapters:

• 00:00 — Cold open & welcome
• 00:20 — Introducing the AI learning penalty research
• 00:50 — Homework productivity vs. actual student learning
• 01:25 — Strength of the study and contextual limitations
• 02:00 — Impact of homework time on AI student performance
• 02:50 — Rethinking AI use: from output to student interaction
• 03:30 — Measurement challenges for teachers and leaders
• 04:10 — Differential impact on subjects, attainment levels, and age groups
• 05:00 — Long-term consequences on Zhongkao and Gaokao results
• 05:40 — Designing for productive AI study habits

What is the AI learning penalty in education?
The AI learning penalty, observed in a study of Chinese secondary students, refers to the phenomenon where students using AI for homework see improved assignment scores but experience a decline in their closed-book examination performance due to outsourcing intellectual effort.

How does AI homework impact student performance on exams?
AI homework can negatively impact student performance on exams if students use AI to complete tasks quickly without engaging in the intellectual work, leading to strong homework scores but weak results on assessments like Zhongkao and Gaokao where AI is not permitted.

What are the long-term effects of generative AI on student learning?
The long-term effects of generative AI on student learning, according to one study, include significantly lower scores on major national examinations like Zhongkao and Gaokao, with losses estimated at 5-7% overall for AI adopters and reaching 24% and 18% respectively for students who used AI consistently for two years.

Featuring: Dan Fitzpatrick, The Generative AI Learning Penalty: Evidence from Chinese Secondary Education, David Strömberg, Victor Lei, Yanhui Wu, Zhongkao, Gaokao.

Follow AI in Education with Dan Fitzpatrick for more on AI in education.