The Thesis Podcast

The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers

Doi：10.1145/3706598.3713778

Abstract：

The rise of Generative AI (GenAI) in knowledge workflows raises questions about its impact on critical thinking skills and practices. We survey 319 knowledge workers to investigate 1) when andhow they perceive the enaction of critical thinking when using GenAI, and 2) when and why GenAI affects their effort to do so. Participants shared 936 first-hand examples of using GenAI in worktasks. Quantitatively, when considering both task- and user-specific factors, a user’s task-specific self-confidence and confidence in GenAI are predictive of whether critical thinking is enacted andthe effort of doing so in GenAI-assisted tasks. Specifically, higher confidence in GenAI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking. Qualitatively, GenAI shifts the nature of critical thinking toward information verification, response integration, and task stewardship. Our insights reveal new design challenges and opportunities fordeveloping GenAI tools for knowledge work.

生成式 AI 對批判性思維的影響：知識工作者調查中自述的認知努力降低與信心效應

論文原著：The Impact of Generative AI on Critical Thinking: Self-ReportedReductions in Cognitive Effort and Confidence Effects From aSurvey of Knowledge Workers

Doi：10.1145/3706598.3713778

摘要：

隨著生成式人工智慧（Generative AI, 簡稱 GenAI）在知識型工作流程中的興起，人們開始關注其對批判性思考能力與實踐方式的影響。本研究調查了 319 位知識工作者，以了解：

1. 他們在使用 GenAI 時何時、如何展現批判性思考；

2. GenAI 在何時、為何會影響他們進行批判性思考的努力。

參與者共分享了 936 個第一手工作情境中的 GenAI 使用實例。

量化分析方面：

當同時考慮任務與使用者因素時，受試者對於特定任務的自信心與對 GenAI 的信任程度，是預測是否會實施批判性思考及其努力程度的關鍵因素。

• 對 GenAI 的信心越高，越不容易產生批判性思考；

• 對自己的自信心越高，則越容易展現批判性思考。

質性分析方面：

GenAI 改變了批判性思考的實踐方式，使用者更傾向於：

• 驗證資訊

• 整合回應

• 掌握任務方向

本研究提供的洞察揭示了在設計用於知識工作的 GenAI 工具時，新的挑戰與機會。

Origin: Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task

Doi：10.48550/arXiv.2506.08872

Abstract：With today's wide adoption of LLM products like ChatGPT from OpenAI, humans and

businesses engage and use LLMs on a daily basis. Like any other tool, it carries its own set of

advantages and limitations. This study focuses on finding out the cognitive cost of using an LLM

in the educational context of writing an essay.

We assigned participants to three groups: LLM group, Search Engine group, Brain-only group,

where each participant used a designated tool (or no tool in the latter) to write an essay. We

conducted 3 sessions with the same group assignment for each participant. In the 4th session

we asked LLM group participants to use no tools (we refer to them as LLM-to-Brain), and the

Brain-only group participants were asked to use LLM (Brain-to-LLM). We recruited a total of 54

participants for Sessions 1, 2, 3, and 18 participants among them completed session 4.

We used electroencephalography (EEG) to record participants' brain activity in order to assess

their cognitive engagement and cognitive load, and to gain a deeper understanding of neural

activations during the essay writing task. We performed NLP analysis, and we interviewed each

participant after each session. We performed scoring with the help from the human teachers

and an AI judge (a specially built AI agent).

We discovered a consistent homogeneity across the Named Entities Recognition (NERs),

n-grams, ontology of topics within each group. EEG analysis presented robust evidence that

LLM, Search Engine and Brain-only groups had significantly different neural connectivity

patterns, reflecting divergent cognitive strategies. Brain connectivity systematically scaled down

with the amount of external support: the Brain‑only group exhibited the strongest, widest‑ranging

networks, Search Engine group showed intermediate engagement, and LLM assistance elicited

the weakest overall coupling. In session 4, LLM-to-Brain participants showed weaker neural

connectivity and under-engagement of alpha and beta networks; and the Brain-to-LLM

participants demonstrated higher memory recall, and re‑engagement of widespread

occipito-parietal and prefrontal nodes, likely supporting the visual processing, similar to the one

frequently perceived in the Search Engine group. The reported ownership of LLM group's

essays in the interviews was low. The Search Engine group had strong ownership, but lesser

than the Brain-only group. The LLM group also fell behind in their ability to quote from the

essays they wrote just minutes prior.

As the educational impact of LLM use only begins to settle with the general population, in this

study we demonstrate the pressing matter of a likely decrease in learning skills based on the

results of our study. The use of LLM had a measurable impact on participants, and while the

benefits were initially apparent, as we demonstrated over the course of 4 months, the LLM

group's participants performed worse than their counterparts in the Brain-only group at all levels:

neural, linguistic, scoring.

We hope this study serves as a preliminary guide to understanding the cognitive and practical

impacts of AI on learning environments.

你的大腦與 ChatGPT：使用 AI 助理撰寫文章任務時的認知負債累積

論文原著：Your Brain on ChatGPT Accumulation of Cognitive Debt when Using an Al Assistant for Essay Writing Task

Doi：10.48550/arXiv.2506.08872

摘要：

隨著像 OpenAI 的 ChatGPT 等大型語言模型（LLM）廣泛被人類與企業日常使用，本研究聚焦於探討在教育情境中使用 LLM 撰寫文章所帶來的認知代價。

研究設計將參與者分為三組：

• LLM 組（使用大型語言模型）、

• 搜尋引擎組（使用搜尋工具）、

• 大腦組（不使用任何工具）。

前三次實驗中，參與者使用各自指定的工具進行寫作；第四次實驗中，LLM 組改為不使用工具（LLM-to-Brain），大腦組則改為使用 LLM（Brain-to-LLM）。共 54 人參與前三輪實驗，其中 18 人完成第四輪。

研究過程中使用腦電波（EEG）監測參與者的腦部活動，以評估他們的認知投入與負荷，並結合自然語言處理（NLP）分析、訪談、人工與 AI 評分。

研究發現包括：

• 各組產出的文章在命名實體、詞組與主題分類上呈現明顯的組內一致性。

• 腦波分析顯示，不同組別在神經連結模式上有顯著差異，反映出不同的認知策略：

  • 大腦組：神經網路最強、範圍最廣；

  • 搜尋引擎組：中等程度；

  • LLM 組：最弱的神經連結與參與度。

• 在第 4 次實驗中，LLM-to-Brain 組出現較弱的腦波連結與 alpha、beta 波段參與度降低；

  Brain-to-LLM 組則表現出更高的記憶召回力與視覺處理區的重新活化，類似搜尋引擎組的特徵。

• 訪談中發現 LLM 組對自己寫的文章的擁有感（ownership）最低，搜尋引擎組次之，大腦組最高。LLM 組也無法準確引用他們剛寫過的段落。

結論指出：雖然 LLM 在初期提供了明顯的便利，但長期觀察顯示，LLM 組在語言表現、腦部活動與寫作評分上都顯著落後於大腦組。

本研究提醒教育界，LLM 的使用可能對學習技能造成影響，也希望作為探索 AI 對學習環境影響的初步指南。

SEO Guide for the Al Era

AI 時代搜尋引擎優化指南 SEO Guide for the AI Era