This research paper assesses ChatGPT's capabilities as a teaching tool in computer science. The authors tested ChatGPT's responses to questions across three levels of difficulty: fundamental concepts, core competencies, and advanced topics. They found that ChatGPT's accuracy decreased significantly as the complexity of the questions increased, with notable limitations in generating high-quality code and accurately addressing advanced concepts like quantum computing. The study highlights both the potential benefits and significant risks of using ChatGPT in computer science education, emphasizing the need for critical evaluation by students and instructors. The paper also discusses related research and suggests teaching strategies to help students understand the limitations of such AI tools.

https://arxiv.org/pdf/2411.16690