Uh-Oh! Anthropic’s Report Suggests Students May Be Using Claude to Cheat

On Tuesday, Anthropic released an education report on how students use Claude. The study attempted to analyse real-world AI usage patterns in higher education. The report included one million anonymised student conversations on Claude.ai. 

To protect user privacy, the company used Claude Insights and Observations (Clio), an automated analysis tool, to get AI usage patterns by breaking down user conversations into high-level usage summaries. The tool eliminates private user information from the conversation to process the analysis.

According to the report, 39.3% of conversations involved students primarily using Claude to generate and refine educational content across various disciplines. About 33.5% of conversations involved technical explanations or solutions for academic assignments, whether to debug/fix errors or solve mathematical problems. 

The rest of the conversations comprised 11% for analysing and visualising data, 6.5% for supporting research design and tool development, 3.2% for creating technical diagrams, and 2.4% for translating or proofreading content.

Most of the usage was found in the computer science discipline, representing 38.6% of conversations with others having a sizeable representation, as shown in the image below.

“Some of this usage might also be cheating,” stated the report. It continued, “How much are students using AI to cheat?” That’s hard to answer, especially as we don’t know the specific educational context where each of Claude’s responses is being used.”

The company shared examples like a direct problem-solving conversation could be for cheating on a take-home exam, or a student checking their work on a practice test. The report also categorised the types of conversations as direct problem solving, direct output creation, collaborative problem solving, and collaborative output creation.

Nearly half (47%) of conversations were direct, which means the students sought answers with minimal engagement. This raises important questions about academic integrity, developing critical thinking skills, and assessing students’ learning.