UC Irvine study shows AI tools like ChatGPT can affect student learning
A study from the University of California, Irvine, shows that access to AI tools like ChatGPT may affect how much students learn.
The researchers worked with McGraw Hill, an education company, to study 3.2 million math questions that students had answered on the company's tests. The questions were fairly simple math problems that AI can solve easily.
UC Irvine researcher Sina Rismanchian, the study's lead author, explained, "We're trying to see if a student's learning behaviors changed after the introduction of ChatGPT. As you know, these models are very good at solving problems at a high school level."
The researchers studied 10 years of test results, from 2015 to 2025. They compared the years before ChatGPT became available with the years after its release. They found that students spent much less time solving these problems once they had access to AI. The results were especially striking for high school students, who took 30% less time to finish the test than students who had taken it before AI was available to them.
This may sound like a real benefit, but there was a problem. The tests were not supervised. When the researchers compared these results with supervised tests, where students could not use AI, they saw an effect on learning. In supervised tests, the data showed that students started getting many more questions wrong in the year ChatGPT became available. This suggested that students who had used AI to solve math problems faster had not actually learned how to solve those problems on their own.
Rismanchian said, "It's been kind of established in learning theories that the students need to struggle with the problems and the cognitive process of solving problems to learn them." He added, "It means that students are spending less time on this struggle here, and then at the end, it comes at the cost of their durable learning outcomes in the test."
Despite these findings, Rismanchian noted that AI can still help students learn when it is used in the right way. For example, AI tools can give students feedback, such as by reviewing essays that students have written themselves and suggesting ways to improve them.
"So there are ways to include AI in students' learning processes to augment that process, but it depends on the way that the students are using these tools," he said.
The study highlights why AI should be used to support learning rather than replace the process of solving problems.