Trust and Reliance on AI in Education: AI Literacy and Need for Cognition as Moderators
As generative AI systems continue to be integrated into educational settings, students are increasingly encountering AI-generated outputs while engaging with learning tasks. This interaction often occurs when students seek assistance or utilize integrated tools within their educational platforms. Understanding the trust students place in these AI systems is crucial, as it significantly influences how they interpret and utilize the information provided.
A recent study, detailed in the arXiv report (arXiv:2604.01114v1), explores the intricate relationship between students’ trust in AI and their reliance on AI-generated assistance during programming problem-solving tasks. The researchers focused on how this dynamic varies according to different learner characteristics, particularly AI literacy and need for cognition.
Research Overview
The study involved 432 undergraduate participants who were tasked with completing Python output-prediction problems. As they worked through these tasks, the students received recommendations and explanations from an AI chatbot. This included both accurate suggestions and intentionally misleading ones, allowing researchers to observe how trust affected reliance on the AI’s advice.
Key Findings
One of the primary objectives was to operationalize reliance in a behavioral context. This meant assessing how well students utilized the AI assistant’s suggestions—accepting them when correct and rejecting them when incorrect. To gather comprehensive data, pre- and post-task surveys were administered. These surveys measured various factors, including:
- Trust in the AI assistant
- AI literacy
- Need for cognition
- Programming self-efficacy
- Programming literacy
Implications of Trust
The results revealed a non-linear relationship between trust and appropriate reliance on AI assistance. Surprisingly, higher levels of trust were associated with lower appropriate reliance, indicating that students with greater trust were less discerning between correct and incorrect recommendations provided by the AI. This raises important questions about the implications of trust in educational technology and the potential for overreliance on AI systems.
Moderating Factors
Notably, the study found that this relationship was significantly moderated by students’ AI literacy and their need for cognition. Those with higher AI literacy exhibited a better ability to critically evaluate the AI’s suggestions, while individuals with a higher need for cognition engaged more deeply with the problem-solving process.
Conclusion and Future Directions
These findings underscore the necessity for future educational frameworks and AI system designs to promote a more reflective evaluation of AI assistance during problem-solving activities. As AI becomes an increasingly integral part of education, fostering critical thinking and discernment in students will be essential to maximize the benefits of these technologies while mitigating the risks of overreliance.