REFINE: Real-world Exploration of Interactive Feedback and Student Behaviour
Summary: arXiv:2603.29142v1 Announce Type: new
Abstract
Formative feedback is central to effective learning, yet providing timely, individualised feedback at scale remains a persistent challenge. While recent work has explored the use of large language models (LLMs) to automate feedback, most existing systems still conceptualise feedback as a static, one-way artifact, offering limited support for interpretation, clarification, or follow-up.
Introduction to REFINE
In this work, we introduce REFINE, a locally deployable, multi-agent feedback system built on small, open-source LLMs that treats feedback as an interactive process. REFINE combines several innovative components:
- Pedagogically-grounded feedback generation agent: This agent creates feedback based on educational principles and best practices.
- LLM-as-a-judge-guided regeneration loop: This process uses a human-aligned judge to enhance and refine the feedback produced.
- Self-reflective tool-calling interactive agent: This agent supports students by answering follow-up questions with context-aware, actionable responses.
Evaluation of REFINE
We evaluate REFINE through controlled experiments and an authentic classroom deployment in an undergraduate computer science course. The evaluation process involves several key methodologies:
- Automatic evaluations: These assessments show that judge-guided regeneration significantly improves feedback quality.
- Comparison with closed-source models: The interactive agent produces efficient, high-quality responses that are comparable to a state-of-the-art closed-source model.
Insights from Student Interactions
Analysis of real student interactions with the REFINE system reveals distinct engagement patterns and indicates that system-generated feedback systematically steers subsequent student inquiry. This aspect of the feedback process highlights the importance of interaction in enhancing the learning experience.
Conclusion
Our findings demonstrate the feasibility and effectiveness of multi-agent, tool-augmented feedback systems for scalable, interactive feedback. REFINE showcases a promising approach to addressing the long-standing challenge of providing personalized, timely feedback in educational settings. By treating feedback as a dynamic, interactive process rather than a static output, REFINE paves the way for transformative educational experiences that cater to individual student needs.
Future Directions
As we move forward, further research will aim to refine these systems, exploring additional pedagogical frameworks and technologies to enhance their effectiveness. The goal is to create a more engaging and supportive learning environment that empowers students to take charge of their educational journeys.