Structuring versus Problematizing: How LLM-based Agents Scaffold Learning in Diagnostic Reasoning
Supporting students in developing diagnostic reasoning is a key challenge across educational domains. Novices often face cognitive biases such as premature closure and over-reliance on heuristics, and they struggle to transfer diagnostic strategies to new cases. Scenario-based learning (SBL) enhanced by Learning Analytics (LA) and large language models (LLM) offers a promising approach by combining realistic case experiences with personalized scaffolding. Yet, how different scaffolding approaches shape reasoning processes remains insufficiently explored.
This study introduces PharmaSim Switch, an SBL environment for pharmacy technician training, extended with an LA- and LLM-powered pharmacist agent that implements pedagogical conversations rooted in two theory-driven scaffolding approaches: structuring and problematizing, as well as a student learning trajectory.
Study Overview
In a between-groups experiment, 63 vocational students completed a learning scenario, a near-transfer scenario, and a far-transfer scenario under one of the two scaffolding conditions. The aim was to investigate how these two distinct approaches to scaffolding impacted the students’ engagement and reasoning processes.
Key Findings
- Effective Scaffolding: Results indicate that both scaffolding approaches were effective in supporting the use of diagnostic strategies.
- Influence of Scenario Complexity: Performance outcomes were primarily influenced by scenario complexity rather than students’ prior knowledge or the scaffolding approach used.
- Structuring Approach: The structuring approach was associated with more accurate Active and Interactive participation.
- Problematizing Approach: Problematizing elicited more Constructive engagement, encouraging students to develop deeper understanding.
Implications for Educational Design
These findings underscore the value of combining scaffolding approaches when designing LA- and LLM-based systems to effectively foster diagnostic reasoning. By integrating both structuring and problematizing strategies, educators can create a more robust learning environment that better prepares students for real-world diagnostic challenges.
As educational technologies continue to evolve, it is essential to understand how different pedagogical strategies can be implemented within these frameworks. The insights from this study could inform future developments in educational tools, ensuring they are not only effective but also adaptable to various learning needs and contexts.
Conclusion
In conclusion, the use of LLM-powered agents in combination with structured and problematizing scaffolding approaches presents a promising avenue for enhancing diagnostic reasoning in vocational education. By addressing cognitive biases and facilitating meaningful engagement with complex scenarios, such systems can significantly improve learning outcomes for students in pharmacy technician training and beyond.