Automatically Inferring Teachers’ Geometric Content Knowledge: A Skills Based Approach
Summary: arXiv:2604.13666v1 Announce Type: cross
Assessing teachers’ geometric content knowledge is essential for ensuring high-quality geometry instruction and enhancing student learning outcomes. However, traditional methods for evaluating this knowledge can be challenging to scale effectively. The Van Hiele model, which characterizes geometric reasoning through five hierarchical levels, has been a cornerstone in this assessment. Unfortunately, conventional assessments based on the Van Hiele model often rely on time-consuming manual expert analysis of open-ended responses, making large-scale evaluations impractical.
This study introduces an innovative automated approach aimed at diagnosing teachers’ Van Hiele reasoning levels, leveraging large language models grounded in educational theory. Our central hypothesis posits that integrating explicit skills information can significantly enhance the accuracy of Van Hiele classification.
Research Methodology
In collaboration with mathematics education researchers, our team developed a structured skills dictionary that breaks down Van Hiele levels into 33 fine-grained reasoning skills. This structured approach allows for a more nuanced understanding of geometric reasoning.
To validate our hypothesis, we created a custom web platform where 31 pre-service teachers engaged with a series of geometry problems, generating a total of 226 responses. These responses were then carefully annotated by expert researchers, each assigned a Van Hiele level and demonstrated skills based on our dictionary.
Classification Approaches
Utilizing the annotated dataset, we implemented two distinct classification approaches:
- Retrieval-Augmented Generation (RAG): This method enhances response generation by retrieving relevant information from the skills dictionary.
- Multi-Task Learning (MTL): This approach allows the model to learn multiple tasks simultaneously, improving its ability to classify responses accurately.
Each classification method included a skills-aware variant that incorporated the skills dictionary, compared to a baseline model that operated without this vital information.
Results and Implications
The results of our study demonstrated that both skills-aware variants significantly outperformed their baseline counterparts across various evaluation metrics. This finding underscores the importance of incorporating explicit skills information in automated assessments.
This pioneering work provides the first automated approach for classifying Van Hiele levels from open-ended responses. It not only offers a scalable method for assessing teachers’ geometric reasoning but also supports the development of adaptive and personalized teacher learning systems. By enabling large-scale evaluations, this approach has the potential to transform how educators are supported in their professional development.
Conclusion
In conclusion, our study represents a significant step forward in the assessment of geometric content knowledge among teachers. By harnessing advanced language models and integrating detailed skills information, we pave the way for more efficient and effective evaluations. This research has far-reaching implications for both mathematics education and the professional growth of educators.