Fighting AI with AI: AI-Agent Augmented DNS Blocking of LLM Services during Student Evaluations
Summary: arXiv:2604.02360v1 Announce Type: cross
The transformative potential of large language models (LLMs) in education is evident, offering benefits such as improved accessibility and personalized learning experiences. However, these advancements are accompanied by significant challenges that threaten the integrity of academic assessment. Concerns are mounting that LLMs facilitate cognitive offloading, allowing students to bypass critical thinking during evaluations. This phenomenon underscores the urgent need to balance the advantages of AI in education with the necessity of maintaining academic rigor.
Introduction to AI-Sinkhole
In response to these challenges, we introduce AI-Sinkhole, a novel framework designed to augment traditional DNS blocking methods with AI capabilities. AI-Sinkhole employs an AI-agent to dynamically discover and semantically classify emerging LLM chatbot services. This enables the framework to implement temporary network-wide blocks during proctored examinations, ensuring a level playing field for all students.
How AI-Sinkhole Works
AI-Sinkhole operates through a combination of explainable classification and dynamic DNS blocking. The framework utilizes quantized large language models, including LLama 3, DeepSeek-R1, and Qwen-3, to classify LLM services effectively. The system’s architecture includes:
- Dynamic Discovery: AI-Sinkhole continuously scans for new LLM services that may emerge.
- Semantic Classification: Leveraging AI models to classify services based on their functionalities and potential impact on assessments.
- Temporary Blocking: Implementing DNS blocking through Pi-Hole to restrict access to identified services during critical evaluation times.
Performance and Results
We conducted extensive evaluations of the AI-Sinkhole framework, focusing on its explainable classification capabilities. Notably, the use of LLMs as classifiers yielded impressive results across various languages, achieving a robust F1-score of over 0.83. This performance highlights the framework’s effectiveness in accurately identifying and blocking LLM services that could compromise exam integrity.
Supporting Future Research
To foster further research and development in this vital area, we have made initial codes available for the AI-Sinkhole framework. Educators and developers can access a readily deployable ‘AI-Sinkhole’ blocklist at the following link:
Conclusion
As educational institutions increasingly adopt AI technologies, the emergence of frameworks like AI-Sinkhole is critical. This innovative approach not only addresses the challenges posed by LLMs during assessments but also showcases the potential of AI to enhance educational integrity. By leveraging AI to combat AI, we can ensure that the benefits of technology in education do not come at the expense of critical thinking and academic rigor.