Human-in-the-Loop Control of Objective Drift in LLM-Assisted Computer Science Education
Summary: arXiv:2604.00281v1 Announce Type: new
Abstract
Large language models (LLMs) are increasingly embedded in computer science education through AI-assisted programming tools, yet such workflows often exhibit objective drift, in which locally plausible outputs diverge from stated task specifications. Existing instructional responses frequently emphasize tool-specific prompting practices, limiting durability as AI platforms evolve.
This paper adopts a human-centered stance, treating human-in-the-loop (HITL) control as a stable educational problem rather than a transitional step toward AI autonomy. Drawing on systems engineering and control-theoretic concepts, we frame objectives and world models as operational artifacts that students configure to stabilize AI-assisted work.
Proposed Curriculum
We propose a pilot undergraduate computer science laboratory curriculum that explicitly separates planning from execution. This approach trains students to specify acceptance criteria and architectural constraints prior to code generation. The curriculum aims to enhance students’ understanding of both the capabilities and limitations of AI tools in programming contexts.
Key Features of the Curriculum
- Separation of Planning and Execution: Students learn to develop comprehensive plans before diving into coding, ensuring that objectives are clearly defined.
- Specification of Acceptance Criteria: By establishing clear benchmarks for success, students are better equipped to assess AI-generated outputs.
- Introduction of Deliberate Drift: Selected labs introduce controlled drift to teach students how to diagnose and recover from specification violations.
Research Methodology
We report a sensitivity power analysis for a three-arm pilot design comparing three different instructional approaches: unstructured AI use, structured planning, and structured planning with injected drift. This analysis establishes detectable effect sizes under realistic section-level constraints, providing a robust foundation for our curriculum.
Significance of the Study
The contribution of this research lies in its theory-driven and methodologically explicit foundation for HITL pedagogy. By emphasizing control competencies, we render these skills teachable across evolving AI tools, preparing students for the future of programming in an AI-assisted environment.
Conclusion
As LLMs continue to play a significant role in computer science education, it is vital to address the challenges posed by objective drift. This paper proposes a human-in-the-loop approach that equips students with the necessary skills to navigate the complexities of AI-assisted programming. By fostering a deeper understanding of planning, specification, and control, our curriculum aims to enhance the educational experience and outcomes for future computer scientists.