Tag: reward hacking
Browse our exclusive articles!
Preventing AI Catastrophes: Risks of Misaligned Objectives
Explore the risks of AI misalignment and how constraining capabilities can prevent catastrophic outcomes in advanced AI systems.
Evaluating AI Strategic Reasoning Risks with ESRRSim Framework
Discover ESRRSim, a taxonomy-driven framework to assess emergent strategic reasoning risks in AI and ensure safer large language model deployment.
Detecting AI Safety Violations with Meerkat Method
Discover how Meerkat improves detecting rare AI safety violations across agent traces, enhancing auditing beyond traditional methods.
MC-CPO: Safe Policy Optimization for Adaptive Tutoring
Discover MC-CPO, a mastery-conditioned constrained policy optimization method that enhances safe and effective adaptive tutoring with reduced reward hackin...
Ensuring Pedagogical Safety in AI Tutoring Systems
Explore how to formalize and detect reward hacking in educational reinforcement learning to improve AI tutoring safety and learning outcomes.