Tag: RLHF
Browse our exclusive articles!
Token-Space Attacks on Reward Models in RLHF
Discover how token-space attacks exploit reward models in RLHF, revealing vulnerabilities beyond semantic manipulation and impacting AI safety.
Limits of Reinforcement Learning Alignment in AI Safety
Explore the generalization limits of reinforcement learning alignment and its impact on AI safety in large language models with compound jailbreaks analysi...
The Silicon Mirror: Reducing Sycophancy in LLMs
Discover how The Silicon Mirror framework dynamically reduces sycophantic behavior in large language models, ensuring factual accuracy and trust.
Why Safety Probes Detect Liars but Miss Fanatics
Discover why AI safety probes catch deceptive models but fail to detect coherent misaligned fanatics, posing new challenges for AI safety.
Enhancing Text Summarization with Human Feedback AI
Discover how reinforcement learning from human feedback improves AI text summarization for better quality, customization, and adaptability.