The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models
In recent advancements within the field of artificial intelligence, particularly concerning large language models (LLMs), researchers have begun to explore the intricate representations of social roles. A new study, detailed in the preprint arXiv:2605.06196v1, investigates whether LLMs can effectively encode the granularity of social roles, ranging from micro-level individual experiences to macro-level organizational or institutional reasoning. The findings reveal significant insights into the internal structures of these models.
The research introduces a concept termed the “Granularity Axis,” which delineates the difference between mean macro- and micro-role hidden states within LLMs. This axis was found to align closely with the principal component of the role representation space in the Qwen3-8B model, boasting a cosine similarity of 0.972 and accounting for 52.6% of the variance in role representation. This suggests that the granularity of social roles is a fundamental geometric axis that organizes how these models interpret prompted social scenarios.
Research Methodology
The study constructed a comprehensive framework involving 75 distinct social roles categorized across five granularity levels. A total of 91,200 role-conditioned responses were collected using a series of shared questions and various prompt variants. Researchers meticulously extracted role-level hidden states and projected them onto the defined Granularity Axis, enabling a thorough analysis of the models’ responses.
- Monotonic Projections: The analysis revealed that role projections increased consistently across all five granularity levels.
- Stability Across Layers: Findings demonstrated that these projections remained stable across different model layers, indicating robustness in the representations.
- Transferability: The results were not limited to the Qwen3-8B model; they also transferred effectively to the Llama-3.1-8B-Instruct model, suggesting a broader applicability of the Granularity Axis concept.
Activation Steering and Controllability
An essential aspect of the research was the examination of activation steering along the Granularity Axis. This involved manipulating the model’s responses to observe changes in granularity. The Llama model, for instance, exhibited a measurable shift from a score of 2.00 to 3.17 on a five-point macro scale when positively steered with prompts conducive to local responses. This underscores the potential for controlling response granularity based on the alignment with specific prompts.
However, the study also highlighted differences in controllability between the two models. The results indicate that the ability to steer responses effectively may depend on each model’s default operating regime, raising intriguing questions about the design and training of LLMs.
Conclusion
The insights from this research suggest that the granularity of social roles in language models is not merely a superficial feature, but rather a structured and causally manipulable latent direction. This discovery opens new avenues for enhancing the performance and applicability of LLMs in socio-linguistic contexts, allowing for more nuanced and contextually appropriate responses. As AI continues to evolve, understanding these latent structures will be crucial for developing more sophisticated and capable language models.
Related AI Insights
- Strat-LLM: AI-Driven Stock Trading with Real-Time Signals
- Skill1: Unified Skill Evolution for AI Agents via RL
- DomLoRA: Optimized Adapter Placement for Efficient Fine-Tuning
- Graphlets Enhance Knowledge Graph Foundation Models
- BioResearcher: Multi-Agent System for Translational Medicine
- Policy Invariance: Ensuring Reliable LLM Safety Judges
- Optimizing OPSD for Enhanced AI Reasoning Models
- Boost Non-Thinking Model Performance with Post-Reasoning
- TheraAgent: AI-Powered Precise Treatment Planning
- MAS-Algorithm: Multi-Agent System for Algorithmic Problems
