Restoring Heterogeneity in LLM-based Social Simulation: An Audience Segmentation Approach
Summary: arXiv:2604.06663v1 Announce Type: cross
Abstract
Large Language Models (LLMs) are increasingly used to simulate social attitudes and behaviors, offering scalable “silicon samples” that can approximate human data. However, current simulation practice often collapses diversity into an “average persona,” masking subgroup variation that is central to social reality. This study introduces audience segmentation as a systematic approach for restoring heterogeneity in LLM-based social simulation.
Introduction
The reliance on LLMs for simulating social interactions and attitudes has grown significantly in recent years. These models, such as Llama 3.1-70B and Mixtral 8x22B, have the potential to generate insights that reflect societal complexities. However, the common practice of aggregating diverse social attitudes into a singular persona undermines the richness of human experience.
Methodology
This study employs U.S. climate-opinion survey data to explore audience segmentation as a means to enhance the heterogeneity of LLM-based simulations. We compare six segmentation configurations while varying:
- Identifier granularity
- Parsimony
- Selection logic (theory-driven, data-driven, and instrument-based)
Evaluation Framework
To assess the performance of these simulations, we developed a three-dimensional evaluation framework that encompasses:
- Distributional fidelity
- Structural fidelity
- Predictive fidelity
Results
Our findings indicate that increasing identifier granularity does not consistently enhance simulation performance. While moderate enrichment can improve outcomes, excessive segmentation may diminish structural and predictive fidelity. Our comparisons reveal that:
- Compact configurations often match or exceed the performance of more comprehensive models, particularly in structural and predictive fidelity.
- Distributional fidelity is contingent on the selected metric.
- Identifier selection logic plays a crucial role in performance outcomes; instrument-based selection excels in preserving distributional shape, while data-driven selection is effective at recovering between-group structures and associations.
Conclusion
The research underscores the importance of audience segmentation as a methodological approach for enhancing the validity of LLM-based social simulations. It demonstrates that a nuanced understanding of heterogeneity, coupled with variance-preserving modeling strategies and evaluations, can lead to more accurate representations of social dynamics. Importantly, our study highlights that there is no one-size-fits-all configuration; rather, performance improvements in one evaluation dimension may lead to trade-offs in another.
This work serves as a pivotal step towards more informed and reliable applications of LLMs in social simulation, challenging researchers to adopt heterogeneity-aware evaluation frameworks in their methodologies.