Min-$k$ Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics
Summary: arXiv:2604.11012v1 Announce Type: new
Abstract: The quality of text generated by large language models depends critically on the decoding sampling strategy. While mainstream methods such as Top-$k$, Top-$p$, and Min-$p$ achieve a balance between diversity and accuracy through probability-space truncation, they share an inherent limitation: extreme sensitivity to the temperature parameter.
Recent logit-space approaches like Top-$n\sigma$ achieve temperature invariance but rely on global statistics that are susceptible to long-tail noise, failing to capture fine-grained confidence structures among top candidates. To address these challenges, we propose Min-$k$ Sampling, a novel dynamic truncation strategy that analyzes the local shape of the sorted logit distribution to identify “semantic cliffs”: sharp transitions from high-confidence core tokens to uncertain long-tail tokens.
Key Features of Min-$k$ Sampling
Min-$k$ Sampling introduces several innovative features aimed at improving the text generation process:
- Dynamic Truncation: By computing a position-weighted relative decay rate, Min-$k$ dynamically determines truncation boundaries at each generation step, allowing for more adaptive sampling.
- Temperature Invariance: We formally prove that Min-$k$ achieves strict temperature invariance, meaning its performance is consistent across various temperature settings, unlike traditional methods.
- Low Sensitivity to Hyperparameters: Empirical results demonstrate that Min-$k$ maintains robust performance even under extreme temperature settings where probability-based methods may collapse.
Impact on Text Quality
Extensive experiments conducted on multiple reasoning benchmarks and creative writing tasks reveal that Min-$k$ consistently improves text quality. This is particularly noteworthy in scenarios where other methods struggle due to their reliance on temperature parameters. Human evaluations further corroborate the effectiveness of Min-$k$, showcasing its ability to generate coherent and contextually appropriate text.
Conclusion and Availability
In conclusion, Min-$k$ Sampling presents a significant advancement in the realm of language model decoding strategies. By decoupling the truncation process from temperature scaling, it offers a more reliable and effective approach to text generation. We are committed to transparency and collaboration in the research community, and thus, we have made our code, models, and analysis tools publicly available for further exploration and enhancement by fellow researchers and practitioners.