Let the Agent Steer: Closed-Loop Ranking Optimization via Influence Exchange
Summary: arXiv:2603.27765v1 Announce Type: new
Abstract
Recommendation ranking is fundamentally an influence allocation problem: a sorting formula distributes ranking influence among competing factors, and the business outcome depends on finding the optimal “exchange rates” among them. However, offline proxy metrics systematically misjudge how influence reallocation translates to online impact, with asymmetric bias across metrics that a single calibration factor cannot correct.
Introduction
In the rapidly advancing field of artificial intelligence and machine learning, the need for effective ranking optimization in recommendation systems has become paramount. This is especially true as businesses aim to enhance user experience and maximize revenue through personalized recommendations. The traditional methods often rely on offline metrics that do not consistently translate into meaningful online results, leading to potential misalignment of strategies.
Introducing Sortify
We present Sortify, the first fully autonomous LLM-driven ranking optimization agent deployed in a large-scale production recommendation system. This innovative agent reframes ranking optimization as a continuous influence exchange, effectively closing the loop from diagnosis to parameter deployment without any human intervention. Sortify addresses several critical structural problems through three mechanisms:
- Dual-Channel Framework: Grounded in Savage’s Subjective Expected Utility (SEU), this mechanism decouples offline-online transfer correction (Belief channel) from constraint penalty adjustment (Preference channel).
- LLM Meta-Controller: Operating on framework-level parameters rather than low-level search variables, this component streamlines the optimization process.
- Persistent Memory Database: Featuring 7 relational tables, this database facilitates cross-round learning, enabling the agent to improve continuously based on past performance.
Core Metric: Influence Share
At the heart of Sortify’s functionality is its core metric, Influence Share, which provides a decomposable measure where all factor contributions sum to exactly 100%. This metric allows for a clear understanding of how different factors contribute to the overall ranking influence, thereby enabling more precise adjustments and optimizations.
Deployment Success
Sortify has been successfully deployed across two Southeast Asian markets, showcasing its effectiveness in real-world applications. In Country A, the agent managed to increase Gross Merchandise Value (GMV) from -3.6% to +9.2% within just 7 rounds, with peak orders reaching an impressive +12.5%. Meanwhile, in Country B, a cold-start deployment achieved +4.15% GMV per Unique User (UU) and +3.58% Ads Revenue during a 7-day A/B test, paving the way for a full production rollout.
Conclusion
The introduction of Sortify marks a significant advancement in the field of recommendation systems, demonstrating how autonomous agents can optimize ranking processes through innovative frameworks and metrics. As businesses continue to seek smarter solutions for enhancing user engagement and profitability, Sortify stands out as a pioneering tool that leverages the power of AI to steer recommendation systems toward greater success.