Towards Platonic Representation for Table Reasoning: A Foundation for Permutation-Invariant Retrieval
Summary: arXiv:2604.12133v1 Announce Type: new
Abstract: Historical approaches to Table Representation Learning (TRL) have largely adopted the sequential paradigms of Natural Language Processing (NLP). We argue that this linearization of tables discards their essential geometric and relational structure, creating representations that are brittle to layout permutations. This paper introduces the Platonic Representation Hypothesis (PRH) for tables, positing that a semantically robust latent space for table reasoning must be intrinsically Permutation Invariant (PI).
To ground this hypothesis, we first conduct a retrospective analysis of table-reasoning tasks, highlighting the pervasive serialization bias that compromises structural integrity. We then propose a formal framework to diagnose this bias, introducing two principled metrics based on Centered Kernel Alignment (CKA):
- PI: Measures embedding drift under complete structural derangement.
- rho: A Spearman-based metric that tracks the convergence of latent structures toward a canonical form as structural information is incrementally restored.
Our empirical analysis quantifies an expected flaw in modern Large Language Models (LLMs): even minor layout permutations induce significant, disproportionate semantic shifts in their table embeddings. This exposes a fundamental vulnerability in RAG systems, in which table retrieval becomes fragile to layout-dependent noise rather than to semantic content.
In response, we present a novel, structure-aware TRL encoder architecture that explicitly enforces the cognitive principle of cell header alignment. This model demonstrates superior geometric stability and moves towards the PI ideal.
Our work provides both a foundational critique of linearized table encoders and the theoretical scaffolding for semantically stable, permutation invariant retrieval, charting a new direction for table reasoning in information systems. The implications of this research are broad, as it not only challenges existing methodologies but also lays the groundwork for future advancements in TRL.
In conclusion, by addressing the limitations of traditional approaches and proposing a robust framework for permutation-invariant retrieval, we aim to enhance the reliability and effectiveness of table reasoning in various applications. This research opens new avenues for exploration in the intersection of AI, data representation, and information retrieval.