The Geometry of Forgetting
Summary: arXiv:2604.06222v1 Announce Type: cross
Why do we forget? Why do we remember things that never happened? The conventional answer points to biological hardware. However, a new study proposes a different perspective: geometry. This research demonstrates that high-dimensional embedding spaces, when subjected to noise, interference, and temporal degradation, can reproduce quantitative signatures of human memory without any specific engineering for phenomena.
Key Findings
The study reveals several intriguing insights into the mechanics of memory and forgetting:
- Power-Law Forgetting: A power-law forgetting rate of $b = 0.460 \pm 0.183$ was observed, closely aligning with the human forgetting rate of approximately $b \approx 0.5$. This suggests that forgetting is largely influenced by interference among competing memories rather than mere decay.
- The Role of Competition: The study found that when competition among memories was eliminated, the decay function yielded a significantly smaller forgetting rate of $b \approx 0.009$, which is fifty times smaller than the power-law forgetting rate. This indicates that time alone does not result in forgetting; it is competition that drives this phenomenon.
- Effective Dimensions: Production embedding models, which typically operate in nominally 384 to 1,024 dimensions, were shown to concentrate their variance in roughly 16 effective dimensions. This positioning places them deep within the interference-vulnerable regime, further emphasizing the impact of memory competition.
- False Memories: The research highlights that false memories can emerge without any deliberate engineering. By applying cosine similarity to unmodified pre-trained embeddings, the study reproduced the Deese–Roediger–McDermott false alarm rate of $0.583$, which is comparable to the human rate of approximately $0.55$. This was achieved with zero parameter tuning and no boundary conditions, indicating that false memories are inherent within the geometry of semantic space.
Implications of the Study
These findings suggest that core memory phenomena are not merely glitches or bugs resulting from biological implementation. Instead, they are fundamental features of any system that organizes information by meaning and retrieves it based on proximity. The geometry of information storage and retrieval plays a critical role in understanding both human memory and artificial intelligence systems.
Conclusion
As research continues to explore the intersection of memory and geometry, the implications for artificial intelligence and cognitive science are profound. Understanding the geometrical underpinnings of forgetting and false memories may lead to more robust AI systems that mimic human-like memory processes. This study invites further investigation into how memory operates across different systems, potentially reshaping our understanding of cognition and memory in both biological and artificial entities.