Pramana: Fine-Tuning Large Language Models for Epistemic Reasoning through Navya-Nyaya
Summary: arXiv:2604.04937v1 Announce Type: new
Abstract: Large language models produce fluent text but struggle with systematic reasoning, often hallucinating confident but unfounded claims. When Apple researchers added irrelevant context to mathematical problems, LLM performance degraded by 65%. This exposes brittle pattern-matching beneath apparent reasoning. This epistemic gap, the inability to ground claims in traceable evidence, limits AI reliability in domains requiring justification.
We introduce Pramana, a novel approach that teaches LLMs explicit epistemological methodology by fine-tuning on Navya-Nyaya logic, a 2,500-year-old Indian reasoning framework. Unlike generic chain-of-thought prompting, Navya-Nyaya enforces structured 6-phase reasoning:
- SAMSHAYA: Doubt analysis
- PRAMANA: Evidence source identification
- PANCHA AVAYAVA: 5-member syllogism with universal rules
- TARKA: Counterfactual verification
- HETVABHASA: Fallacy detection
- NIRNAYA: Ascertainment distinguishing knowledge from hypothesis
This integration of logic and epistemology provides cognitive scaffolding absent from standard reasoning approaches. We fine-tune Llama 3.2-3B and DeepSeek-R1-Distill-Llama-8B on 55 Nyaya-structured logical problems, which include:
- Constraint satisfaction
- Boolean SAT
- Multi-step deduction
In Stage 1, our models achieved 100% semantic correctness on held-out evaluations despite only 40% strict format adherence. This finding reveals that models internalize reasoning content even when structural enforcement is imperfect. Our ablation studies demonstrate that format prompting and temperature critically affect performance. Notably, optimal configurations differ by stage.
To foster further research on epistemic frameworks for AI reasoning, we are releasing all models, datasets, and training infrastructure on Hugging Face. This commitment aims to encourage collaboration and exploration in the realm of AI epistemology, providing researchers with the tools necessary to advance the field.
As AI continues to evolve, the integration of structured reasoning methodologies like Navya-Nyaya into large language models holds the potential to enhance their reliability and effectiveness in complex reasoning tasks. With Pramana, we pave the way for more robust AI systems capable of justifiable reasoning, bridging the epistemic gap that has persisted in the field.