INFORM-CT: Integrating LLMs and VLMs FOR Incidental Findings Management in Abdominal CT
Incidental findings in CT scans, while often benign, can have significant clinical implications. These findings should be reported following established guidelines to ensure proper patient care. Traditional manual inspection by radiologists, however, is a time-consuming and variable process. A new paper, available on arXiv (arXiv:2512.14732v2), proposes a novel framework designed to improve the efficiency and precision of incidental findings detection, classification, and reporting in abdominal CT scans.
The proposed framework, referred to as INFORM-CT, leverages the capabilities of large language models (LLMs) and foundational vision-language models (VLMs). By adopting a plan-and-execute agentic approach, INFORM-CT automates the management of incidental findings based on established medical guidelines for abdominal organs.
Framework Overview
INFORM-CT operates through a structured planner-executor framework. The planner utilizes LLMs to generate Python scripts that are built upon predefined base functions. These scripts are then executed by the executor, which employs VLMs, segmentation models, and image processing subroutines to perform the necessary checks and detections.
Key Components of INFORM-CT
- Large Language Models (LLMs): These models are responsible for understanding medical guidelines and generating the appropriate scripts needed for incidental findings management.
- Vision-Language Models (VLMs): VLMs play a crucial role in interpreting visual data from CT scans, enabling the system to accurately identify and classify incidental findings.
- Segmentation Models: These models help in delineating specific structures or anomalies within the CT images, thereby enhancing the detection process.
- Image Processing Subroutines: These components support the overall framework by applying various algorithms and techniques to improve image analysis and ensure accurate reporting.
Experimental Validation
The effectiveness of the INFORM-CT approach was demonstrated through experiments conducted on a benchmark dataset of abdominal CT scans, focusing on three specific organs. The results revealed that the proposed framework significantly outperforms existing pure VLM-based approaches in terms of both accuracy and efficiency.
By fully automating the incidental findings detection and reporting process, INFORM-CT not only alleviates the burden on radiologists but also enhances the overall reliability of incidental findings management in clinical practice.
Conclusion
The integration of LLMs and VLMs within the INFORM-CT framework represents a significant advancement in the realm of medical imaging. As the healthcare industry continues to embrace artificial intelligence, frameworks like INFORM-CT will play a pivotal role in improving diagnostic accuracy and patient outcomes, ultimately leading to more effective healthcare delivery.