LumiVideo: An Intelligent Agentic System for Video Color Grading
Video color grading plays a vital role in post-production, transforming raw footage into visually captivating cinematic presentations. The process typically involves enhancing the color and tone of footage to evoke emotions and tell compelling stories. Traditional automated methods have largely fallen short, functioning as static, black-box solutions that provide limited interpretability and lack the iterative control professionals require. However, a new system known as LumiVideo aims to revolutionize this aspect of video production.
Introducing LumiVideo
LumiVideo is an intelligent agentic system designed to mimic the cognitive workflow of professional colorists through a structured four-stage process: Perception, Reasoning, Execution, and Reflection. This innovative approach enables LumiVideo to autonomously generate a cinematic base grade from raw log video footage by analyzing both the physical lighting conditions and the semantic content of the scene.
Four Stages of LumiVideo
- Perception: LumiVideo begins with a thorough analysis of the raw footage, assessing physical lighting and scene composition.
- Reasoning: The system utilizes a Reasoning engine that merges a large language model’s (LLM) cinematic knowledge with a Retrieval-Augmented Generation (RAG) framework, employing a Tree of Thoughts (ToT) search to navigate the complex non-linear color parameter space.
- Execution: Instead of generating pixels directly, LumiVideo compiles the deduced parameters into industry-standard configurations, such as the ASC-CDL, while also creating a globally consistent 3D Look-Up Table (LUT) to ensure temporal consistency.
- Reflection: An optional Reflection loop allows creators to provide natural language feedback, enabling them to refine the grading results further.
Benchmarking and Results
In addition to its grading capabilities, the developers of LumiVideo have introduced LumiGrade, the first log-encoded video benchmark designed specifically for evaluating automated color grading systems. Initial experiments demonstrate that LumiVideo can achieve a high level of quality that approaches that of human experts, particularly in fully automatic mode. Moreover, when directed, the system allows for precise iterative control, catering to the specific needs of professional colorists.
Conclusion
LumiVideo represents a significant advancement in the field of video color grading, combining automation with the nuanced understanding typically associated with human professionals. By integrating a systematic approach that balances perception, reasoning, execution, and reflection, LumiVideo not only enhances the efficiency of color grading but also preserves the artistic integrity of the final product. This innovative system is poised to become an invaluable tool in the toolkit of modern filmmakers, providing a bridge between technology and creativity.