CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design
Summary: arXiv:2603.13431v2 Announce Type: replace-cross
Abstract: Computational antibody design has seen rapid methodological progress, with dozens of deep generative methods proposed in the past three years. However, the field lacks a standardized benchmark for fair comparison and model development. Existing methods are often evaluated on different SAbDab snapshots, non-overlapping test sets, and incompatible metrics. Moreover, the literature fragments the design problem into numerous sub-tasks without a common definition.
To address these challenges, we introduce Chimera-Bench (CDR Modeling with Epitope-guided Redesign), a unified benchmark built around a single canonical task: epitope-conditioned CDR sequence-structure co-design.
Key Features of Chimera-Bench
Chimera-Bench offers several innovative features designed to support the antibody design community:
- Curated Dataset: It includes a deduplicated dataset of 2,922 antibody-antigen complexes, complete with epitope and paratope annotations.
- Biologically Motivated Splits: The benchmark provides three distinct splits that test generalization to unseen epitopes, unseen antigen folds, and prospective temporal targets.
- Comprehensive Evaluation Protocol: A detailed evaluation framework is included, featuring five metric groups, which encompass novel epitope-specificity measures.
Benchmarking and Results
In our study, we benchmarked various representative methods that span different generative paradigms. The results were reported across all splits, highlighting the performance and generalizability of these methods.
Chimera-Bench stands as the largest dataset of its kind focused on the antibody design problem. It provides a robust platform for researchers to develop and test novel methods, enabling thorough evaluations of their generalizability.
Availability of Resources
The source code and dataset for Chimera-Bench are publicly available, fostering transparency and collaboration within the scientific community. Interested parties can access the resources at the following link:
https://github.com/mansoor181/chimera-bench.git
Conclusion
The introduction of Chimera-Bench marks a significant advancement in the field of computational antibody design. By providing a unified benchmark and a wealth of resources, it paves the way for improved methodologies and more reliable comparisons across different research efforts.