An eleven-slide breakdown of Caduceus: why DNA is not plain text, how the Mamba state-space engine reads 131k bases at single-base resolution in near-linear time, and how BiMamba and MambaDNA build bi-directionality and reverse-complement equivariance into pre-training. A small model that respects the biology beats attention models ten times its size on the long-range tasks that need it, even edging Enformer past 100,000 bases from a gene’s start.
Read the paper: Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling (ICML, 2024).