Card deck

HyenaDNA: a million bases, nothing skipped

HyenaDNA is a genomic language model that reads up to 1 million DNA letters in a single pass, at single-base resolution. By swapping attention for the faster Hyena operator, it reaches context up to 500 times longer than earlier DNA models while keeping every nucleotide in view, in a strikingly small model.

Genomics × AIPaper

1 / 12 HyenaDNA: a million bases, nothing skipped

A twelve-slide breakdown of HyenaDNA, the long-context breakthrough: how the attention-free Hyena operator reaches 1 million bases at single-nucleotide resolution, why it needs no k-mers, how next-base pretraining and sequence-length warm-up work, what it was pretrained on and the limitation the authors flag, and how a model roughly 1,500 times smaller than the Nucleotide Transformer still beats it on 12 of 18 benchmarks.

Read the paper: HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution (NeurIPS, 2023).