Card deck

DNABERT-2: smaller, faster, and a benchmark to prove it

DNABERT-2 is the efficient successor to DNABERT. It swaps brittle k-mer chunks for byte-pair tokens, drops the input-length limit, and matches a 21-times-bigger model on the GUE benchmark it also introduced, a fair multi-species test the whole field can compare on.

Genomics × AIPaper

DNABERT-2: smaller, faster, and a benchmark to prove it — 1 / 11

DNABERT-2: smaller, faster, and a benchmark to prove it — 2 / 11

An eleven-slide breakdown of DNABERT-2: why k-mer tokenization held DNA models back, how byte-pair encoding cuts sequence length about 5x and avoids the leakage problem, and how a 117-million-parameter model matched the 2.5-billion Nucleotide Transformer on GUE at roughly 92x less GPU time. It also ships GUE, a standardized multi-species benchmark of 36 datasets across 9 task types.

Read the paper: DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes (ICLR, 2024).