Card deck

GROVER: a model that finds the words in DNA

GROVER is a language model for the human genome, built on a BERT architecture. DNA has letters but no spaces between words, so GROVER learns its own frequency-balanced vocabulary from the sequence, then reads the genome like text. Studying what it learns starts to reveal the grammar of the code of life.

Genomics × AIPaper

1 / 12 GROVER: a model that finds the words in DNA

A twelve-slide breakdown of GROVER: how byte-pair encoding lets the genome pick its own 601-token vocabulary, why next-k-mer prediction chooses the best one, how masked token prediction teaches it sequence context, and what the trained model reveals about genes, repeats, chromatin and lexical ambiguity, unsupervised, from the human genome alone.

Read the paper: DNA language model GROVER learns sequence context in the human genome (Nature Machine Intelligence, 2024).