Card deck

DNABERT: the first BERT for the genome

DNABERT adapted BERT, the breakthrough language model from natural language AI, to the human genome. It reads DNA as overlapping k-letter words, pre-trains once on the raw genome, and is then fine-tuned to find promoters, splice sites and protein binding sites. Published in 2021, it opened the field of DNA language models.

Genomics × AIPaper

1 / 12 DNABERT: the first BERT for the genome

A twelve-slide breakdown of DNABERT, the origin of the DNA language model: how it reads DNA as overlapping k-mers, why pre-training once and fine-tuning for each task changed genomics, how contiguous-span masking teaches it context, and how one model reached above 0.9 across 690 ENCODE binding-site datasets and even transferred from human to mouse.

Read the paper: DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome (Bioinformatics, 2021).