Cheat sheet

gLM cheat sheet: Evo

One reference card for Evo: architecture, tokenization, size, context, training data, what it was shown to do and what it does not do. One card in a set that covers one genomic language model at a time.

GenomicsLLMMachine Learning

Genomic language model · Cheat sheet

Evo

The first genomic language model whose designed sequences were built and tested in a lab

First public
bioRxiv, February 2024
Published
Science, November 2024
Lab
Arc Institute, Stanford, TogetherAI
Architecture
StripedHyena. Hyena convolutions with a few attention layers. First genomic model to use it; the architecture itself is Poli et al. 2023.
Tokenization
Single nucleotide, byte-level tokenizer
Parameters
7B
Context
131,072 tokens. Pretrained at 8,192, then extended in a second stage.
Training data
OpenGenome. 300 billion nucleotides from prokaryotic genomes, phage and plasmid sequences.
Used for
Prediction and generation, from molecular to genome scale

Headline result

Zero-shot function prediction competitive with domain-specific language models, without task-specific training. Generated CRISPR-Cas and IS200/IS605 transposon systems whose functional activity was experimentally validated: a synthesized Evo-generated sgRNA showed in vitro cleavage activity comparable to SpCas9.

What it does not do

Prokaryotes only. No eukaryotic genomes in training, which is exactly what Evo 2 changed.

Commonly confused

Often mixed up with Evo 2. Evo holds 131,072 tokens in context; the 1 million token context belongs to Evo 2. Evo does generate sequences over 1 megabase, but those were assessed computationally, not built.

Download this card PNG, 1080 x 1350, sized for a post or a slide

Every model in the landscape figure is one dot with a tooltip. That is enough to compare them and not enough to use one. These cards are the other half: the same ten fields for every model, in the same order, so you can read one without having read any of the others.

The colour on the tokenization row is the same colour the model carries on the landscape chart, which makes each card a zoom-in on that figure rather than a separate thing to learn.

Two of the fields matter more than the rest. Headline result gives the metric and the condition it was measured under, because the number on its own does not tell you much. What it does not do is the field summaries usually leave out, and it is often the one that tells you whether the model fits your problem.

On Evo specifically, one thing is worth stating plainly because it goes wrong often. Evo generates sequences longer than 1 megabase, and its context window is 131,072 tokens. Those are different numbers and they get merged. The 1 million token context belongs to Evo 2, on StripedHyena 2. Evo is also the first genomic language model built on StripedHyena, though the architecture itself came from Poli et al. in 2023 and was built for text first.

Numbers are from the Science paper. Sizes are the largest reported checkpoint, and release is the first public preprint, matching the conventions used on the landscape figure.