Building Evo 2: A Frontier DNA Language Model

Evolution is fascinating. It is the unifying theory of biology. The supervising force that connects genotype and the phenotype and allows them to actually influence each other. Over the last half century, we've generated massive data sets of DNA sequences and then the question is, how do we interpret it? EVO is the largest open source model for biology. The EVO series of biological foundation models is based on a core insight that this machine learning paradigm of doing next token prediction can be applied not just in natural language or in vision, but also on DNA, the fundamental code of all biology. All of the preceding technologies in DNA for reading it, writing it, editing it, we've built on these technologies. EVO2 itself is trained on whole genome sequences, 9 trillion base pairs spanning all domains of life. This was assembled through a massive scientific effort spanning 20 or 30 years of people sequencing these genomes that we use to train the model and the model learns from these genome sequences. The rules underlying these sequences and then using those rules to generate new DNA. We can then use the technologies for writing and editing DNA to test in the lab. We can read biological information, we can write it into cells, and now with EVO we have the ability to think about this information. We start to unlock a sense of reasoning, genomic reasoning, by learning this universal energy landscape of DNA. We did these experiments to try to break down the quote-unquote neural circuits of the ML model into human interpretable features and we found some pretty profound things that the model is never been told what a gene is, what a start codon is, what an exon is, but the model seems to learn these things emergently just by looking at the next token alone. So this idea of biological control is something that's model enabled and I think it starts to give you a sense of something that can be therapeutically very useful. Potentially you could not only use the model to generate a single gene or a single protein, you can use it to design entire collections of genes or entire collections of proteins. It's kind of the best of both worlds. The human can select the design tasks, see what they want to design, the nitty-gritty details of learning those biological rules and the complexity of biology, that can actually be learned from data by the models and then the human can specify maybe more higher level design tasks. Your mind starts to go crazy with the possibilities and we're really excited about exploring those not just computationally but actually designing them and putting them to real living cells and testing that out in the lab. ARC is a convening place for different institutions in the Bay Area. This is a big collaborative effort. UCSF, Stanford, Berkeley, NVIDIA, and ARC together contributed unique expertise that enabled the the model. We trained Evo2 to 2,000 NVIDIA H100 GPUs that allow us to compute over trillions of tokens of DNA to actually understand what the black box neural network is actually learning about biology. The best scientific papers open up more questions than they start out with. They open up new realms of inquiry. I think that's definitely the case for Evo2. You know, we have specific biological applications that we're excited about here. You know, we want to come up with rational therapeutic targets, build sequence to function to state models that allow us to start to make these prediction design tasks possible computationally. Hopefully we can cure Alzheimer's and aging. That would be a good start.