Generative modeling and representational learning from multi-sequence alignment and phylogenetic tree data
Generative modeling from phylogenetic data is provided. The method comprises creating a multi-sequence alignment (MSA) based on a nucleic acid or protein sequence and generating a phylogenetic tree based on the MSA. The phylogenetic tree is fed into a number of machine learning models, which generate vector representations of the nucleic acid or protein sequences based on the phylogenetic tree. The machine learning models generate from the vector representation predicted nucleic acid or protein sequences for at least one of an evolution sequence, regression sequence, or sibling sequences of nucleic acids or proteins according to the phylogenetic tree.
1 . A computer-implement method of generative modeling from phylogenetic data, the method comprising:
using a number of processors to perform:
creating a multi-sequence alignment (MSA) based on a nucleic acid or protein sequence;
generating a phylogenetic tree based on the MSA;
feeding the phylogenetic tree into a number of machine learning models;
generating, by the machine learning models, vector representations of the nucleic acid or protein sequences based on the phylogenetic tree; and
generating, by the machine learning models from the vector representation, complete predicted nucleic acid or protein sequences for at least one of an evolution sequence, regression sequence, or sibling sequences of nucleic acids or proteins according to the phylogenetic tree.
2 . The method of claim 1 , wherein the machine learning models comprise at least one of:
transformers with condition generation heads; or
transformers with masked language model heads.
3 . The method of claim 1 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and minimize cross entropy loss of sequence generation tasks.
4 . The method of claim 1 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and ignore tokens in the output sequence that have matching tokens in the input nucleic acid or protein sequence.
5 . The method of claim 1 , further comprising creating training data for predicting sibling sequences by:
finding siblings of a given leaf node of the phylogenetic tree; and
organizing the siblings and the given leaf node into a number of different pairs having alternate sequences.
6 . The method of claim 1 , further comprising creating training data for predicting regression sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is first in sequence in the pair.
7 . The method of claim 1 , further comprising creating training data for predicting evolution sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is second in sequence in the pair.
8 . A system for generative modeling from phylogenetic data, the system comprising:
a storage device that stores program instructions;
one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to:
create a multi-sequence alignment (MSA) based on a nucleic acid or protein sequence;
generate a phylogenetic tree based on the MSA;
feed the phylogenetic tree into a number of machine learning models;
generate, by the machine learning models, vector representations of the nucleic acid or protein sequences based on the phylogenetic tree; and
generate, by the machine learning models from the vector representation, complete predicted nucleic acid or protein sequences for at least one of an evolution sequence, regression sequence, or sibling sequences of nucleic acids or proteins according to the phylogenetic tree.
9 . The system of claim 8 , wherein the machine learning models comprise at least one of:
transformers with condition generation heads; or
transformers with masked language model heads.
10 . The system of claim 8 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and minimize cross entropy loss of sequence generation tasks.
11 . The system of claim 8 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and ignore tokens in the output sequence that have matching tokens in the input nucleic acid or protein sequence.
12 . The system of claim 8 , wherein the program instructions further cause the system to create training data for predicting sibling sequences by:
finding siblings of a given leaf node of the phylogenetic tree; and
organizing the siblings and the given leaf node into a number of different pairs having alternate sequences.
13 . The system of claim 8 , wherein the program instructions further cause the system to create training data for predicting regression sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is first in sequence in the pair.
14 . The system of claim 8 , wherein the program instructions further cause the system to create training data for predicting evolution sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is second in sequence in the pair.
15 . A computer program product for generative modeling from phylogenetic data, the computer program product comprising:
a persistent storage medium having program instructions configured to cause one or more processors to:
create a multi-sequence alignment (MSA) based on a nucleic acid or protein sequence;
generate a phylogenetic tree based on the MSA;
feed the phylogenetic tree into a number of machine learning models;
generate, by the machine learning models, vector representations of the nucleic acid or protein sequences based on the phylogenetic tree; and
generate, by the machine learning models from the vector representation, complete predicted nucleic acid or protein sequences for at least one of an evolution sequence, regression sequence, or sibling sequences of nucleic acids or proteins according to the phylogenetic tree.
16 . The computer program product of claim 15 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and minimize cross entropy loss of sequence generation tasks.
17 . The computer program product of claim 15 , wherein, during training, the machine learning models are provided an input nucleic acid or protein sequence and an output nucleic acid or protein sequence and ignore tokens in the output sequence that have matching tokens in the input nucleic acid or protein sequence.
18 . The computer program product of claim 15 , wherein the program instructions are further configured to cause the processors to create training data for predicting sibling sequences by:
finding siblings of a given leaf node of the phylogenetic tree; and
organizing the siblings and the given leaf node into a number of different pairs having alternate sequences.
19 . The computer program product of claim 15 , wherein the program instructions are further configured to cause the processors to create training data for predicting regression sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is first in sequence in the pair.
20 . The computer program product of claim 15 , wherein the program instructions are further configured to cause the processors to create training data for predicting evolution sequences by:
for a given leaf node in the phylogenetic tree, finding the closest leaf node that is a sibling of a parent node of the given node in the phylogenetic tree; and
pairing the given leaf node with the closest leaf node that is a sibling of the parent node, wherein the given leaf node is second in sequence in the pair.