BIOLOGICAL STRUCTURE TOKENIZER
For a specific amino acid in a protein, physically neighboring amino acids of the specific amino acid are determined in a local physical protein structure. Representations of the determined physically neighboring amino acids are included in a structure encoder input for the specific amino acid. The structure encoder input is provided to an autoencoder trained using geometric loss to determine a token representing the local physical protein structure for the specific amino acid.
1 . A method, comprising:
for a specific amino acid in a protein, determining physically neighboring amino acids of the specific amino acid based on physical distances with respect to the specific amino acid in a local physical protein structure;
including representations of the determined physically neighboring amino acids in a structure encoder input for the specific amino acid;
providing the structure encoder input to an autoencoder trained using geometric loss to determine a local structure token representing the local physical protein structure for the specific amino acid separate from an individual amino acid token for the specific amino acid;
combining the local structure token in a second input track with the individual amino acid token in a first input track into a combined sequence data and using the combined sequence data to train a biological language reasoning machine learning model;
predicting a protein property including by unmasking a masked token in an input token sequence of the biological language reasoning machine learning model; and
causing physical synthesis of a physical protein having assembled physical amino acids and having the protein property predicted using the biological language reasoning machine learning model.
2 . The method of claim 1 , wherein determining the physically neighboring amino acids of the specific amino acid in the local physical protein structure includes:
determining a physical distance value between the specific amino acid and a candidate amino acid; and
based on the determined physical distance value, including the candidate amino acid as one of the physically neighboring amino acids.
3 . The method of claim 2 , wherein determining the physical distance value between the specific amino acid and the candidate amino acid includes:
determining a reference location for the specific amino acid; and
determining a reference location for the candidate amino acid;
wherein the physical distance value is based on the reference location for the specific amino acid and the reference location for the candidate amino acid.
4 . The method of claim 3 , wherein the reference location for the specific amino acid includes a coordinate for an origin location of the specific amino acid and a corresponding rotation matrix for the specific amino acid.
5 . The method of claim 4 , wherein the origin location corresponds to coordinates of a nitrogen (N), alpha-carbon (CA), or carbon (C) atom of the specific amino acid.
6 . The method of claim 3 , wherein the determined physical distance value corresponds to a Euclidean distance calculation.
7 . The method of claim 1 , wherein the autoencoder is configured to: encode the structure encoder input as a latent structure; and quantize the encoded latent structure to determine the local structure token.
8 . The method of claim 7 , wherein the autoencoder is configured with a learned codebook to quantize the encoded latent structure to determine the local structure token.
9 . The method of claim 1 , wherein the autoencoder is configured with one or more geometric reasoning blocks, and wherein at least one of the one or more geometric reasoning blocks includes a geometric attention mechanism.
10 . The method of claim 1 , wherein the geometric loss is modeled using a function that determines an error loss value based on relative orientations of bond vectors in a predicted structure and a ground truth structure.
11 . A system, comprising:
one or more processors configured to:
for a specific amino acid in a protein, determine physically neighboring amino acids of the specific amino acid based on physical distances with respect to the specific amino acid in a local physical protein structure;
include representations of the determined physically neighboring amino acids in a structure encoder input for the specific amino acid;
provide the structure encoder input to an autoencoder trained using geometric loss to determine a local structure token representing the local physical protein structure for the specific amino acid separate from an individual amino acid token for the specific amino acid;
combine the local structure token in a second input track with the individual amino acid token in a first input track into a combined sequence data and use the combined sequence data to train a biological language reasoning machine learning model;
predict a protein property including by being configured to unmask a masked token in an input token sequence of the biological language reasoning machine learning model; and
cause physical synthesis of a physical protein having assembled physical amino acids and having the protein property predicted using the biological language reasoning machine learning model; and
a memory coupled to at least one of the one or more processors and configured to provide instructions.
12 . The system of claim 11 , wherein the one or more processors are configured to:
determine a physical distance value between the specific amino acid and a candidate amino acid; and
based on the determined physical distance value, include the candidate amino acid as one of the physically neighboring amino acids.
13 . The system of claim 12 , wherein the one or more processors are configured to:
determine a reference location for the specific amino acid; and
determine a reference location for the candidate amino acid;
wherein the physical distance value is based on the reference location for the specific amino acid and the reference location for the candidate amino acid.
14 . The system of claim 13 , wherein the reference location for the specific amino acid includes a coordinate for an origin location of the specific amino acid and a corresponding rotation matrix for the specific amino acid.
15 . The system of claim 14 , wherein the origin location corresponds to coordinates of a nitrogen (N), alpha-carbon (CA), or carbon (C) atom of the specific amino acid.
16 . The system of claim 11 , wherein the autoencoder is configured to: encode the structure encoder input as a latent structure; and quantize the encoded latent structure to determine the local structure token.
17 . The system of claim 16 , wherein the autoencoder is configured with a learned codebook to quantize the encoded latent structure to determine the local structure token.
18 . The system of claim 11 , wherein the autoencoder is configured with one or more geometric reasoning blocks, and wherein at least one of the one or more geometric reasoning blocks includes a geometric attention mechanism.
19 . The system of claim 18 , wherein the geometric loss is modeled using a function that determines an error loss value based on relative orientations of bond vectors in a predicted structure and a ground truth structure.
20 . A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
for a specific amino acid in a protein, determining physically neighboring amino acids of the specific amino acid based on physical distances with respect to the specific amino acid in a local physical protein structure;
including representations of the determined physically neighboring amino acids in a structure encoder input for the specific amino acid;
providing the structure encoder input to an autoencoder trained using geometric loss to determine a local structure token representing the local physical protein structure for the specific amino acid separate from an individual amino acid token for the specific amino acid;
combining the local structure token in a second input track with the individual amino acid token in a first input track into a combined sequence data and using the combined sequence data to train a biological language reasoning machine learning model;
predicting a protein property including by unmasking a masked token in an input token sequence of the biological language reasoning machine learning model; and
causing physical synthesis of a physical protein having assembled physical amino acids and having the protein property predicted using the biological language reasoning machine learning model.