System and method for retrieval-based controllable molecule generation
A machine learning framework is described for performing generation of candidate molecules for, e.g., drug discovery or other applications. The framework utilizes a pre-trained encoder-decoder model to interface between representations of molecules and embeddings for those molecules in a latent space. A fusion module is located between the encoder and decoder and is used to fuse an embedding for an input molecule with embeddings for one or more exemplary molecules selected from a database that is constructed according to a design criteria. The fused embedding is decoded using the decoder to generate a candidate molecule. The fusion module is trained to reconstruct a nearest neighbor to the input molecule from the database based on the sample of exemplary molecules. An iterative approach may be used during inference to dynamically update the database to include newly generated candidate molecules.
1. A method for determining, using a machine learning framework, a candidate molecule for satisfying a design criteria, the method comprising:
receiving an input molecule data structure and the design criteria;
selecting, based on the input molecule data structure and the design criteria, a plurality of exemplary molecule data structures from a database;
providing the input molecule data structure as input to a trained encoder of the machine learning framework and processing, by the trained encoder, the input molecule data structure to generate an embedding of the input molecule data structure, wherein the embedding of the input molecule data structure is a numerical vector or tensor of a pre-defined dimension;
providing the plurality of exemplary molecule data structures as input to the trained encoder and processing by the trained encoder, the plurality of exemplary molecule data structures to generate embeddings of the plurality of exemplary molecule data structures, wherein each respective embedding of a respective exemplary molecule data structure is a numerical vector or tensor of a pre-defined dimension;
fusing, by a trained cross-attention mechanism, the embedding of the input molecule data structure and the embeddings of the exemplary molecule data structures to generate a fused embedding; and
providing the fused embedding as input to a trained decoder of the machine learning framework and processing, by the trained decoder, the fused embedding to generate a candidate molecule data structure.
2. The method of claim 1 , wherein the encoder comprises a bidirectional encoder and the decoder comprises an autoregressive decoder.
3. The method of claim 2 , wherein the encoder and decoder are trained using a ZINC dataset.
4. The method of claim 1 , wherein the input molecule data structure and the plurality of exemplary molecule data structures are simplified molecular-input line-entry system (SMILES) string data structures.
5. The method of claim 1 , wherein the pre-trained cross-attention mechanism is trained in accordance with an objective to predict a nearest neighbor of the input molecule data structure in a training data set stored in the database, given as:
(θ)=Σ i=1 CE ( DEC ( f CA ( e in (i) ,E r (i) ;θ)),x 1NN (i) ).
6. The method of claim 1 , wherein the selecting, based on the input molecule data structure and the design criteria, the plurality of exemplary molecule data structures from the database comprises:
calculating, in accordance with a score function, a score value for each of a plurality of molecule data structures stored in the database; and
selecting, via a retriever of the machine learning framework, K molecule data structures from the database as the plurality of exemplary molecule data structures, wherein the K exemplary molecule data structures are the K molecule data structures in the database having the top score values.
7. The method of claim 6 , wherein the design criteria specifies L properties for the candidate molecule, and wherein each molecule stored in the database has at least one predicted property value of L properties that is greater than a threshold value.
8. The method of claim 1 , wherein the encoder, the decoder, and the cross-attention mechanism comprise instructions configured to be executed by one or more processors of a computer device.
9. The method of claim 1 , further comprising:
generating, based on the fused embedding, a plurality of perturbed embeddings by adding noise to the fused embedding;
providing each perturbed embedding in of the plurality of perturbed embeddings to the trained decoder and processing, by the trained decoder, the plurality of perturbed embeddings to generate a plurality of second candidate molecule data structures;
calculating a score value for each second candidate molecule data structure of the plurality of second candidate molecule data structures; and
selecting a respective second candidate molecule data structure with the highest score value of the score values calculated for the second candidate molecule data structures as a best candidate molecule data structure.
10. The method of claim 9 , further comprising:
calculating a score value for the input molecule data structure;
comparing the score value for the input molecule data structure with the score value for the best candidate molecule data structure;
in response to determining that the score value for the best candidate molecule data structure is greater than the score value for the input molecule data structure, updating the database by adding the best candidate molecule data structure to the database; and
repeating the method for a new input molecule using the updated database.
11. The method of claim 1 , wherein providing the input molecule data structure and generating the embedding of the input molecule data structure is performed by a first instance of the trained encoder,
wherein the providing the plurality of exemplary molecule data structures as input to the trained encoder and generating the embeddings of the plurality of exemplary molecule data structures is performed via a plurality of second instances of the trained encoder, and
wherein the first instance of the trained encoder and the plurality of second instances of the trained encoder generate the embedding of the input molecule data structure and the embeddings of the plurality of exemplary molecule data structures in parallel.
12. The method of claim 1 , wherein (i) the providing the input molecule data structure as input to the trained encoder and generating the embedding of the input molecule data structure and (ii) the providing the plurality of exemplary molecule data structures as input to the trained encoder and generating the embeddings of the plurality of exemplary molecule data structures are performed sequentially using a single instance of the trained encoder.
13. A system for determining, using a machine learning framework, a candidate molecule for satisfying a design criteria, the system comprising:
a memory storing a database containing a plurality of molecule data structures; and
at least one processor, communicatively coupled to the memory, and the at least one processor being configured to:
receive an input molecule data structure and the design criteria;
select, based on the input molecule data structure and the design criteria, a plurality of exemplary molecule data structures from the database;
provide the input molecule data structure as input to a trained encoder of the machine learning framework and process, via the trained encoder, the input molecule data structure to generate an embedding of the input molecule data structure, wherein the embedding of the input molecule data structure is a numerical vector or tensor of a pre-defined dimension;
provide the plurality of exemplary molecule data structures as input to the trained encoder and process via the trained encoder, the plurality of exemplary molecule data structures to generate embeddings of the plurality of exemplary molecule data structures, wherein each respective embedding of a respective exemplary molecule data structure is a numerical vector or tensor of a pre-defined dimension;
fuse, via a trained cross-attention mechanism, the embedding of the input molecule data structure and the embeddings of the exemplary molecule data structures to generate a fused embedding; and
provide the fused embedding as input to a trained decoder of the machine learning framework and process, via the trained decoder, the fused embedding to generate a candidate molecule data structure.
14. The system of claim 13 , wherein the encoder comprises a bidirectional encoder and the decoder is comprises an autoregressive decoder.
15. The system of claim 13 , wherein the input molecule data structure and the plurality of exemplary molecule data structures are simplified molecular-input line-entry system (SMILES) string data structures.
16. The system of claim 13 , wherein the at least one processor is configured to select, based on the input molecule data structure and the design criteria, the plurality of exemplary molecule data structures from the database by:
calculating, in accordance with a score function, a score value for each of a plurality of molecule data structures stored in the database; and
selecting, via a retriever of the machine learning framework, K molecule data structures from the database as the number of exemplary molecules, wherein the K exemplary molecules are the K molecules in the database having the top score values.
17. The system of claim 13 , wherein the at least one processor is further configured to:
generate, based on the fused embedding, a plurality of perturbed embeddings by adding noise to the fused embedding;
provide each perturbed embedding of the plurality of perturbed embeddings to the trained decoder and process, via the trained decoder, the plurality of perturbed embeddings, to generate a plurality of second candidate molecule data structures;
calculate a score value for each second candidate molecule data structure of the plurality of second candidate molecule data structures; and
select a respective second candidate molecule data structure with the highest calculated score value of the score values calculated for the second candidate molecule data structures as a best candidate molecule data structure.
18. The system of claim 17 , wherein the at least one processor is further configured to:
calculate a score value for the input molecule data structure;
compare the score value for the input molecule data structure with the score value for the best candidate molecule data structure;
in response to determining that the score value for the best candidate molecule data structure is greater than the score value for the input molecule data structure, update the database by adding of the best candidate molecule data structure to the database; and
repeat the method for a new input molecule using the updated database.
19. A non-transitory computer readable medium storing instructions that, in response to being executed by a computing device, cause the computing device to:
select, based on an input molecule data structure and a design criteria, a plurality of exemplary molecule data structures from a database;
provide the input molecule data structure as input to a trained encoder of the machine learning framework and process, by the trained encoder, the input molecule data structure to generate an embedding of the input molecule data structure, wherein the embedding of the input molecule data structure is a numerical vector or tensor of a pre-defined dimension:
provide the plurality of exemplary molecule data structures as input to the trained encoder and process, by the trained encoder, the plurality of exemplary molecule data structures to generate embeddings of the plurality of exemplary molecule data structures, wherein each respective embedding of a respective exemplary molecule data structure is a numerical vector or tensor of a pre-defined dimension;
fuse, via a trained cross-attention mechanism, the embedding of the input molecule data structure and the embeddings of the exemplary molecule data structures to generate a fused embedding; and
provide the fused embedding as input to a trained decoder of the machine learning framework and process, via the trained decoder, the fused embedding to generate a candidate molecule data structure.