Data platform for automated pharmaceutical research using knowledge graph
A system and method for an automated pharmaceutical research data platform comprising a data curation platform which searches for and ingests a plurality of unstructured, heterogenous medical data sources, extracts relevant information from the ingested data sources, and creates a massive, custom-built and intricately related knowledge graph using the extracted data, and a data analysis engine which receives data queries from a user interface, conducts analyses in response to queries, and returns results based on the analyses. The system hosts a suite of modules and tools, integrated with the custom knowledge graph and accessible via the user interface, which may provide a plurality of functions such as statistical and graphical analysis, similarity based searching, and edge prediction among others.
1. A system for automated pharmaceutical research, comprising:
a computer system comprising a memory and a processor;
a data curation platform, comprising a first plurality of programming instructions stored in the memory of, and operating on the processor, wherein the first plurality of programming instructions, when operating on the processor, cause the computer system to:
scrape data from a plurality of data sources accessible via the internet;
parse the scraped data into a format that may be stored in a relational database;
train an encoder on a subset of the data stored in the relational database to identify the key features that define the data stored in the relational database; and
use the trained encoder to create an abstract latent space represented as a three-dimensional convolutional neural network, wherein the abstract latent space contains vectorized representations of all data points in the relational database; and
a data analysis engine, comprising a second plurality of programming instructions stored in the memory of, and operating on the processor, wherein the second plurality of programming instructions, when operating on the processor, cause the computer system to:
receive a similarity search query, the similarity search query comprising a search term;
input the search term into the trained encoder to create an abstract vector representation of the input search term, existing in the abstract latent space of the three-dimensional convolutional neural network;
compute the distance between two abstract vectors within the latent space;
compare the computed distance to a predetermined threshold of similarity, wherein a computed distance less than the threshold indicates non-similarity between the two abstract vectors and a computed distance greater than the threshold indicates similarity; and
decode the abstract vector that is greater than the threshold and return data related to the decoded vector as a response to the received similarity search query.
2. A method for automated pharmaceutical research, comprising the steps of:
scraping data from a plurality of data sources accessible via the internet;
parsing the scraped data into a format that may be stored in a relational database;
training an encoder on a subset of the data stored in the relational database to identify the key features that define the data stored in the relational database;
using the trained encoder to create an abstract latent space represented as a three-dimensional convolutional neural network, wherein the abstract latent space contains vectorized representations of all data points in the relational database;
computing the distance between two abstract vectors within the latent space;
comparing the computed distance to a predetermined threshold of similarity, wherein a computed distance less than the threshold indicates non-similarity between the two abstract vectors and a computed distance greater that the threshold indicates similarity; and
decoding the abstract vector that is greater than the threshold and return data related to the decoded vector as a response to the received similarity search query.