Systems and Methods for Improved Machine Learning Using Data Completeness and Collaborative Learning Techniques
Systems and methods for improved machine learning using data completeness and collaborative learning techniques are provided. The system receives one or more sets of data, and classifies samples within the data into a multi-dimensional tree data structure. Next, the system identifies outliers and null values within the tree. Then, the system fills in the outliers and null values based on neighboring values. Collaborative filtering AI technology can be utilized to fill the rest of the missing values of all data attributes.
1 . A system for improved machine learning, comprising:
a memory storing one or more sets of data; and
a processor in communication with the memory, the processor performing the steps of:
receiving the one or more sets of data from the memory;
processing the one or more sets of data to classify samples within the one or more sets of data into a tree data structure;
processing the tree data structure to identify outliers and null values within the tree data structure;
updating the tree structure by filling in the outliers and the null values based on neighboring values in the tree data structure; and
storing the updated tree structure.
2 . The system of claim 1 , wherein the one or more sets of data comprises data corresponding to one or more of physical characteristics to be examined, well characteristics, performance metrics, energy resource site characteristics, sensor measurement data, or human survey data.
3 . The system of claim 1 , wherein the processor performs the step of filtering noise from the one or more data sets.
4 . The system of claim 1 , wherein the processor performs the step of updating the tree structure by identifying data having location attributes that are in physical proximity to one another.
5 . The system of claim 1 , wherein the processor performs the step of updating the tree structure by identifying data generated by similar sensors.
6 . The system of claim 5 , wherein the similar sensors operate at the same time and under similar conditions.
7 . The system of claim 1 , wherein the processor performs the step of updating the tree structure by identifying demographically identical or similar persons or objects.
8 . The system of claim 1 , wherein the processor performs the step of updating the tree structure using a matrix factorization process.
9 . The system of claim 1 , wherein the step of processing the one or more sets of data to classify samples within the one or more sets of data into the tree data structure is performed by indexing and partitioning of the one or more sets of data.
10 . The system of claim 9 , wherein the step of processing the one or more sets of data to classify samples within the one or more sets of data into the tree data structure is performed using a k-dimensional B-tree algorithm.
11 . A method for improved machine learning, comprising the steps of:
receiving by a processor one or more sets of data from a memory;
processing the one or more sets of data to classify samples within the one or more sets of data into a tree data structure;
processing the tree data structure to identify outliers and null values within the tree data structure;
updating the tree structure by filling in the outliers and the null values based on neighboring values in the tree data structure; and
storing the updated tree structure in the memory.
12 . The method of claim 11 , wherein the one or more sets of data comprises data corresponding to one or more of physical characteristics to be examined, well characteristics, performance metrics, energy resource site characteristics, sensor measurement data, or human survey data.
13 . The method of claim 11 , further comprising filtering noise from the one or more data sets.
14 . The method of claim 11 , further comprising updating the tree structure by identifying data having location attributes that are in physical proximity to one another.
15 . The method of claim 11 , further comprising updating the tree structure by identifying data generated by similar sensors.
16 . The method of claim 15 , wherein the similar sensors operate at the same time and under similar conditions.
17 . The method of claim 11 , further comprising updating the tree structure by identifying demographically identical or similar persons or objects.
18 . The method of claim 11 , further comprising updating the tree structure using a matrix factorization process.
19 . The method of claim 11 , further comprising indexing and partitioning the one or more sets of data.
20 . The method of claim 19 , wherein the step of processing the one or more sets of data to classify samples within the one or more sets of data into the tree data structure is performed using a k-dimensional B-tree algorithm.