PERFORMING REGRESSION ANALYSIS ON PERSONAL DATA RECORDS
Disclosed herein are system, method, and computer program product embodiments for performing a regression analysis on lawfully collected personal data records. The analysis enables discovery of individuals likely to perform certain actions based on their personal data records and the personal data records and actions of others. The disclosed system, method, and computer program product may process vast quantities of data, including personal data records with thousands of categories and lawfully stored databases with millions of personal data records. Through the regression analysis, the disclosed system, method, and computer program product learn the most relevant categories for predicting an individual's actions based on input data provided by a user. The analysis then analyzes the categories of personal data records stored in a lawfully stored database to predict actions of individuals associated with those records and outputs results to the user.
1 . A computer implemented method, comprising:
dividing a training set of personal data records into a first subset and a second subset, wherein each personal data record of the training set includes an action category corresponding to an action taken by an individual corresponding to the personal data record;
training a first prediction function with the first subset;
in response to training the first prediction function, applying the first prediction function to the second subset to generate a first predictor score indicating accuracy of the first prediction function based on action values for the action category of each of the personal data records of the second subset;
training a second prediction function with the second subset;
in response to training the second prediction function, applying the second prediction function to the first subset to generate a second predictor score indicating accuracy of the second prediction function based on action values for the action category of each of the personal data records of the first subset;
determining that the first predictor score indicates a higher accuracy of action values relative to the second predictor score;
in response to the determining, applying the first prediction function to a personal data record from a data set different from the training set to determine an outcome score indicating a probability of an action value corresponding to the personal data record from the data set.
2 . The computer implemented method of claim 1 , wherein applying the first prediction function to the second subset further comprises:
determining a mean squared error of outcome values of the first prediction function relative to the action values for the action category of each of the personal data records of the second subset.
3 . The computer implemented method of claim 1 , further comprising:
parsing data within the personal data records of the training set; and
assigning labels corresponding to predetermined categories, wherein the labels include the action category.
4 . The computer implemented method of claim 1 , further comprising:
comparing a first personal data record of the training set with a second personal data record stored in a database; and
determining that the first personal data record and the second personal data record correspond to a common individual by determining that a similarity score based on a pair-wise function exceeds a predetermined threshold.
5 . The computer implemented method of claim 4 , further comprising:
in response to the determining, adding a category value from the second personal data record to the first personal data record in the training set; and
in response to the adding, dividing the training set into the first subset and the second subset.
6 . The computer implemented method of claim 1 , wherein the action values indicate a number of occurrences of an action.
7 . The computer implemented method of claim 1 , wherein the outcome score is a decimal value between 0.0 and 1.0.
8 . A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
divide a training set of personal data records into a first subset and a second subset, wherein each personal data record of the training set includes an action category corresponding to an action taken by an individual corresponding to the personal data record;
train a first prediction function with the first subset;
in response to training the first prediction function, apply the first prediction function to the second subset to generate a first predictor score indicating accuracy of the first prediction function based on action values for the action category of each of the personal data records of the second subset;
train a second prediction function with the second subset;
in response to training the second prediction function, apply the second prediction function to the first subset to generate a second predictor score indicating accuracy of the second prediction function based on action values for the action category of each of the personal data records of the first subset;
determine that the first predictor score indicates a higher accuracy of action values relative to the second predictor score;
in response to the determining, apply the first prediction function to a personal data record from a data set different from the training set to determine an outcome score indicating a probability of an action value corresponding to the personal data record from the data set.
9 . The system of claim 8 , wherein to apply the first prediction function to the second subset, the at least one processor is further configured to:
determine a mean squared error of outcome values of the first prediction function relative to the action values for the action category of each of the personal data records of the second subset.
10 . The system of claim 8 , wherein the at least one processor is further configured to:
parse data within the personal data records of the training set; and
assign labels corresponding to predetermined categories, wherein the labels include the action category.
11 . The system of claim 8 , wherein the at least one processor is further configured to:
compare a first personal data record of the training set with a second personal data record stored in a database; and
determine that the first personal data record and the second personal data record correspond to a common individual by determining that a similarity score based on a pair-wise function exceeds a predetermined threshold.
12 . The system of claim 11 , wherein the at least one processor is further configured to:
in response to the determining, add a category value from the second personal data record to the first personal data record in the training set; and
in response to the adding, divide the training set into the first subset and the second subset.
13 . The system of claim 8 , wherein the action values indicate a number of occurrences of an action.
14 . The system of claim 8 , wherein the outcome score is a decimal value between 0.0 and 1.0.
15 . A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
dividing a training set of personal data records into a first subset and a second subset, wherein each personal data record of the training set includes an action category corresponding to an action taken by an individual corresponding to the personal data record;
training a first prediction function with the first subset;
in response to training the first prediction function, applying the first prediction function to the second subset to generate a first predictor score indicating accuracy of the first prediction function based on action values for the action category of each of the personal data records of the second subset;
training a second prediction function with the second subset;
in response to training the second prediction function, applying the second prediction function to the first subset to generate a second predictor score indicating accuracy of the second prediction function based on action values for the action category of each of the personal data records of the first subset;
determining that the first predictor score indicates a higher accuracy of action values relative to the second predictor score;
in response to the determining, applying the first prediction function to a personal data record from a data set different from the training set to determine an outcome score indicating a probability of an action value corresponding to the personal data record from the data set.
16 . The non-transitory computer-readable device of claim 15 , wherein applying the first prediction function to the second subset further comprises:
determining a mean squared error of outcome values of the first prediction function relative to the action values for the action category of each of the personal data records of the second subset.
17 . The non-transitory computer-readable device of claim 15 , the operations further comprising:
parsing data within the personal data records of the training set; and
assigning labels corresponding to predetermined categories, wherein the labels include the action category.
18 . The non-transitory computer-readable device of claim 15 , the operations further comprising:
comparing a first personal data record of the training set with a second personal data record stored in a database; and
determining that the first personal data record and the second personal data record correspond to a common individual by determining that a similarity score based on a pair-wise function exceeds a predetermined threshold.
19 . The non-transitory computer-readable device of claim 18 , the operations further comprising:
in response to the determining, adding a category value from the second personal data record to the first personal data record in the training set; and
in response to the adding, dividing the training set into the first subset and the second subset.
20 . The non-transitory computer-readable device of claim 15 , wherein the action values indicate a number of occurrences of an action.