Performance Optimization System and Method for a Client Advertising Campaign

1 - 33 . (canceled)

34 . A method for optimizing performance, comprising:

receiving, by a performance optimization system (POS) comprising a POS data platform configured to store data usable to determine a POS score, the POS data platform comprising a profile store comprising a plurality of profiles of end users, the POS data platform further comprising a model store configured to store a predictive model usable to determine the POS score that the model builder builds, the performance optimization system further comprising a machine learning platform configured to use machine learning to determine the POS score, the machine learning platform operably connected to the POS data platform, the machine learning platform comprising a model builder configured to build the predictive model and a model scorer operably connected to the model builder, the model scorer configured to receive the predictive model from the model builder, the model scorer further configured to use the predictive model to determine the POS score, the performance optimization system further comprising a prediction server operably connected to the machine learning platform, the prediction server comprising a server configured to receive an advertisement request from a client demand-side platform (DSP), the prediction server further configured to create a prediction request from the advertisement request, the prediction server further configured to score the prediction request to determine a likelihood to influence an end user by exposing the end user to the brand advertisement, the prediction server configured to receive the prediction request from the client DSP, and a prediction request log operably connected to the prediction server, the prediction request log configured to log the scored prediction request, an advertisement request;

selecting, by the performance optimization system, relevant prediction request data from the advertisement request;

building, by the performance optimization system, using the model builder, the predictive model;

determining, by the performance optimization system, using the model scorer, the model scorer using the predictive model, the POS score;

creating, by the performance optimization system, the prediction request by copying the relevant prediction request data from the advertisement request to the prediction request;

adding, by the performance optimization system, the POS score to the prediction request, creating a scored prediction request; and

sending, by the performance optimization system, the POS score to the client DSP.

35 . The method of claim 34 , wherein the prediction request data comprises end user data.

36 . The method of claim 35 , wherein the end user data comprises one or more of end user personal data, end user device data, contextual data, advertisement spot data, website data, mobile app data, network data, and privacy data.

37 . The method of claim 36 , wherein the POS score comprises an end user engagement metric predicting end user engagement with an advertisement.

38 . The method of claim 37 , wherein the POS score comprises one or more of a brand awareness score, a purchase intent score, a brand consideration score, and another metric configured to estimate awareness of an end user of an advertised brand.

39 . The method of claim 34 , wherein the profile store comprises predictive end user data usable by the model builder to build the predictive model.

40 . The method of claim 34 , wherein the determining step comprises determining the POS score without using end user data.

41 . The method of claim 40 , wherein the determining step comprises determining the POS score without requiring the end user data.

42 . The method of claim 34 , wherein the determining step comprises determining the POS score without using personally identifiable information (PII) regarding the end user.

43 . The method of claim 42 , wherein the determining step comprises determining the POS score without requiring the PII.

44 . The method of claim 34 , wherein the determining step comprises determining the POS score in real time.

45 . The method of claim 34 , wherein the prediction request comprises a request for a POS score from a client DSP to the prediction server.

46 . The method of claim 34 , wherein the receiving step comprises receiving the advertisement request directly from a supply-side platform (SSP).

47 . The method of claim 46 , wherein the method further includes additional steps, performed after the step of receiving the advertisement request directly from the SSP, of:

creating, by the performance optimization system, a prediction request from the advertisement request;

determining, by the performance optimization system, the POS score;

adding, by the performance optimization system, the POS score to the prediction request, creating a scored prediction request; and

sending, by the performance optimization system, the scored prediction request to the client DSP.

48 . The method of claim 34 , wherein the building step comprises building the model using customer engagement data.

49 . The method of claim 48 , wherein the customer engagement data comprises survey responses.

50 . The method of claim 49 , wherein the survey responses comprise end user profiles that comprise the end user's response to a customer engagement campaign.

51 . The method of claim 34 , wherein the method further comprises an additional step, performed after the determining step, of:

logging, by the performance optimization system, the scored prediction request in the prediction request log.

52 . The method of claim 34 , wherein the determining step comprises sub-steps of:

creating a plurality of population groups, one population group comprising a control group comprising a baseline population group against which other population groups' POS scores can be compared, the plurality of population groups usable to determine the POS score for the prediction request;

assigning the prediction request to a population group; and

determining the POS score for the prediction request based in part on the prediction request's population group.

53 . The method of claim 52 , wherein there is one population group other than the control group.

54 . The method of claim 52 , wherein the assigning step comprises assigning a predetermined fraction of prediction requests to the control group.

55 . The method of claim 54 , wherein the assigning step comprises assigning prediction requests not belonging to the control group to a treatment group.

56 . The method of claim 52 , wherein the determining step comprises determining, by the performance optimization system, using the model scorer, the model scorer using the predictive model, the POS score for the treatment group prediction request, and wherein the determining step further comprises assigning, by the performance optimization system, for the control group prediction request, a non-optimized POS score to the control group prediction request, the non-optimized POS score usable to determine the performance of the treatment group.

57 . The method of claim 52 , wherein the assigning step comprises labeling a predetermined percentage of the prediction requests as control prediction requests belonging to the control group.

58 . The method of claim 57 , wherein the assigning step comprises randomly labeling the predetermined percentage of the prediction requests as control prediction requests.

59 . The method of claim 57 , wherein the predetermined percentage comprises 1 percent.

60 . The method of claim 57 , wherein the non-optimized POS score comprises one or more of a random POS score, a fixed POS score and a POS score that the prediction server chooses in any way without applying output from the model scorer.

61 . The method of claim 60 , wherein the random POS score comprises a POS score that the prediction server randomly selects from a range of all possible POS scores.

62 . The method of claim 60 , wherein the random POS score comprises a random number drawn from a distribution identical to a distribution of POS scores for the treatment group.

63 . The method of claim 60 , wherein the fixed POS score comprises a POS score equal to an average of POS scores that the model scorer has determined for the treatment group over a selected time period.

64 . The method of claim 63 , wherein the time period comprises a previous 24 hours.

65 - 67 . (canceled)