Extracting predictive segments from sampled data
View Patent ↗A system and method is disclosed which predicts the relative occurrence or presence of an event or item based on sample data consisting of samples which contain and samples which do not contain the event or item. The samples also consist of any number of descriptive attributes, which may be continuous variables, binary variables, or categorical variables. Given the sampled data, the system automatically creates statistically optimal segments from which a functional input/output relationship can be derived. These segments can either be used directly in the form of a lookup table or in some cases as input data to a secondary modeling system such as a linear regression module, a neural network, or other predictive system.
1. A computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, comprising the steps of:
providing segmentation attributes and sampled data; and
analyzing the distribution of sampled data;
wherein said analysis of the distribution of sampled data comprises the steps of:
ordering the transactions and occurrences by dimension and treating each dimension one at a time and independently;
creating a cumulative sequence by adding P to the previous value when the next transaction contains the item of interest and subtracting A when the item is not present, such that P=1/N P , where N P is the total number of transactions containing the item of interest, and A=1/N A , where N A is the total number of transactions not containing the item of interest, and where the total number of transactions is N total =N P +N A ;
determining the sequence of maximum relative probability of the item wherein the max and the min correspond to the candidate partition points of the dimension;
partitioning the dimension using the point furthest from the edge of the domain of the dimension in sample order; and
calculating the density factor d=r/s, whereby r=(number of items of interest in peak sequence) and s=(number of all items in peak sequence) and d is a number between 0 and 1;
whereby the density factor is considered significant if R=(r−r avg )/√{square root over (r)}≧T, where r avg =s·N P /N total and T is a predetermined user specified significance threshold.
2. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , wherein during said partitioning process the results of each step are bookmarked and said densities need only be recalculated for two partitioned cells resulting in functional pairs.
3. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 2 , wherein said functional pairs are defined as:
Y ( Z )=(number of items in the bin)/(all items in the bin),
where Z is the center of the bin.
4. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 2 ; wherein said distribution of sampled data is analyzed and the results of said analysis are stored and accessed directly using a partitioning table as a lookup table.
5. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , further comprising the steps of:
providing an application client having an end-user interface and a recommendation server having a profiler, comprising a service system to collect sample transactions both with and without an item of interest, along with associated subject profiles represented as attribute values;
providing a data collector that feeds outcome and attribute data to a service system;
having said data collector feed outcome and attribute data to the system, and storing said data in a system database wherein the outcomes include the conversion events for any items that a service customer would like to target to its users;
providing a profiling module which then builds the predictive segments for each item based on the outcomes and attributes of each user transaction; and
storing the resulting models in a model database;
whereby during operation the models are either accessed directly from the model database or cached into web servers for faster processing and higher system throughput.
6. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 5 , wherein said service system is used to recommend a specific item to a subject, comprising the steps of:
providing an application using the recommendation service;
having said application request makes a service customer request to the system, wherein said request includes attribute data;
whereby said attribute data includes the attributes that are available and relevant to the request, which include information about the page being viewed, including category, search result, or specific item being viewed; information about the subject, including age, gender, income, number of children, marital status, income, or lifetime value; information about the subject's location, including location, latitude, longitude, altitude, state, country, city, or postal code; and information about the nature of the subject's visit to the site, including time-of-day adjusted for location, type of device, type of browser, connection speed, referring URL, or search engine keyword.
7. The computer implemented web-based predictive modeling, method to extract predictive segments from sampled data used for predicting subject response, according to claim 5 , including providing previously trained segmentation models wherein said context attributes are processed through said previously trained segmentation models for the item of interest, which is stored in the model database as model training data.
8. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 7 , further including model training data consisting of a set of samples wherein an item of interest is present in a subset of said samples and an item of interest is absent from a subset of said samples and the system considers two distributions against a set of segmentation attributes that are to be used as the input variables to a predictive model as:
the distribution of occurrences D j,present where the item of interest j is present in the transaction; and
the distribution of occurrences D j,absent where the transactions include items other than j.
9. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 8 , wherein a functional relationship between a subject with attributes Z and an item j is determined by the relative probability of said subject with attributes Z making a purchase of said item j versus purchasing any other item; and further wherein said functional relationship can be quantified as:
Y j ( Z )= D j,present ( Z )/ D j,absent ( Z ),
where D j,present (Z) is the probability of a consumer with attributes Z will purchase the item of interest j, and D j,absent (Z) is probability of a consumer with attributes Z making any other kind of purchase without the item of interest j.
10. The computer implemented web-based predictive modeling, method to extract predictive segments from sampled data used for predicting subject response, according to claim 5 ; wherein said segmentation models return a score for each possible recommendable item, whereby the scores indicate the relative probabilities of the subject transacting the items.
11. The computer implemented web-based predictive modeling, method to extract predictive segments from sampled data used for predicting subject response, according to claim 5 , further including combining the predictive segment scores with the scores from other affinity or behavioral models and then re-ranking the results by the combined scores.
12. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 5 , further including filtering the results with business rules, wherein said business rules are based on attributes of the context including subject attributes and content metadata.
13. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , wherein said end-user interface is implemented using a personal computer, an in-store kiosk, a personal digital assistant (PDA), a mobile phone, a mobile tablet, a set top box, a wireless device or an ordinary phone with voice capability.
14. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , wherein said end use external application includes a customer service website that is external to the system and that communicates with the system via web services from the customer website or directly from the customer website's end user's client browser.
15. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , further including using web services on a distributed network wherein said web services on a distributed network include a website, content management, and web analytics modules which are external to the system and communicate with a data collector, a profiling engine, a recommendation engine and one or more catalog management modules via conventional web services.
16. The computer implemented web-based predictive modeling method to extract predictive segments from sampled data used for predicting subject response, according to claim 1 , wherein said segmentation attributes includes attributes that are continuous, Boolean, or categorical.