Methods and apparatus for keyword assignment predictive intelligence modeling
In various implementations, a keyword assignment and predictive intelligence model scrapes content of web pages and stack ranks them according to a coordinates-based embedding system. The model further analyzes weblog data associated with user identifiers and stack ranks those user identifiers according to keywords associated with the weblog data. In particular embodiments, machine learning is applied to account for the entire web journey associated with the user identifiers associated with users, predicting future page URLs as well as content, products, services, information, etc., that the users may seek.
1. A method comprising:
scraping content associated with a plurality of page uniform resource locators (URLs);
analyzing the plurality of page URLs to stack rank the plurality of page URLs according to a plurality of page URL keywords having a plurality of dimensions;
obtaining activity data including weblog data of a plurality of user devices corresponding to a plurality of user identifiers, the weblog data obtained from a plurality of sources including a plurality of web servers;
analyzing the activity data to stack rank the plurality of user identifiers according to a plurality of user identifier keywords having the plurality of dimensions, wherein the activity data is continuously analyzed to update the stack rank of the user identifiers according to the plurality of the user identifier keywords as well as to develop a stack rank of the user identifiers according to a plurality of user road mapping keywords, the plurality of user road mapping keywords used to predict the page URLs that the plurality of user identifiers will visit.
2. The method of claim 1 , wherein the plurality of page URL keywords are associated with the plurality of user road mapping keywords to match users with content of interest.
3. The method of claim 1 , wherein scraping content associated with the plurality of page URLs comprises checking pages cache for a URL, applying a priority formula, and storing a result in a pages database.
4. The method of claim 1 , wherein a priority formula evaluates whether a domain of the URL is known or unknown.
5. The method of claim 1 , wherein a priority formula evaluates how many URLs were downloaded from a same domain.
6. The method of claim 1 , wherein a priority formula evaluates whether a domain is associated with an active campaign.
7. The method of claim 1 , wherein URLs associated with unknown domains and having a type of path have higher priority.
8. The method of claim 1 , wherein a plurality of priority formula factors in scraping type, average segment filling, URL type, whether a domain has active campaigns, and a number of downloaded URLs for the domain.
9. A system comprising:
an scraping manager configured to obtain and scrape content associated with a plurality of page uniform resource locators (URLs);
a processor configured to analyze the plurality of page URLs to stack rank the plurality of page URLs according to a plurality of page URL keywords having a plurality of dimensions;
a scheduler and task processor configured to obtaining activity data including weblog data of a plurality of user devices corresponding to a plurality of user identifiers, the weblog data obtained from a plurality of sources including a plurality of web servers;
wherein the activity data is analyzed to stack rank the plurality of user identifiers according to a plurality of user identifier keywords having the plurality of dimensions, wherein the activity data is continuously analyzed to update the stack rank of the user identifiers according to the plurality of the user identifier keywords as well as to develop a stack rank of the user identifiers according to a plurality of user road mapping keywords, the plurality of user road mapping keywords used to predict the page URLs that the plurality of user identifiers will visit.
10. The system of claim 9 , wherein the plurality of page URL keywords are associated with the plurality of user road mapping keywords to match users with content of interest.
11. The system of claim 9 , wherein scraping content associated with the plurality of page URLs comprises checking pages cache for a URL, applying a priority formula, and storing a result in a pages database.
12. The system of claim 9 , wherein a priority formula evaluates whether a domain of the URL is known or unknown.
13. The system of claim 9 , wherein a priority formula evaluates how many URLs were downloaded from a same domain.
14. The system of claim 9 , wherein a priority formula evaluates whether a domain is associated with an active campaign.
15. The system of claim 9 , wherein URLs associated with unknown domains and having a type of path have higher priority.
16. The system of claim 9 , wherein a plurality of priority formula factors in scraping type, average segment filling, URL type, whether a domain has active campaigns, and a number of downloaded URLs for the domain.
17. A computer readable medium comprising:
computer code for scraping content associated with a plurality of page uniform resource locators (URLs);
computer code for analyzing the plurality of page URLs to stack rank the plurality of page URLs according to a plurality of page URL keywords having a plurality of dimensions;
computer code for obtaining activity data including weblog data of a plurality of user devices corresponding to a plurality of user identifiers, the weblog data obtained from a plurality of sources including a plurality of web servers;
computer code analyzing the activity data to stack rank the plurality of user identifiers according to a plurality of user identifier keywords having the plurality of dimensions, wherein the activity data is continuously analyzed to update the stack rank of the user identifiers according to the plurality of the user identifier keywords as well as to develop a stack rank of the user identifiers according to a plurality of user road mapping keywords, the plurality of user road mapping keywords used to predict the page URLs that the plurality of user identifiers will visit.
18. The computer readable medium of claim 17 , wherein the plurality of page URL keywords are associated with the plurality of user road mapping keywords to match users with content of interest.