Systems and methods for identifying and analyzing internet users
This disclosure describes systems, methods, and apparatus for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content.
1. A method for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content, the method comprising:
collecting content or other raw data via a crawler module that accesses webpages;
associating the content or other raw data with a social profile residing in or being added to a memory;
calculating scores for the social profile based on terms parsed from the content or other raw data, wherein the scores are based at least in part on a number and quality of relationships between a user associated with the social profile and other users;
receiving a query, via the network interface, for users fitting one or more contexts;
identifying social profiles fitting the one or more contexts; and
returning a report in response to the query comprising the social profiles fitting the one or more contexts wherein an order of the social profiles is based on the scores.
2. The method of claim 1 , wherein the calculating scores is performed based on one or more of reach, relevance, and impact scores for each of the terms parsed from the content or other raw data.
3. The method of claim 2 , wherein:
the relevance scores represents a quantity of content generated over a period of time that is germane to one of the terms and how relevant the content is to the one of the terms;
the reach scores represents a number of relationships between a user and other users as well as a quality of those relationships; and
the impact scores represents a number and quality of actions taken in response to a user's publication of content on the Internet.
4. The method of claim 1 , further comprising:
receiving another query for contexts fitting one or more users;
identifying the contexts fitting the one or more users; and
returning another report in response to the another query having the contexts.
5. The method of claim 1 , further comprising generating a graph datastore including nodes representing users and including edges between the nodes, where the edges represent relationships between the users, and where the edges are weighted based on a quality of each relationship.
6. The method of claim 5 , further comprising adding a node to the graph datastore for every URL crawled.
7. The method of claim 5 , wherein the relationships are identified based on (1) explicit links between users and (2) user actions that imply relationship.
8. A method for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content, the method comprising:
seeding a crawler module;
crawling the Internet to find new users that can be used to create new social profiles and crawling the Internet to populate and update existing social profiles, the crawling based on the seeding;
parsing content or other raw data generated by the first and second crawling, into terms;
associating the terms and the content or other raw data with the existing social profiles and the new social profiles;
computing scores for each term, based on one or more of reach, relevance, and impact, where:
reach is based on a number of relationships between a user associated with a social profile and other users as well as a weight assigned to each relationship,
relevance is based upon a quantity of content generated relative to a term over a period of time that is germane to the term and how relevant each piece of content is to the term, and
impact is based on a number and quality of events triggered by content generated by a user relative to one or more terms;
associating the scores for each term with the existing social profiles and the new social profiles;
receiving a query via the network interface for a ranking of social profiles for one or more contexts, each context describing a combination of any one or more terms;
generating contextual scores based on the scores and the one or more contexts; and
returning a report via the network interface, the report including one or more social profiles matching the one or more contexts and ranked in terms of the contextual scores.
9. The method of claim 8 , further comprising generating a graph datastore including nodes representing users and including edges between the nodes, where the edges represent relationships between the users, and where the edges are weighted based on a quality of each relationship.
10. The method of claim 9 , further comprising adding a node to the graph datastore for every URL crawled.
11. The method of claim 9 , wherein reach is based on a number of nodes that a given node is connected to as well as a weight assigned to the edges between the given node and nodes connected to the given node.
12. The method of claim 8 , wherein a marketing client provides customer data used in the seeding and the customer data is used to generate the existing social profiles.
13. The method of claim 8 , wherein the seeding is based at least in part on a query including one or more contexts.
14. A method for generating reports enhancing an understanding of Internet users based on their generated content and actions taken by others in response to the generated content, the method comprising:
providing, via an API, a query for users fitting a context, the context describing products, services, and/or a type of customer;
providing customer data describing existing customers;
creating first existing social profiles based on the customer describing existing customers;
generating URLs from the context; seeding a crawler module with the URLs;
crawling the Internet to create first new social profiles and to update and populate the first existing social profiles, the crawling based on the URLs, the first new social profiles and the first existing social profiles together referred to as second existing social profiles;
extracting additional URLs from content and metadata returned via the crawling;
seeding the crawler module with the additional URLs; crawling the Internet to create second new social profiles and to update and populate the second existing social profiles, the crawling based on the additional URLs, the second new social profiles and the second existing social profiles together referred to as third existing social profiles;
associating terms parsed from the content and metadata returned by the first and second crawling with the third existing social profiles;
computing scores for each term for the context, and assigning the scores to associated social profiles in the third existing social profiles, the scores reflecting one or more of a reach, relevance, and impact for the context; and
returning a report via the network interface, the report comprising the social profiles fitting the context wherein an order of the social profiles is based on one or more of the reach, relevance, and impact for the context.
15. The method of claim 14 , wherein the scores are based at least in part on a number and quality of relationships between a user associated with the at least one of the third existing social profiles and other users.
16. The method of claim 15 , wherein the relationships are identified based on (1) explicit links between users and (2) user actions that imply relationships.
17. The method of claim 15 , wherein the computing is based on a graph datastore including nodes representing users, and includes edges between the nodes, where the edges represent relationships between the users, and wherein the edges are weighted based on a quality of each relationship.
18. The method of claim 17 , wherein the crawler module adds a node to the graph datastore for every URL crawled.
19. The method of claim 17 , wherein a weight assigned to an edge is greater for an edge generated based on an explicit link between users than an edge generated based on user actions that imply a relationship between users.
20. The method of claim 15 , wherein:
the relevance represents a quantity of content generated over a period of time that is germane to one of the terms and how relevant the content is to the one of the terms;
the reach represents a number of relationships between a user and other users as well as a quality of those relationships; and
the impact represents a number and quality of actions taken in response to a user's publication of content on the Internet.