Behavioral content discovery
A system and a method for automatically collecting content, the method comprising the steps of: defining a plurality of content sites, creating a collection of virtual agents data including user characteristic data and user behavioral data, presenting the collection of virtual agents to the plurality of content sites; receiving content from the visited internet site; and storing the received content or presenting it to a user.
1. A computer-implemented method of fetching content over the internet, comprising:
receiving a request, the request comprising parameters selected by a user of a user device, including an address associated with a content server, a geographic location associated with a virtual user, and user data associated with the virtual user;
selecting a proxy based at least on the received geographic location parameter;
sending a request for the selected proxy to initiate a request to the content server associated with the address, the request to the content server including at least a portion of the user data;
receiving a response that includes at least a portion of a response received by the proxy from the content server; and
providing the received response to the user device or enabling the user device to access the received response.
2. The computer-implemented method of claim 1 , wherein the user data is identified in one or more cookies.
3. The computer-implemented method of claim 1 , wherein the user data comprises web browsing history data.
4. The computer-implemented method of claim 3 , wherein the web browsing history data identifies one or more web pages or websites that the content server determines the virtual user may visit before accessing the content.
5. The computer-implemented method of claim 1 , wherein selecting the proxy further comprises selecting an IP address of a proxy device from which to initiate the request to the content server.
6. The computer-implemented method of claim 1 , wherein the response comprises at least one of: a web page with an advertisement placeholder or an advertisement associated with the placeholder that is generated based on the user data.
7. The computer-implemented method of claim 1 , wherein selecting the proxy further comprises selecting a proxy device within the same country, city, or town as the received geographic location.
8. The computer-implemented method of claim 1 , wherein selecting the proxy further comprises selecting a proxy device within the same country, city, or town as the content server.
9. The computer-implemented method of claim 1 , wherein the address comprises a universal resource locator (URL) and the content server is associated with the URL.
10. The computer-implemented method of claim 1 , wherein the user data comprises personal data associated with the virtual user, including at least one of: a gender, an age or age range, an education level, an income, or a salary.
11. The computer-implemented method of claim 1 , wherein the user data comprises a session length defining a period of time for the proxy to maintain a session with the content server.
12. The computer-implemented method of claim 1 , wherein the received request from the user device is an HTTP request.
13. The computer-implemented method of claim 1 , wherein:
the received request from the user device further comprises instructions to repeat selecting a proxy based on one or more parameters, and requesting the selected proxies to initiate requests to the content server associated with the address.
14. The computer-implemented method of claim 13 , wherein each of the proxies selected according to the one or more parameters comprise different IP addresses.
15. A computer-implemented method of fetching content over the internet, comprising:
receiving a plurality of requests, each request comprising parameters selected by a user of a user device, including a content address, and a geographic location and user data associated with a virtual user;
selecting a plurality of proxies based at least on the received content addresses and received geographic locations;
sending one or more requests for the selected plurality of proxies to initiate a request to a content server associated with the content addresses, each request to the content server including at least a portion of the user data;
receiving one or more responses that include at least a portion of a response received by the proxies; and
providing the received one or more responses to the user device or enabling the user device to access the received responses.
16. The computer-implemented method of claim 15 , wherein the responses comprise at least one of: a web page with an advertisement placeholder or an advertisement associated with the placeholder that is generated based on the user data.
17. The computer-implemented method of claim 15 , wherein selecting each proxy based on the geographic locations comprises selecting each proxy within the same country, city, or town as the geographic locations.
18. The computer-implemented method of claim 15 , wherein selecting each proxy based on the content addresses comprises selecting each proxy within the same country, city, or town as the content server associated with the content addresses.
19. The computer-implemented method of claim 15 , wherein the user data for each of the one or more virtual users comprises at least one of: internet browsing history or personal data associated with the one or more of the virtual users.
20. The computer-implemented method of claim 15 , wherein the plurality of requests each include the same content address.