System and Method for Improving Content Fetching by Selecting Tunnel Devices
A method for fetching a content from a web server to a client device is disclosed, using tunnel devices serving as intermediate devices. The tunnel device is selected based on an attribute, such as IP Geolocation. A tunnel bank server stores a list of available tunnels that may be used, associated with values of various attribute types. The tunnel devices initiate communication with the tunnel bank server, and stays connected to it, for allowing a communication session initiated by the tunnel bank server. Upon receiving a request from a client to a content and for specific attribute types and values, a tunnel is selected by the tunnel bank server, and is used as a tunnel for retrieving the required content from the web server, using standard protocol such as SOCKS, WebSocket or HTTP Proxy. The client only communicates with a super proxy server that manages the content fetching scheme.
1 . A method for use with a group of client devices that are each connectable to the Internet using a wireless network connection, that are each addressed by a respective Internet Protocol (IP) address, and that are each associated with a respective value, and for use with a content identified by a content identifier, the method comprising:
storing, by a first server, a list of the IP addresses of the client devices in the group and the values associated with the client devices in the group;
receiving, by the first server from an additional client device over the Internet, initiated by the additional client device, an additional value associated with the additional client device;
adding, by the first server, in response to the receiving of the additional value, an IP address of the additional device to the list;
storing, by the first server, the received additional value;
identifying, by the first server, the content;
identifying, by the first server, a first value;
selecting from the list, by the first server, a first IP address associated with a first client device that is associated with a value that is equal to the first value;
selecting from the list, by the first server, a second IP address that is different from the first IP address and is associated with a second client device that is associated with a value that is equal to the first value;
sending, by the first server to the selected first client device over the Internet, by using the first IP address, the content identifier;
receiving, by the first server from the selected first client device over the Internet, the content, information other than the content, an error indication, or any combination thereof, in response to the sending of the content identifier to the selected first client device;
sending, by the first server to the selected second client device over the Internet, by using the second IP address, the content identifier; and
receiving, by the first server from the selected second client device over the Internet, the content, information other than the content, an error indication, or any combination thereof, in response to the sending of the content identifier to the selected second client device.
2 . The method according to claim 1 , wherein the sending of the content identifier, by the first server to the selected first and second client devices, is performed simultaneously.
3 . The method according to claim 1 , wherein the sending of the content identifier, by the first server to the selected second client device over the Internet, is performed before the receiving of a response after sending of the content identifier by the first server to the selected first client device.
4 . The method according to claim 3 , wherein the sending of the content identifier, by the first server to the selected second client device, is performed in response to the receiving a response, by the first server from the selected first client device, to the sent content identifier.
5 . The method according to claim 4 , wherein the sending of the content identifier, by the first server to the selected second client device over the Internet, is performed in response to the receiving of the error indication by the first server from the selected first client device.
6 . The method according to claim 1 , wherein the identifying of the content comprises receiving, by the first server from a first device over the Internet, the content identifier.
7 . The method according to claim 6 , wherein the identifying of the first value comprises receiving, by the first server from the first device over the Internet, the first value.
8 . The method according to claim 6 , further comprising sending, by the first server to the first device over the Internet, the received content from the first or second client device.
9 . The method according to claim 1 , wherein the content comprises a web page or a part thereof, and wherein the content identifier comprises a Uniform Resource Identifier (URI) or Uniform Resource Locator (URL) of the web page.
10 . The method according to claim 1 , wherein the sending, by the first server to the selected first or second client device of the content identifier comprises sending a HyperText Transfer Protocol (HTTP) or HTTP Secure (HTTPS) request that comprises the content identifier.
11 . The method according to claim 1 , further comprising:
selecting from the list, by the first server, a third IP address that is different from the first and second IP addresses and is associated with a third client device that is associated with a value that is equal to the first value; and
sending, by the first server to the selected third client device over the Internet, by using the third IP address, the content identifier.
12 . The method according to claim 1 , further comprising receiving, by the first server from the selected third client device over the Internet, the content, information other than the content, an error indication, or any combination thereof, in response to the sending of the content identifier to the selected third client device.
13 . The method according to claim 1 , further comprising storing, operating, or using, by the first server, an operating system.
14 . The method according to claim 13 , further comprising storing or using a virtualized operating system, wherein the operating system is executed as a guest operating system as part of a Virtual Machine (VM).
15 . The method according to claim 14 , for use with a host computer that implement the VM, wherein the method further comprising executing, by the host computer, a hypervisor or a Virtual Machine Monitor (VMM), and wherein the guest operating system uses or interfaces virtual hardware.
16 . The method according to claim 14 , wherein the virtualization includes, is based on, or uses, full virtualization, para-virtualization, or hardware assisted virtualization.
17 . The method according to claim 1 , further comprising storing, operating, or using, by the first server, a server operating system.
18 . The method according to claim 1 , wherein the selecting of the first or second IP addresses comprises randomly selecting.
19 . The method according to claim 18 , wherein the randomly selecting comprises using one or more random numbers generated by a random number generator that is based on software or hardware.
20 . The method according to claim 19 , wherein the random number generator is using thermal noise, shot noise, nuclear decaying radiation, photoelectric effect, or quantum phenomena.
21 . The method according to claim 19 , wherein the random number generator is based on executing an algorithm for generating pseudo-random numbers.
22 . The method according to claim 1 , further comprising storing, operating, or using, by the additional client first device, a client operating system.
23 . The method according to claim 22 , wherein the client operating system uses one out of Microsoft Windows 7, Microsoft Windows XP, Microsoft Windows 8, Microsoft Windows 8.1, Linux, and Google Chrome OS.
24 . The method according to claim 1 , wherein the list comprises at least 10,000, 20,000, 50,000, 100,000, 200,000, 500,000, 1,000,000, 2,000,000, 5,000,000, or 10,000,000 IP addresses.
25 . A non-transitory computer readable medium containing computer instructions that, when executed by a computer processor, cause the processor to perform at least part of the steps of claim 1 .
26 . The method according to claim 1 , wherein the content is stored in a web server, and the method further comprising fetching, by the selected first or second client device, the content from the web server.
27 . The method according to claim 1 , wherein the wireless network comprises, or consists of, a Wireless Wide Area Network (WWAN).
28 . The method according to claim 27 , wherein the WWAN is a wireless broadband network.
29 . The method according to claim 28 , wherein the wireless network uses a WiMAX network, and the WiMAX network is compatible with IEEE 802.16-2009.
30 . The method according to claim 1 , wherein the wireless network comprises—a cellular telephone network.
31 . The method according to claim 30 , wherein the cellular telephone network is a Third Generation (3G) network that uses a protocol selected from the group consisting of UMTS W-CDMA, UMTS HSPA, UMTS TDD, CDMA2000 1×RTT, CDMA2000 EV-DO, and GSM EDGE-Evolution, or wherein the cellular telephone network uses a protocol selected from the group consisting of a Fourth Generation (4G) network that uses HSPA+, Mobile WiMAX, LTE, LTE-Advanced, MBWA, or is based on IEEE 802.20-2008.
32 . The method according to claim 1 , wherein the wireless network comprises, or consists of, a Wireless Personal Area Network (WPAN).
33 . The method according to claim 32 , wherein the WPAN is compatible with IEEE 802.15.1-2005 standard, or wherein the WPAN is a wireless control network that is compatible with IEEE 802.15.4-2003-standard.
34 . The method according to claim 1 , wherein the wireless network is compatible with a Wireless Local Area Network (WLAN).
35 . The method according to claim 34 , wherein the WLAN is according to, compatible with, or is based on, IEEE 802.11-2012, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, or IEEE 802.11ac standard.
36 . The method according to claim 1 , wherein the first or second client device is housed in a single enclosure that is a hand-held enclosure or a portable enclosure.
37 . The method according to claim 36 , wherein the first or second client device is further integrated with a notebook computer, a laptop computer, a media player, a Digital Still Camera (DSC), a Digital video Camera (DVC or digital camcorder), a Personal Digital Assistant (PDA), a cellular telephone, a digital camera, a video recorder, a smartphone, or any combination thereof.
38 . The method according to claim 37 , wherein the first or second client device comprises a smartphone that is an Apple iPhone 6 or a Samsung Galaxy S6.
39 . The method according to claim 1 , further comprising storing, operating, or using an operating system, by at least one of the client devices in the group, or by the selected first or second client device.
40 . The method according to claim 39 , wherein the operating system is a mobile operating system.
41 . The method according to claim 1 , further comprising storing, operating, or using, by at least one of the client devices in the group, or by the selected first or second client device, a web browser.
42 . The method according to claim 41 , wherein the web browser is a mobile web browser.
43 . The method according to claim 1 , wherein at least one of the client devices in the group, or the selected first or second client device, is integrated in part or entirely in an appliance.
44 . The method according to claim 43 , wherein a primary functionality of the appliance is food storage, handling, or preparation.
45 . The method according to claim 44 , wherein a primary function of the appliance is heating food, and wherein the appliance is a microwave oven, an electric mixer, a stove, an oven, or an induction cooker.
46 . The method according to claim 44 , wherein the appliance is a refrigerator, a freezer, a food processor, a dishwasher, a food blender, a beverage maker, a coffeemaker, or an iced-tea maker.
47 . The method according to claim 43 , wherein a primary function of the appliance is environmental control, and the appliance comprises, or is part of, an HVAC system.
48 . The method according to claim 47 , wherein a primary function of the appliance is temperature control, and wherein the appliance is an air conditioner or a heater.
49 . The method according to claim 43 , wherein a primary function of the appliance is cleaning, wherein the primary function is associated with clothes cleaning, and the appliance is a washing machine or a clothes dryer, or wherein the appliance is a vacuum cleaner.
50 . The method according to claim 43 , wherein a primary function of the appliance is water control or water heating.
51 . The method according to claim 43 , wherein the appliance is an answering machine, a telephone set, a home cinema method, a HiFi method, a CD or DVD player, an electric furnace, a trash compactor, a smoke detector, a light fixture, or a dehumidifier.
52 . The method according to claim 43 , wherein the appliance is a battery-operated portable electronic device, and the appliance is a notebook, a laptop computer, a media player, a cellular phone, a Personal Digital Assistant (PDA), an image processing device, a digital camera, a video recorder, or a handheld computing device.
53 . The method according to claim 43 , wherein the integration comprises sharing a component.
54 . The method according to claim 53 , wherein the integration involves housing in same enclosure, sharing same processor, or mounting onto same surface.
55 . The method according to claim 53 , wherein the integration involves sharing a same connector.
56 . The method according to claim 55 , wherein the connector is a power connector for connecting to a power source, and wherein the integration involves sharing the same connector for being powered from same power source, or wherein the integration involves sharing same power supply.
57 . The method according to claim 1 , wherein each of the values comprises a geographical location that comprises a name or an identifier of a country, a region, a city, a street, a ZIP code, or a timezone.
58 . The method according to claim 57 , wherein each of the geographical location is based on IP geolocation.
59 . The method according to claim 58 , wherein the IP geolocation is based on W3C Geolocation API.
60 . The method according to claim 58 , for use with a database that associates geographical to locations IP addresses.
61 . The method according to claim 60 , wherein the database is stored in the first server, or wherein the method further comprising receiving and storing, by the first server, the database.
62 . The method according to claim 1 , wherein each of the values comprises a respective identifier of an Internet Service Provider (ISP) or Autonomous System Number (ASN).
63 . The method according to claim 1 , wherein each of the values comprises corresponds to a hardware of software of client devices.
64 . The method according to claim 63 , wherein each of the values comprises the hardware of client devices.
65 . The method according to claim 64 , wherein each of the values comprises stationary or portable values, based on the respective client device being stationary or portable.
66 . The method according to claim 63 , wherein each of the values comprises a software application installed, used, or operated, in the respective client devices.
67 . The method according to claim 66 , wherein each of the values respectively comprise a type, make, model, or version of the software.
68 . The method according to claim 66 , wherein the software comprises an operating system.