System and method for main page identification in web decoding
Web pages may be rendered from a main page data element and a plurality of embedded data elements, which are separately fetched by a browser. Herein is provided a web decoder which includes a learning engine adapted to receive human indications of data elements which are unimportant and accordingly to adjust the web decoder's procedures for determining which data elements are displayed to the user. The learning engine may receive human indications of important data elements and uses both types of indications in its further determinations. Optionally, rule generalizations are performed in a manner which searches for parameters which differentiate between important and unimportant data elements. The rule generalizations optionally concentrate on groups of data elements having at least a predetermined number of parameters having the same values for both important and unimportant data elements, reducing the chances that a generalization rule will find important data elements as unimportant.
1. A method for communication analysis, comprising the steps of:
by a decoding processor, intercepting data communication packets exchanged over a computer network during at least one web browsing session associated with a target user;
by the decoding processor, processing the packets so as to identify data elements viewed by the target user during the web browsing session;
by the decoding processor, determining for a specific identified data element, URLs of a plurality of data elements requested by the specific data element for embedding therein;
by the decoding processor, matching the URLs of the requested data elements to respective identified data elements, at least some of the matches performed although the URLs are not identical, wherein matching the URLs of the requested data elements to respective identified data elements comprises finding for each requested data element an identified data element having a most similar URL within a respective time window and determining whether the difference between the URLs of the requested data element and the most similar identified data element follow a known difference trend; and
by the decoding processor, determining the further handling of the matched identified element responsive to the matching.
2. The method according to claim 1 , wherein determining the URLs of embedded elements of the specific data element comprises providing the data element to a browser and collecting the requests generated by the browser, responsive to the specific data element.