Snapshot generation for search results page preview
Methods, systems, and programming for providing web page snapshots are disclosed. A URL is received. A snapshot of the web page associated with the URL is generated. A plurality of features is extracted from the snapshot. A determination is made regarding whether the snapshot is high quality based on the plurality of extracted features of the snapshot. The generated snapshot is provided as a viewable and actionable link to the URL.
1. A method implemented on at least one computing device, each computing device having at least one processor, storage, and a communication platform connected to a network for providing web page snapshots, the method comprising:
receiving a URL;
generating a snapshot of the web page associated with the URL;
extracting a plurality of features from the snapshot;
determining quality of the snapshot based on the plurality of extracted features of the snapshot in accordance with a model that is established based on previous evaluation on a plurality of snapshots; and
associating, if the snapshot is of acceptable quality, the snapshot with the URL as a viewable and actionable representation of the URL.
2. The method of claim 1 , wherein the generating a snapshot comprises:
detecting whether the web page associated with the URL includes an error; and
storing the URL in a failed URL database if the web page associated with the URL includes an error.
3. The method of claim 1 , wherein the extracting comprises:
analyzing the snapshot to extract features related to a color distribution of the snapshot; and
analyzing the source code of web page and its snapshot to extract features related to a web page title and snapshot size of the snapshot.
4. The method of claim 3 , wherein the analyzing the snapshot comprises:
partitioning the snapshot into a plurality of grids; and
determining features based on a color distribution of at least one of the plurality of grids.
5. The method of claim 4 , wherein the determining features based on a color distribution comprises:
determining feature combinations of adjacent grids of the plurality of grids.
6. The method of claim 1 further comprising: generating the model by:
receiving the training data;
extracting features from the training data;
scoring the extracted features to generate a training feature set; and
generating the model using the training feature set.
7. A machine readable non-transitory and tangible medium having information recorded for providing web page snapshots, wherein the information, when read by the machine, causes the machine to perform the steps comprising:
receiving a URL;
generating a snapshot of the web page associated with the URL;
extracting a plurality of features from the snapshot;
extracting a plurality of features from the source of the web page associated with the URL;
determining quality of the snapshot based on the plurality of extracted features of the snapshot in accordance with a model that is established based on previous evaluation on a plurality of snapshots; and
associating, if the snapshot is of acceptable quality, the snapshot with the URL as a viewable and actionable representation of the URL.
8. The machine readable non-transitory and tangible medium of claim 7 , wherein the generating a snapshot comprises:
detecting whether the web page associated with the URL includes an error; and
storing the URL in a failed URL database if the web page associated with the URL includes an error.
9. The machine readable non-transitory and tangible medium of claim 7 , wherein the extracting comprises:
analyzing the snapshot to extract features related to a color distribution of the snapshot; and
analyzing the snapshot to extract features related to a web page title and snapshot size of the snapshot.
10. The machine readable non-transitory and tangible medium of claim 9 , wherein the analyzing the snapshot comprises:
partitioning the snapshot into a plurality of grids; and
determining features based on a color distribution of at least one of the plurality of grids.
11. The machine readable non-transitory and tangible medium of claim 10 , wherein the determining features based on a color distribution comprises:
determining feature combinations of adjacent grids of the plurality of grids.
12. The machine readable non-transitory and tangible medium of claim 7 , wherein the information, when read by the machine, causes the machine to perform the steps comprising:
generating the model by:
receiving the training data;
extracting features from the training data;
scoring the extracted features to generate a training feature set; and
generating the model using the training feature set.
13. A system for providing web page snapshots, comprising:
a snapshot generator configured for generating a snapshot of a web page associated with a received URL;
a feature extractor configured for extracting a plurality of features from the snapshot;
a testing unit configured for determining quality of the snapshot based on the plurality of extracted features of the snapshot in accordance with a model that is established based on previous evaluation on a plurality of snapshots; and
a snapshot repository configured for associating, if the snapshot is of acceptable quality, the snapshot as a viewable and actionable representation of the URL.
14. The system of claim 13 , wherein the snapshot generator is further configured for detecting whether the web page associated with the URL includes an error; and storing the URL in a failed URL database if the web page associated with the URL includes an error.
15. The system of claim 13 , wherein the feature extractor is further configured for analyzing the snapshot to extract features related to a color distribution of the snapshot and the snapshot's size; and analyzing the source codes of the web page to extract features related to a web page title.
16. The system of claim 15 , wherein the feature extractor is further configured for partitioning the snapshot into a plurality of grids; and determining features based on a color distribution of at least one of the plurality of grids.
17. The system of claim 16 , wherein the feature extractor is further configured for determining feature combinations of adjacent grids of the plurality of grids.
18. The system of claim 13 further comprising:
a model generator for generating the model by:
receiving the training data;
extracting features from the training data;
scoring the extracted features to generate a training feature set; and
generating the model using the training feature set.