Automatic detection and transfer of relevant image data to content collections
This application is directed to a method for automatically identifying and transferring relevant image data implemented at a server with a processor and memory including one or programs executable by the processor. The server obtains content items from a personal content collection of a user, analyzes the content items to identify attributes associated with the images, generates rule sets based on the identified attributes, applies the generated rule sets to a plurality of unclassified images, identifies a first set of one or more of the unclassified images, associates each image of the first set with the content items and stores the first set in the personal content collection. A second set of unclassified images is placed in a candidate list to be periodically, without user intervention, monitored and updated for storing in the personal content collection. The generated rule sets are updated after images are stored in the content collection.
1. A method comprising:
at a server having one or more processors and memory storing programs executed by the one or more processors,
obtaining content items from a personal content collection of a user, wherein the content items include images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more categories, groupings, and image relevance scores for a plurality of unclassified images of the user, wherein the generated rule sets include respective weights based on the one or more categories, groupings, and image relevance scores;
applying the generated rule sets to the plurality of unclassified images to determine for each unclassified image of the plurality of unclassified images one or more categories, groupings, and image relevance scores;
determining an aggregate relevance score for each unclassified image, wherein the aggregate relevance score is based on a respective subset of the generated rule sets applied to each unclassified image;
identifying a first set of one or more of the unclassified images with determined aggregate relevance scores satisfying a predetermined relevance criteria, wherein the predetermined relevance criteria includes a respective threshold score for each of the image data, image categories, and image groupings of the content items from a personal content collection;
determining respective categories and groupings for each image of the first set, the respective categories and groupings corresponding to image categories and image groupings of the content items in the personal content collection;
storing each image of the first set in the personal content collection with a corresponding image category and image grouping in accordance with their respective categories and groupings;
identifying a second set of one or more of the unclassified images with determined aggregate relevance scores below the predetermined relevance criteria; and
placing the second set in a candidate list of the personal content collection, the candidate list including one or more candidate images to be reevaluated for storage in the personal content collection with a corresponding image category and image grouping.
2. The method of claim 1 , further comprising notifying the user about the storage of the first set.
3. The method of claim 2 , further comprising prompting the user for approval before storing the first set.
4. The method of claim 1 , further comprising analyzing the content items to detect text within the images.
5. The method of claim 4 , further comprising:
recognizing fully or partially the text within the respective images;
generating image search indexes with the fully or partially recognized text; and
including the search indexes in the generated rule sets.
6. The method of claim 5 , further comprising analyzing the plurality of unclassified images to detect text within the unclassified images.
7. The method of claim 6 , further comprising:
recognizing fully or partially the text within the unclassified images; and
applying the generated rule sets to the plurality of unclassified images to determine for each unclassified image of the plurality of unclassified images one or more categories, groupings, and image relevance scores.
8. The method of claim 1 , further comprising:
periodically without user intervention monitoring the candidate list of the personal content collection;
periodically without user intervention updating respective determined aggregate relevance scores of the one or more candidate images;
identifying a third set of one or more unclassified image of the one or more candidate images with determined aggregate relevance scores satisfying the predetermined relevance criteria;
determining respective categories and groupings for each image of the third set, the respective categories and groupings corresponding to image categories and image groupings of the content items in personal content collection; and
storing each image of the third set in the personal content collection with a corresponding image category and image grouping in accordance with their respective categories and groupings.
9. The method of claim 1 , further comprising updating the respective weights of the generated rule sets after associating each image of the first set with the content items and storing the first set.
10. The method of claim 8 , further comprising updating the respective weights of the generated rule sets after associating each image of the third set with the content items and storing the third set.
11. The method of claim 1 , wherein the associated image information includes text, keywords, user preferences, image content, URLs, location, time, forms, receipts, and handwritten notes.
12. The method of claim 1 , further comprising displaying a user interface on a client device connected to the server, the user interface including a text entry pane for entering note information or a search query, a content collection displaying one or more category or grouping of images, the one or more category or grouping of images displaying one or more of content items, content item notes, or content attributes.
13. A server system, comprising:
one or more processors; and
memory storing one or more instructions that, when executed by the one or more processors, cause the computing system to perform operations including:
obtaining content items from a personal content collection of a user, wherein the content items include images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more categories, groupings, and image relevance scores for a plurality of unclassified images of the user, wherein the generated rule sets include respective weights based on the one or more categories, groupings, and image relevance scores;
applying the generated rule sets to the plurality of unclassified images to determine for each unclassified image of the plurality of unclassified images one or more categories, groupings, and image relevance scores;
determining an aggregate relevance score for each unclassified image, wherein the aggregate relevance score is based on a respective subset of the generated rule sets applied to each unclassified image;
identifying a first set of one or more of the unclassified images with determined aggregate relevance scores satisfying a predetermined relevance criteria, wherein the predetermined relevance criteria includes a respective threshold score for each of the image data, image categories, and image groupings of the content items from a personal content collection;
determining respective categories and groupings for each image of the first set, the respective categories and groupings corresponding to image categories and image groupings of the content items in the grouping personal content collection;
storing each image of the first set in the personal content collection with a corresponding image category and image grouping in accordance with their respective categories and groupings;
identifying a second set of one or more of the unclassified images with determined aggregate relevance scores below the predetermined relevance criteria; and
placing the second set in a candidate list of the personal content collection, the candidate list including one or more candidate images to be reevaluated for storage in the personal content collection with a corresponding image category and image grouping.
14. The server system of claim 13 , wherein the operations further comprise analyzing the content items to detect text within the images.
15. The server system of claim 14 , wherein the operations further comprise:
recognizing fully or partially the text within the respective images;
generating image search indexes with the fully or partially recognized text; and
including the search indexes in the generated rule sets.
16. The server system of claim 13 , wherein the operations further comprise:
periodically without user intervention monitoring the candidate list of the personal content collection;
periodically without user intervention updating respective determined aggregate relevance scores of the one or more candidate images;
identifying a third set of one or more unclassified image of the one or more candidate images with determined aggregate relevance scores satisfying the predetermined relevance criteria;
determining respective categories and groupings for each image of the third set, the respective categories and groupings corresponding to image categories and image groupings of the content items in the personal content collection; and
storing each image of the third set in the personal content collection with a corresponding image category and image grouping in accordance with their respective categories and groupings.
17. The server system of claim 13 , wherein the operations further comprise displaying a user interface on a client device connected to the server, the user interface including a text entry pane for entering note information or a search query, a content collection displaying one or more category or grouping of images, the one or more category or grouping of images displaying one or more of content items, content item notes, or content attributes.
18. A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer, the one or more programs comprising instructions for:
obtaining content items from a personal content collection of a user, wherein the content items include images and associated image information;
analyzing the content items to identify attributes associated with the images, wherein the attributes include one or more of image data, image categories, and image groupings;
generating rule sets based on the identified attributes to be used in determining one or more categories, groupings, and image relevance scores for a plurality of unclassified images of the user, wherein the generated rule sets include respective weights based on the one or more categories, groupings, and image relevance scores;
applying the generated rule sets to the plurality of unclassified images to determine for each unclassified image of the plurality of unclassified images one or more categories, groupings, and image relevance scores;
determining an aggregate relevance score for each unclassified image, wherein the aggregate relevance score is based on a respective subset of the generated rule sets applied to each unclassified image;
identifying a first set of one or more of the unclassified images with determined aggregate relevance scores satisfying a predetermined relevance criteria, wherein the predetermined relevance criteria includes a respective threshold score for each of the image data, image categories, and image groupings of the content items from a personal content collection;
determining respective categories and groupings for each image of the first set, the respective categories and groupings corresponding to image categories and image groupings of the content items in the personal content collection;
storing each image of the first set in the personal content collection with a corresponding image category and image grouping in accordance with their respective categories and groupings;
identifying a second set of one or more of the unclassified images with determined aggregate relevance scores below the predetermined relevance criteria; and
placing the second set in a candidate list of the personal content collection, the candidate list including one or more candidate images to be reevaluated for storage in the personal content collection with a corresponding image category and image grouping.