Methods and apparatus for detecting features of scanned images, associating tags with images and/or using tagged images
Methods and apparatus for using one or more feature recognition system(s) to identify features in a scanned image and to associate tags with identified features are described. Probabilities of a feature being present are taken into consideration in some embodiments. In various embodiments once a tag has been determined as corresponding to a feature which has been confirmed as being in the scanned image synonyms for the tag word are identified and also associated with the scanned image. By using results of multiple automated feature recognition systems and generating overall probabilities that a feature is present in an image more reliable tagging can be implemented in an automated manner than in system which rely on a single AI system without the need for extensive operator input with respect to identification of all features in the image which are to be tagged.
1. A method of processing one or more images, the method comprising:
processing a scanned image using a first artificial intelligence (AI) system to detect features in the scanned image corresponding to word tags and to generate corresponding probabilities that the detected features are present in the scanned image, said features including a first feature, said first feature being an image of a first object;
processing the scanned image using a second AI system to detect features in the scanned image corresponding to word tags and to generate corresponding probabilities that the detected features are present in the scanned image;
identifying consensus features in the scanned image, consensus features being features detected by multiple AI systems which processed said scanned image, said multiple AI systems including at least the first AI system and the second AI system, said consensus features including at least a first consensus feature, said first consensus feature being said first feature;
determining for at least the first identified consensus feature, an overall probability, based on a first probability generated by the first AI system for the first consensus feature and on a second probability generated by the second AI system for the first consensus feature; and
determining, based on the overall probability, whether or not to include a first word tag corresponding to the first consensus feature in a tag list corresponding to the scanned image, said first word tag being a word.
2. The method of claim 1 , wherein determining for at least the first identified consensus feature, the overall probability includes:
computing a first posterior density probability using as a first input a probability value generated based on an average detection rate with respect to identification of features associated the first AI system and a probability value generated based on an average detection rate with respect to identification of features associated the second AI system and using the probability generated by the first AI system for the first consensus feature as a second input.
3. The method of claim 2 , wherein determining for at least the first identified consensus feature, the overall probability further includes:
computing a second posterior density probability using as a first input the first posterior density probability and using the probability generated by the second AI system for the first consensus feature as a second input.
4. The method of claim 3 , wherein determining for at least the first identified consensus feature, the overall probability further includes:
computing a third posterior density probability using as a first input the second posterior density probability and using the probability generated by a third AI system for the first consensus feature as a second input.
5. The method of claim 4 , further comprising:
using said third posterior density as the overall probability for the first consensus feature.
6. The method of claim 5 , further comprising:
storing the tag list associated with the scanned image.
7. The method of claim 6 , further comprising:
receiving a query from a user; and
using the stored tag list to determine if a word in a search corresponds to the scanned image.
8. The method of claim 7 , further comprising:
responding to a search by providing the scanned image to a display and displaying the image.
9. The method of claim 1 , wherein the first and second AI systems are different Internet based systems, the method further comprising:
sending, from a home appliance the first scanned image to the first AI and second AI systems via an Internet communications network.
10. The method of claim 1 ,
wherein said steps of processing the scanned image using the first AI system and processing the scanned image using the second AI system are performed in parallel, the method further comprising:
receiving at an apparatus in a home, from the first AI system a first set of information indicating features detected in the first image, word tags corresponding to the detected features and probabilities that the detected features are present in the scanned image; and
receiving, at the apparatus in the home, from the second AI system a second set of information indicating features detected in the first image by the second AI system, word tags corresponding to the detected features, and probabilities that the detected features are present in the scanned image.
11. The method of claim 10 ,
wherein the first AI system and second AI system are located outside the home; and
wherein said steps of identifying consensus features in the scanned image, determining for at least a first identified consensus feature an overall probability, and determining whether or not to include a first word tag corresponding to the first consensus feature in a tag list corresponding to the scanned image are performed by said apparatus in the home.
12. The method of claim 1 , further comprising:
receiving from the first AI system information indicating the first probability determined by the first AI system that the first feature is present in the scanned image and a word tag provided by the first AI system corresponding to the first feature.
13. The method of claim 1 , further comprising:
receiving from the second AI system information indicating the second probability determined by the second AI system that the first feature is present in the scanned image and a word tag provided by the second AI system corresponding to the first feature.
14. The method of claim 13 , wherein said object is an image of animal.
15. The method of claim 14 wherein said animal is a dog.
16. The method of claim 15 , wherein the first word tag is the word dog.
17. The method of claim 15 wherein said tag list includes multiple words or word phrases corresponding to the first feature, said tag list including the word dog and a name for a breed of dog.
18. An apparatus for processing one or more images, the apparatus comprising:
a receiver configured to receive: i) first information indicating a first list of features detected by a first artificial intelligence (AI) system to be present in a scanned image and corresponding probabilities determined by the first AI system that the detected features are present in the scanned image, said first list of detected features including a first feature, said first feature being an image of a first object; and ii) second information indicating a second list of features detected by a second AI system to be present in the scanned image and corresponding probabilities determined by the second AI system that the detected features are present in the scanned image, said second list of features including the first feature;
memory storing said first and second information and said scanned image; and
a processor, configured to control the apparatus to:
identify consensus features in the scanned image, consensus features being features detected by multiple AI systems which processed said scanned image, said multiple AI systems including at least the first AI system and the second AI system, said consensus features including at least a first consensus feature, said first consensus feature being said first feature;
determine for at least the first identified consensus feature, an overall probability, based on a first probability generated by the first AI system for the first consensus feature and on a second probability generated by the second AI system for the first consensus feature; and
determine, based on the overall probability, whether or not to include a first word tag corresponding to the first consensus feature in a tag list corresponding to the scanned image, said first word tag being a word.
19. The apparatus of claim 18 , wherein the processor is configured, as part of determining, for at least the first identified consensus feature, the overall probability, to control the apparatus to:
compute a first posterior density probability using as a first input, a probability value generated based on an average detection rate with respect to identification of features associated the first AI system and a probability value generated based on an average detection rate with respect to identification of features associated the second AI system and using the probability generated by the first AI system for the first consensus feature as a second input.
20. The apparatus of claim 19 , wherein the processor is further configured, as part of controlling the apparatus to determine, for at least the first identified consensus feature, the overall probability, to control the apparatus to:
compute a second posterior density probability using as a first input the first posterior density probability and using the probability generated by the second AI system for the first consensus feature as a second input.
21. The apparatus of claim 20 , wherein the processor is further configured to control the apparatus, as part of determining, for at least a first identified consensus feature, the overall probability, to:
compute a third posterior density probability using as a first input the second posterior density probability and using the probability generated by a third AI system for the first consensus feature as a second input.
22. The apparatus of claim 21 , wherein said processor is further configured to control the apparatus to:
use said third posterior density as the overall probability for the first consensus feature.
23. The apparatus of claim 22 , wherein the memory stores the tag list associated with the scanned image.
24. The apparatus of claim 23 , further comprising:
a user input device configured to receive a query from a user; and
wherein the processor is further configured to:
use the stored tag list to determine if a word in a search corresponds to the scanned image.
25. The apparatus of claim 24 , wherein the processor is further configured to control the apparatus to:
respond to a search by providing the scanned image to a display and display the image.