Systems and methods for image description generation
Systems, methods, and non-transitory computer-readable media can receive an image. One or more concepts depicted in the image are identified based on machine learning techniques. The one or more concepts are filtered based on filtering criteria to identify one or more selected concepts. An image description is generated comprising the one or more selected concepts.
1. A computer-implemented method comprising:
receiving, by a computing system, an image;
identifying, by the computing system, one or more concepts depicted in the image based on machine learning techniques;
filtering, by the computing system, the one or more concepts based on filtering criteria to identify one or more selected concepts;
generating, by the computing system, an image description comprising the one or more selected concepts; and
providing, by the computing system, the image description for presentation to a user.
2. The computer-implemented method of claim 1 , further comprising:
assigning each concept of the one or more concepts a confidence score indicative of a likelihood that the concept is depicted in the image.
3. The computer-implemented method of claim 2 , wherein the filtering criteria comprises one or more confidence score thresholds.
4. The computer-implemented method of claim 3 , wherein the filtering the one or more concepts comprises:
for each concept of the one or more concepts,
querying a whitelist comprising a plurality of concepts and a plurality of confidence score thresholds to determine a confidence score threshold associated with the concept; and
determining whether the confidence score assigned to the concept satisfies the confidence score threshold.
5. The computer-implemented method of claim 4 , wherein the filtering the one or more concepts comprises excluding from the one or more selected concepts any concepts of the one or more concepts that are not listed in the whitelist.
6. The computer-implemented method of claim 1 , further comprising embedding the image description in the image as alternative text.
7. The computer-implemented method of claim 1 , wherein the identifying one or more concepts further comprises identifying one or more concepts based on optical character recognition techniques.
8. The computer-implemented method of claim 1 , further comprising
receiving a request for additional information about the image.
9. The computer-implemented method of claim 8 , further comprising, responsive to receiving the request for additional about the image, presenting additional image information, wherein the additional image information comprises at least one concept of the one or more concepts that was not included in the one or more selected concepts.
10. The computer-implemented method of claim 1 , wherein the providing the image description for presentation to the user comprises providing the image description for audible presentation to the user.
11. A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
receiving an image;
identifying one or more concepts depicted in the image based on machine learning techniques;
filtering the one or more concepts based on filtering criteria to identify one or more selected concepts;
generating an image description comprising the one or more selected concepts; and
providing the image description for presentation to a user.
12. The system of claim 11 , wherein the method further comprises:
assigning each concept of the one or more concepts a confidence score indicative of a likelihood that the concept is depicted in the image.
13. The system of claim 12 , wherein the filtering criteria comprises one or more confidence score thresholds.
14. The system of claim 13 , wherein the filtering the one or more concepts comprises:
for each concept of the one or more concepts,
querying a whitelist comprising a plurality of concepts and a plurality of confidence score thresholds to determine a confidence score threshold associated with the concept; and
determining whether the confidence score assigned to the concept satisfies the confidence score threshold.
15. The system of claim 14 , wherein the filtering the one or more concepts comprises excluding from the one or more selected concepts any concepts of the one or more concepts that are not listed in the whitelist.
16. A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
receiving an image;
identifying one or more concepts depicted in the image based on machine learning techniques;
filtering the one or more concepts based on filtering criteria to identify one or more selected concepts;
generating an image description comprising the one or more selected concepts; and
providing the image description for presentation to a user.
17. The non-transitory computer-readable storage medium of claim 16 , wherein the method further comprises:
assigning each concept of the one or more concepts a confidence score indicative of a likelihood that the concept is depicted in the image.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the filtering criteria comprises one or more confidence score thresholds.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the filtering the one or more concepts comprises:
for each concept of the one or more concepts,
querying a whitelist comprising a plurality of concepts and a plurality of confidence score thresholds to determine a confidence score threshold associated with the concept; and
determining whether the confidence score assigned to the concept satisfies the confidence score threshold.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the filtering the one or more concepts comprises excluding from the one or more selected concepts any concepts of the one or more concepts that are not listed in the whitelist.