Image-based CAPTCHA exploiting context in object recognition
Techniques for an image-based CAPTCHA for object recognition are described. The disclosure describes adding images to a database by collecting images by querying descriptive keywords to an image search engine or crawling images from the Internet. The disclosure describes generating the image-based CAPTCHA. The image is retrieved from the database, along with objects having significant values. An object is cropped from its image. The portion on the image where the object has been cropped is filled with image inpainting. The process obtains other objects from the database. The object is mixed among the other objects to form a set of candidate objects. A user is asked to select “the object” from the set of candidate objects that fits or matches the image. The image-based CAPTCHA evaluates whether a response, the selection, is from a human or a bot.
1. A method at least partially controlled by a computing device, the method comprising:
segmenting an image into objects;
assigning a value to each of the segmented objects, wherein the value is an estimation of a significance of an object being recognized by humans;
identifying an object such that the image surrounding the object visually associates the object as being a part of the image;
selecting the object for the object recognition;
cropping the object from the image, wherein the object is expanded beyond a boundary of the image without conveying a contour of the object;
filling a region on the image where the object has been cropped; and
generating candidate objects that have similar low level features to the object cropped from the image, wherein the low level features are features that are defined without using human perception.
2. The method as recited in claim 1 , wherein the image segmented into objects has an image size that is within a predetermined image size range.
3. The method as recited in claim 1 , wherein segmenting the image into objects comprises:
dividing the image into segments;
merging neighboring segments that have low level features that are similar to the low level features of the segments;
determining when the segment is smaller or larger than a predetermined size:
in response to determining that the segment is smaller than the predetermined size and does not fit a shape requirement, merging the segment with a neighboring segment; or
in response to determining that the segment is larger than the predetermined size, partitioning the segment into two separate segments by evaluating a shape and an area of resulting segment and to separate the segment along parts that are different in appearance; and
classifying the segments into foreground segments and background segments, wherein the foreground segments are used as the objects for object recognition.
4. The method as recited in claim 1 , wherein cropping the object comprises:
cropping the object from the image; and
generating a buffer region to surround the object that that has been cropped from the image by expanding a previously cropped region on the image.
5. The method as recited in claim 1 , wherein selecting the object comprises at least one of:
identifying the object based at least in part on an equal probability that the objects have a same probability of being selected;
identifying the object having a significant value based at least in part on a weighted probability that the object with a higher significance value has a greater probability of being selected; and
identifying the object having the significant value based at least in part on a probability that the object to be identified follows a probability distribution of significant values of the objects.
6. The method as recited in claim 1 , wherein the image is procured by:
crawling images from the Internet; or
querying descriptive words using an image search engine.
7. The method as recited in claim 1 , further comprising scaling sizes of the image and the set of candidate objects in accordance with a corresponding scaling factor.
8. The method as recited in claim 1 , further comprising
presenting a user interface to illustrate the image and a set of candidate objects, wherein the set of candidate objects comprises the object cropped from the image and the other candidate objects;
receiving input at the user interface indicating an item selected from the set of candidate objects; and
evaluating the selected item for the object recognition, wherein:
in response to determining that the selected item does not match as being part of the image, access is denied to an application; or
in response to determining that the selected item matches as being part of the image, access is granted to the application.
9. The method as recited in claim 1 , further comprising:
receiving a first input at the user interface indicating an item selected from the set of candidate objects;
evaluating the selected item for the object recognition, wherein:
in response to determining that the selected item does not match as being part of the image, access is denied to an application; or
in response to determining that the selected item matches as being part of the image, access is granted to the application;
receiving a second input at the user interface indicating an aligned position of the item selected from the set of candidate objects overlaid on the image; and
evaluating the aligned position of the item selected, wherein:
in response to determining that the selected item does not match a position for a cropped object in the image, access is denied to the application; or
in response to determining that the selected item matches the position for the cropped object in the image, access is granted to the application.
10. A computer storage device storing computer-executable instructions that, when executed on one or more processors, perform operations comprising:
segmenting an image into objects;
assigning a value to each of the segmented objects, wherein the value is corresponds to a likelihood of an object being recognized by humans;
identifying an object such that the image surrounding the object visually associates the object as being a part of the image;
selecting the object for the object recognition;
cropping the object from the image, wherein the object is expanded beyond a boundary of the image without conveying a contour of the object;
filling a region on the image where the object has been cropped; and
generating candidate objects that have similar low level features to the object cropped from the image, wherein the low level features are features that are determined to be recognizable by a computer.
11. The computer storage device of claim 10 , wherein the image segmented into objects has an image size that is within a predetermined image size range.
12. The computer storage device of claim 10 , wherein segmenting the image into objects comprises:
dividing the image into segments;
merging neighboring segments that have low level features that are similar to the low level features of the segments;
determining when the segment is smaller or larger than a predetermined size:
in response to determining that the segment is smaller than the predetermined size and does not fit a shape requirement, merging the segment with a neighboring segment; or
in response to determining that the segment is larger than the predetermined size, partitioning the segment into two separate segments by evaluating a shape and an area of resulting segment and to separate the segment along parts that are different in appearance; and
classifying the segments into foreground segments and background segments, wherein the foreground segments are used as the objects for object recognition.
13. The computer storage device of claim 10 , wherein the cropping of the object from the image comprises:
cropping the object from the image; and
generating a buffer region to surround the object that that has been cropped from the image by expanding a previously cropped region on the image.
14. The computer storage device of claim 10 , wherein selecting the object comprises at least one of:
identifying the object based at least in part on an equal probability that the objects have a same probability of being selected;
identifying the object having a significant value based at least in part on a weighted probability that the object with a higher significance value has a greater probability of being selected; and
identifying the object having the significant value based at least in part on a probability that the object to be identified follows a probability distribution of significant values of the objects.
15. The computer storage device of claim 10 , wherein the image is procured by:
crawling images from the Internet; or
querying descriptive words using an image search engine.
16. The computer storage device of claim 10 , wherein the operations performed by the execution of the computer-executable instructions by the one or more processors further comprise scaling sizes of the image and the set of candidate objects by a in accordance with a corresponding scaling factor.
17. The computer storage device of claim 10 , wherein the operations performed by the execution of the computer-executable instructions by the one or more processors further comprise:
presenting a user interface to illustrate the image and a set of candidate objects, wherein the set of candidate objects comprises the object that is cropped from the image and the other candidate objects;
receiving input at the user interface indicating an item selected from the set of candidate objects; and
evaluating the selected item for the object recognition, wherein:
in response to determining that the selected item does not match as being part of the image, access is denied to an application; or
in response to determining that the selected item matches as being part of the image, access is granted to the application.
18. The computer storage device of claim 10 , wherein the operations performed by the execution of the computer-executable instructions by the one or more processors further comprise:
receiving a first input at the user interface indicating an item selected from the set of candidate objects;
evaluating the selected item for the object recognition, wherein:
in response to determining that the selected item does not match as being part of the image, access is denied to an application; or
in response to determining that the selected item matches as being part of the image, access is granted to the application; and
receiving a second input at the user interface indicating an aligned position of the item selected from the set of candidate objects overlaid on the image; and
evaluating the aligned position of the item selected, wherein:
in response to determining that the selected item does not match a position for a cropped object in the image, access is denied to the application; or
in response to determining that the selected item matches the position for the cropped objet in the image, access is granted to the application.
19. An apparatus comprising:
a memory; and
one or more processors, wherein the one or more processors are configured to perform operations of:
segmenting an image into objects;
assigning a value to each of the segmented objects, wherein the value is associated with a likelihood of an object being perceived by humans;
identifying significant objects of the image having assigned values indicating that an object would be perceived by humans;
comparing each of the significant objects with a remainder of the image to determine if a corresponding significant object has low level features associated with a portion of the remaining image;
removing significant objects determined to have low level features associated with a portion of the remaining image;
selecting a significant object for the object recognition;
cropping the selected object from the image;
filling a region on the image where the selected object has been cropped; and
generating a set of candidate objects that have similar low level features to the object cropped from the image, wherein the low level features are determined to be recognizable by a computer.
20. The apparatus recited in claim 19 , wherein the image segmented into objects has an image size that is within a predetermined image size range.