Systems and methods for classifying objects in digital images captured using mobile devices
In one embodiment, a method includes receiving a digital image captured by a mobile device; and using a processor of the mobile device: generating a first representation of the digital image, the first representation being characterized by a reduced resolution; generating a first feature vector based on the first representation; comparing the first feature vector to a plurality of reference feature matrices; classifying an object depicted in the digital image as a member of a particular object class based at least in part on the comparing; and determining one or more object features of the object based at least in part on the particular object class. Corresponding systems and computer program products are also disclosed.
1. A computer-implemented method, comprising:
receiving a digital image captured by a mobile device; and
using a processor of the mobile device:
generating a first representation of the digital image, the first representation being characterized by a reduced resolution;
generating a first feature vector based on the first representation;
comparing the first feature vector to a plurality of reference feature matrices;
classifying an object depicted in the digital image as a member of a particular object class based at least in part on the comparing;
determining one or more object features of the object based at least in part on the particular object class; and
detecting the object within the digital image based on the one or more object features, wherein the detecting comprises directly detecting an object color profile within the digital image rather than detecting a transition from a first color profile to a second color profile.
2. The method as recited in claim 1 , comprising:
receiving an object class identification code; and
retrieving the one or more object features based on the object class identification code.
3. The method as recited in claim 1 , wherein the one or more object features comprise a location of a subregion of the digital image, the subregion depicting one or more reference objects.
4. The method as recited in claim 3 , wherein the location corresponds to a particular object orientation state.
5. The method as recited in claim 1 , wherein the one or more object features are selected from a group consisting of: an object color profile; and an object subregion color profile.
6. The method as recited in claim 1 , wherein the one or more object features comprise a location of text depicted within the object.
7. The method as recited in claim 1 , wherein the one or more object features are selected from a group consisting of: one or more dimensions of the object; an object shape; an object color; and one or more reference features of the object class.
8. The method as recited in claim 1 , comprising rectangularizing the object based at least in part on the one or more object features.
9. The method as recited in claim 8 , wherein the one or more object features comprise one or more known characteristics describing a true configuration of the object.
10. The method as recited in claim 9 , wherein the rectangularizing comprises transforming the object from a native appearance within the digital image to the true configuration based at least in part on the known characteristics, rather than estimating the true configuration based on the native appearance of the object within the digital image.
11. The method as recited in claim 1 , comprising predicting an identity of text depicted on the object using optical character recognition (OCR), wherein the predicting is based at least in part on the particular object class.
12. The method as recited in claim 11 , comprising modifying at least one identity predicted using OCR, the modifying being based at least in part on an expected format of text depicted on the object; and
wherein the expected format is determined based on the particular object class.
13. The method as recited in claim 1 , comprising cropping the digital image based at least in part on the one or more object features.
14. A computer program product, comprising a non-transitory computer readable medium having stored thereon program instructions readable/executable by a processor of a mobile device to cause the processor to:
generate a first representation of a digital image, the first representation being characterized by a reduced resolution;
generate a first feature vector based on the first representation; compare the first feature vector to a plurality of reference feature matrices;
classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparing; and
determine one or more object features of the object based at least in part on the particular object class; and
detect the object within the digital image based on the one or more object features, wherein the detecting comprises directly detecting an object color profile within the digital image rather than detecting a transition from a first color profile to a second color profile.
15. The computer program product as recited in claim 14 , wherein the one or more object features are selected from a group consisting of: one or more dimensions of the object; an object shape; an object color; and one or more reference features of the object class.
16. A computer-implemented method, comprising:
receiving a digital image captured by a mobile device; and
using a processor of the mobile device:
generating a first representation of a digital image, the first representation being characterized by a reduced resolution;
generating a first feature vector based on the first representation;
comparing the first feature vector to a plurality of reference feature matrices;
classifying an object depicted in the digital image as a member of a particular object class based at least in part on the comparing; and
determining one or more object features of the object based at least in part on the particular object class;
predicting an identity of text depicted on the object using optical character recognition (OCR), wherein the predicting is based at least in part on the particular object class; and
modifying at least one identity predicted using OCR, the modifying being based at least in part on an expected format of text depicted on the object, wherein the expected format is determined based on the particular object class.
17. A computer program product, comprising a non-transitory computer readable medium having stored thereon program instructions readable/executable by a processor of a mobile device to cause the processor to:
generate a first representation of a digital image, the first representation being characterized by a reduced resolution;
generate a first feature vector based on the first representation;
compare the first feature vector to a plurality of reference feature matrices;
classify an object depicted in the digital image as a member of a particular object class based at least in part on the comparing; and
determine one or more object features of the object based at least in part on the particular object class;
predict an identity of text depicted on the object using optical character recognition (OCR), wherein the predicting is based at least in part on the particular object class; and
modify at least one identity predicted using OCR, the modifying being based at least in part on an expected format of text depicted on the object, wherein the expected format is determined based on the particular object class.