Content-based detection and three dimensional geometric reconstruction of objects in image and video data

1. A computer-implemented method of reconstructing an object depicted in a digital image, the method comprising:

detecting a plurality of identifying features of the object depicted in the digital image, wherein the plurality of identifying features are depicted in the digital image and located internally with respect to the object;

reconstructing the digital image of the object within a three dimensional coordinate space based at least in part on some or all of the plurality of identifying features, wherein reconstructing the digital image of the object comprises estimating a homography transform H, wherein estimating the homography transform H comprises an iterative process, and wherein each iteration of the iterative process comprises:

randomly selecting four key points from among the plurality of identifying features of the object;

using a four point algorithm to estimate an i th homography transform H i based on the four key points; and

applying the estimated i th homography transform H i to a set of corresponding key points;

wherein the set of corresponding key points comprises a plurality of point correspondences, each point correspondence including:

an other key point from the digital image and corresponding to one of the plurality of identifying features of the object; and

a corresponding key point from a reference image corresponding to the digital image;

determining one or more outlier key points from among each set of corresponding key points;

identifying, from among each set of corresponding key points, the set of corresponding key points having a lowest number of outlier key points;

defining a set of inlier key points from among the set of corresponding key points having the lowest number of outlier key points, the set of inlier key points excluding the outlier key points determined from among the set of corresponding key points having the lowest number of outlier key points; and

estimating the homography transform H based on the set of inlier key points;

wherein the digital image is a single digital image; and

wherein the plurality of identifying features are detected from within the single digital image.

2. The computer-implemented method as recited in claim 1 , wherein reconstructing the digital image of the object comprises transforming the object to represent dimensions of the object as viewed from an angle normal to the object.

3. The computer-implemented method as recited in claim 1 , wherein reconstructing the digital image of the object is based on four of the plurality of identifying features; and

wherein the four of the plurality of identifying features are randomly selected from among the plurality of identifying features.

4. The computer-implemented method as recited in claim 1 , wherein reconstructing the digital image of the object is based at least in part on applying a four-point algorithm to at least some of the plurality of identifying features.

5. The computer-implemented method as recited in claim 1 , wherein estimating H comprises detecting one or more point correspondences p i ↔P i ′ with p i =(x i ,y i ,1) T .

6. The computer-implemented method as recited in claim 5 , wherein each point correspondence p i ↔P i ′ corresponds to a position p i of one of the plurality of identifying features of the object, and a respective position P i ′ of a corresponding identifying feature of the reconstructed digital image of the object.

7. The computer-implemented method as recited in claim 5 , wherein estimating H comprises normalizing coordinates of some or all of the point correspondences.

8. The computer-implemented method as recited in claim 1 , wherein determining the one or more outlier key points from among each set of corresponding key points comprises:

determining whether each of the plurality of point correspondences fits a transformation model corresponding to the estimated i th homography transform H i ; and, for each of the plurality of point correspondences, either:

designating the other key point of the point correspondence as an outlier key point in response to determining the point correspondence does not fit the transformation model; or

designating the other key point of the point correspondence as an inlier key point in response to determining the point correspondence does fit the transformation model.

9. The computer-implemented method as recited in claim 1 , wherein the plurality of identifying features correspond to boilerplate content of the object.

10. The computer-implemented method as recited in claim 1 , wherein at least one edge of the object is either obscured or missing from the digital image.

11. The computer-implemented method as recited in claim 1 , comprising cropping the reconstructed digital image of the object based at least in part on a projected location of one or more edges of the object within the reconstructed digital image.

12. The computer-implemented method as recited in claim 11 , wherein the projected location of the one or more edges of the object is based at least in part on an estimated homography transform H.

13. The computer-implemented method as recited in claim 1 , comprising classifying the reconstructed digital image of the object.

14. A computer program product for reconstructing an object depicted in a digital image, comprising a non-transitory computer readable medium having stored thereon computer readable program instructions configured to cause a processor, upon execution thereof, to:

detect, using the processor, a plurality of identifying features of the object depicted in the digital image, wherein the plurality of identifying features are depicted in the digital image and located internally with respect to the object;

reconstruct, using the processor, the digital image of the object within a three dimensional coordinate space based at least in part on some or all of the plurality of identifying features, wherein reconstructing the digital image of the object comprises estimating a homography transform H, wherein estimating the homography transform H comprises an iterative process, and wherein each iteration of the iterative process comprises:

randomly selecting four key points from among the plurality of identifying features of the object;

using a four point algorithm to estimate an i th homography transform H i based on the four key points; and

applying the estimated i th homography transform H i to a set of corresponding key points;

wherein the set of corresponding key points comprises a plurality of point correspondences, each point correspondence including:

an other key point from the digital image and corresponding to one of the plurality of identifying features of the object; and

a corresponding key point from a reference image corresponding to the digital image;

determine one or more outlier key points from among each set of corresponding key points;

identify, from among each set of corresponding key points, the set of corresponding key points having a lowest number of outlier key points;

define a set of inlier key points from among the set of corresponding key points having the lowest number of outlier key points, the set of inlier key points excluding the outlier key points determined from among the set of corresponding key points having the lowest number of outlier key points; and

estimate the homography transform H based on the set of inlier key points;

wherein the digital image is a single digital image; and

wherein the plurality of identifying features are detected from within the single digital image.

15. The computer program product as recited in claim 14 , wherein reconstructing the digital image of the object is based on four of the plurality of identifying features; and

wherein the four of the plurality of identifying features are randomly selected from among the plurality of identifying features.

16. The computer program product as recited in claim 14 , wherein reconstructing the digital image of the object is based at least in part on applying a four-point algorithm to at least some of the plurality of identifying features.

17. A system for reconstructing an object depicted in a digital image, comprising a processor and logic embodied with the processor, the logic being configured to cause the processor, upon execution thereof, to:

detect a plurality of identifying features of the object depicted in the digital image, wherein the plurality of identifying features are depicted in the digital image and located internally with respect to the object;

reconstruct the digital image of the object within a three dimensional coordinate space based at least in part on some or all of the plurality of identifying features, wherein reconstructing the digital image of the object comprises estimating a homography transform H, wherein estimating the homography transform H comprises an iterative process, and wherein each iteration of the iterative process comprises:

randomly selecting four key points from among the plurality of identifying features of the object;

using a four point algorithm to estimate an i th homography transform H i based on the four key points; and

applying the estimated i th homography transform H i to a set of corresponding key points;

wherein the set of corresponding key points comprises a plurality of point correspondences, each point correspondence including:

an other key point from the digital image and corresponding to one of the plurality of identifying features of the object; and

a corresponding key point from a reference image corresponding to the digital image;

determine one or more outlier key points from among each set of corresponding key points;

identify, from among each set of corresponding key points, the set of corresponding key points having a lowest number of outlier key points;

define a set of inlier key points from among the set of corresponding key points having the lowest number of outlier key points, the set of inlier key points excluding the outlier key points determined from among the set of corresponding key points having the lowest number of outlier key points; and

estimate the homography transform H based on the set of inlier key points;

wherein the digital image is a single digital image; and

wherein the plurality of identifying features are detected from within the single digital image.