System and method for scene rectification via homography estimation
Disclosed herein is a system and method for performing pose-correction on images containing objects within a scene, or the entire scene, to compensate for off-centered camera views. The system and method generates a more frontal view of the object or scene by applying planar homography by identifying corner endpoints of the object or the scene and repositioning the corner endpoints to provide a more frontal view. The pose-corrected scene may then be input to an object detector to determine a location of a bounding box of an object-of-interest which would be more accurate than a bounding box from the original off-centered image.
1. A method comprising:
collecting an image containing one or more objects-of-interest;
determining that objects-of-interest appearing in the image have been captured from an off-centered point-of-view;
submitting the image to a trained object detector to determine bounding boxes enclosing objects-of-interest in the image;
for each object-of-interest in the image:
identifying corner endpoints of the object-of-interest;
applying homography to reposition the identified endpoints; and
generating a novel view of the object-of-interest based on the repositioned endpoints, the novel view comprising a more frontal view of the object-of-interest; and
submitting the bounding boxes to one or more downstream tasks.
2. The method of claim 1 , wherein the objects-of-interest within the image are identified via a trained object detector.
3. The method of claim 2 , wherein the step of determining that the objects-of-interest within the image are off-centered comprises:
submitting the objects identified by the trained object detector to a machine learning model trained to detect objects that have been captured from an off-centered point-of-view.
4. The method of claim 1 wherein the one or more downstream tasks include a classifier for identifying the objects-of-interest.
5. A system for performing pose correction on an image captured from an off-centered point-of-view comprising:
a processor; and
software that, when executed by the processor, cause the system to:
collect an image containing one or more objects-of-interest;
determine that one or more objects-of-interest appearing in the image have been captured from an off-centered point-of-view;
submit the image to a trained object detector to determine bounding boxes enclosing objects-of-interest in the image;
for each object-of-interest:
identify corner endpoints of the object-of-interest;
apply homography to reposition the identified endpoints; and
generate a novel view of the object-of-interest based on the repositioned endpoints, the novel view comprising a more frontal view of the object-of-interest; and
submit the bounding boxes to one or more downstream tasks.
6. The system of claim 5 , wherein the objects-of-interest within the image are identified via a trained object detector.
7. The system of claim 6 , the software performing the step of determining that the objects-of-interest within the image are off-centered by causing the system to:
submit the objects identified by the trained object detector to a machine learning model trained to detect objects captured from an off-centered point-of-view; and
receiving an indication that the objects in the image were captured from an off-centered point-of-view.
8. The system of claim 5 wherein the one or more downstream tasks include a classifier for identifying the objects-of-interest.