Pose estimation and correction
Various embodiments are directed to a Pose Correction Engine (“Engine”). The Engine generates a reference image of the object of interest. The reference image portrays the object of interest oriented according to a first pose. The Engine receives a source image of an instance of the object. The source image portrays the instance of the object oriented according to a variation of the first pose. The Engine determines a difference between the first pose of the reference image and the variation of the first pose of the source image. The Engine identifies, based on the determined difference, one or portions of a three-dimensional (3D) map of a shape of the object obscured by the variation of the first pose portrayed in the source image. The Engine generates a pose corrected image of the instance of the object that portrays at least a portion of the source image and at least the identified portion of the 3D map of the shape of the object.
1. A computer-implemented method, comprising:
receiving a plurality of source images of an object placed at a physical structure associated with a plurality of image capture devices, a first source image associated with a first image capture device positioned relative to placement of the object at the physical structure;
identifying a reference image portraying the object according to a second pose, a first pose of the object portrayed in the first source image comprising a variation of the second pose of the reference image;
generating a three-dimensional (3D) map of a shape of the object based on the first source image;
generating a 3D reference map of the object based on the reference image;
aligning respective portions of the 3D map of the shape of the object with respective portions of the 3D reference map;
identifying at least one obscured portion of the shape of the object that corresponds to a respective misalignment between the 3D map of the shape of the object and the 3D reference map;
generating one or more pose correction parameters, based on the positioning of the first image capture device at the physical structure, for each received additional source image associated with a respective image capture device at the physical structure; and
generating a pose corrected image of the object, via utilization of the one or more pose correction parameters, that includes at least a portion of the first source image and at least one identified obscured portion of the shape of the object.
2. The computer-implemented method as in claim 1 , wherein identifying a reference image comprises:
prior to identifying at least one obscured portion:
generating a plurality of reference images of the object, wherein each respective reference image portrays the object according to a different predefined pose;
selecting, from the plurality of reference image, a reference image portraying a predefined pose of the object related to the first pose of the first source image.
3. The computer-implemented method as in claim 1 , wherein generating the 3D map of a shape of the object based on the source image comprises:
generating a segmented mask image of the first source image (“segmented source image”); and
generating a depth map image based on the segmented source image (“depth map source image”).
4. The computer-implemented method as in claim 3 , wherein generating a 3D reference map of the object based on the reference image comprises:
identifying one or more scaling factors associated with the first source image;
generating a segmented mask image of the reference image (“segmented reference image”);
generating a depth map image based on the segmented reference image (“depth map reference image”); and
generating a scaled-depth map reference image by modifying the depth map reference image according to the one or more identified scaling factors.
5. The computer-implemented method as in claim 4 , wherein aligning respective portions of the 3D map of the shape of the object comprises:
predicting a source 3D map of the shape of the object based on the depth map source image;
predicting a reference 3D map of the object based on the scaled-depth map reference image;
projecting each of the source 3D map and the reference 3D map into a 3D space; and
aligning the projected source and reference 3D maps to identify one or more differences, wherein the one or more respective difference comprises a misalignment between a respective section of the source 3D map with a corresponding section of the reference 3D map.
6. The computer-implemented method as in claim 5 , wherein generating one or more pose correction parameters, comprises:
determining at least one of a rotation parameter and a translation parameter based on the first source image;
determining additional rotation parameters and translation parameters for at least one particular additional source image based on the rotation parameter and the translation parameter for the first source image; and
applying the respective parameters to the section of the source 3D map to generate a portion of the pose-corrected image.
7. The computer-implemented method as in claim 6 , wherein determining additional rotation parameters and translation parameters for at least one additional source image comprises:
determining the additional rotation parameters and the translation parameters based on the position of the first image capture device relative to the object and further relative to a position in the physical structure of a particular image capture device that generated the particular additional source image.