IP Library Patent Application 16518501
Patent Application
App. No. 16/518,501

AUTOMATIC VIEW MAPPING FOR SINGLE-IMAGE AND MULTI-VIEW CAPTURES

Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US None
App. No.
16/518,501
Abstract

A three-dimensional (3D) skeleton may be determined based on a plurality of vertices and a plurality of faces in a two-dimensional (2D) mesh in a top-down image of an object. A correspondence mapping between a designated perspective view image and the top-down object image may be determined based on the 3D skeleton. The correspondence mapping may link a respective first location in the top-down object image to a respective second location in the designated perspective view image for each of a plurality of points in the designated perspective view image. A top-down mapped image of the object may be created by determining a first respective pixel value for each of the first locations, with each first respective pixel value being determined based on a second respective pixel value for the respective second location linked with the respective first location via the correspondence mapping.

Claims (41)

1 . A method comprising:

determining via a processor a three-dimensional (3D) skeleton based at least in part on a plurality of vertices and a plurality of faces in a two-dimensional (2D) mesh in a top-down image of an object;

determining via the processor a designated correspondence mapping between a designated perspective view image and the top-down object image based at least in part on the 3D skeleton, the designated correspondence mapping linking a respective first location in the top-down object image to a respective second location in the designated perspective view image for each of a plurality of points in the designated perspective view image;

creating a top-down mapped image of the object by determining a first respective pixel value for each of the first locations, each first respective pixel value being determined based on a second respective pixel value for the respective second location linked with the respective first location via the designated correspondence mapping; and

storing the top-down mapped image on a storage device.

2 . The method recited in claim 1 , the method further comprising:

determining one or more visibility angles for a designated one of the vertices, wherein the 3D skeleton is determined based in part on the one or more visibility angles.

3 . The method recited in claim 1 , the method further comprising:

based on the designated correspondence mapping, determining a tag location mapping that maps the location of a visual tag from a standard perspective view image to the top-down mapped image, and from the top-down mapped image to the designated perspective view image.

4 . The method recited in claim 1 , the method further comprising:

determining an object type based on the designated perspective view image.

5 . The method recited in claim 4 , wherein the 3D skeleton is determined at least in part based on the object type.

6 . The method recited in claim 4 , the method further comprising:

determining an object subtype based at least in part on the 3D skeleton.

7 . The method recited in claim 1 , wherein each respective second location indicates a value in a 2D matrix that overlays the perspective view image.

8 . The method recited in claim 1 , wherein each respective first location indicates a value in a 2D matrix that overlays the top-down object image.

9 . The method recited in claim 1 , wherein the designated perspective view image is one of a plurality of images in a multi-view capture, the multi-view capture including a plurality of images of the object, each of the images of the object being captured from a different perspective view.

10 . The method recited in claim 9 , wherein a respective correspondence mapping is determined for each of the plurality of images, and wherein creating a top-down mapped image of the object comprises aggregating pixel values associated with the plurality of correspondence mappings.

11 . The method recited in claim 1 , wherein the object is a vehicle, and wherein the top-down object image depicts each of a left vehicle door, a right vehicle door, and a windshield.

12 . A computing system comprising:

a memory module configured to store a three-dimensional ( 3 D) skeleton based at least in part on a plurality of vertices and a plurality of faces in a two-dimensional ( 2 D) mesh in a top-down image of an object;

a processor configured to:

determine a designated correspondence mapping between a designated perspective view image and the top-down object image based at least in part on the 3D skeleton, the designated correspondence mapping linking a respective first location in the top-down object image to a respective second location in the designated perspective view image for each of a plurality of points in the designated perspective view image, and

create a top-down mapped image of the object by determining a first respective pixel value for each of the first locations, each first respective pixel value being determined based on a second respective pixel value for the respective second location linked with the respective first location via the designated correspondence mapping; and

a storage device configured to store the top-down mapped image.

13 . The computing system recited in claim 12 , wherein the processor is further configured to

14 . The computing system recited in claim 13 , wherein the 3D skeleton is determined based in part on the one or more visibility angles.

15 . The computing system recited in claim 12 , wherein the processor is further configured to determine an object type based on the designated perspective view image.

16 . The computing system recited in claim 15 , wherein the 3D skeleton is determined at least in part based on the object type.

17 . The computing system recited in claim 16 , wherein the processor is further configured to:

determine an object subtype based at least in part on the 3D skeleton.

18 . The computing system recited in claim 12 , wherein the designated perspective view image is one of a plurality of images in a multi-view capture, the multi-view capture including a plurality of images of the object, each of the images of the object being captured from a different perspective view, wherein the multi-view capture includes inertial measurement unit (IMU) data collected from an IMU in a mobile phone.

19 . One or more non-transitory computer-readable method having instructions stored thereon for performing a method, the method comprising:

determining via a processor a three-dimensional (3D) skeleton based at least in part on a plurality of vertices and a plurality of faces in a two-dimensional (2D) mesh in a top-down image of an object;

determining via the processor a designated correspondence mapping between a designated perspective view image and the top-down object image based at least in part on the 3D skeleton, the designated correspondence mapping linking a respective first location in the top-down object image to a respective second location in the designated perspective view image for each of a plurality of points in the designated perspective view image;

creating a top-down mapped image of the object by determining a first respective pixel value for each of the first locations, each first respective pixel value being determined based on a second respective pixel value for the respective second location linked with the respective first location via the designated correspondence mapping; and

storing the top-down mapped image on a storage device.

20 . The one or more non-transitory computer readable media recited in claim 19 , the method further comprising:

determining one or more visibility angles for a designated one of the vertices, wherein the 3D skeleton is determined based in part on the one or more visibility angles;

determining an object type based on the designated perspective view image, wherein the 3D skeleton is determined at least in part based on the object type; and

determining an object subtype based at least in part on the 3D skeleton, wherein the designated perspective view image is one of a plurality of images in a multi-view capture, the multi-view capture including a plurality of images of the object, each of the images of the object being captured from a different perspective view, the multi-view capture including inertial measurement unit (IMU) data collected from an IMU in a mobile phone.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2019
From: HOLZER, STEFAN JOHANNES JOSEF; MUNARO, MATTEO; LIAUDANSKAS, AIDAS; KAR, ABHISHEK; CHANDE, KRUNAL KETAN; RUSU, RADU BOGDAN
To: FYUSION, INC.
Reel/Frame 049825/0153 →