IP Library › Granted Patent US 10,372,968
Granted Patent B2
US 10,372,968 · App. 15/192,857 · Granted Aug 6, 2019

Object-focused active three-dimensional reconstruction

Inventors: Aliakbar Aghamohammadi (San Diego, CA); Seyed Hesameddin Najafi Shoushtari (San Diego, CA); Regan Blythe Towal (La Jolla, CA)
Assignee: QUALCOMM Incorporated
G06K9/00201B25J9/163B25J9/1697G05D1/0088G06K9/4604G06K9/52G06T7/13G06T7/529G06T7/60Y10S901/47Y10S901/50
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,372,968
App. No.
15/192,857
Granted
Aug 6, 2019
Kind
B2
Abstract

A method for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning includes recognizing and localizing an object in a two-dimensional (2D) image. The method also includes computing 3D depth maps for the localized object. A 3D object map is constructed from the depth maps. A sampling based structure is grown around the 3D object map and a cost is assigned to each edge of the sampling based structure. The sampling based structure may be searched to determine a lowest cost sequence of edges that may, in turn be used to guide the robot.

Claims (62)

1. A method for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:

identifying an object of interest;

searching for the object of interest in an environment comprising a plurality of objects;

recognizing and localizing the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;

constructing a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;

growing a sampling based structure around the 3D object map;

assigning a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;

searching the sampling based structure to determine a lowest cost sequence of edges; and

guiding the robot through the environment based on the lowest cost sequence of edges.

2. The method of claim 1 , further comprising:

computing a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and

constructing the 3D object map from the plurality of 3D depth maps.

3. The method of claim 1 , further comprising guiding the robot through the environment based on texture information of the object of interest.

4. The method of claim 1 , further comprising guiding the robot through the environment based on importance weights assigned to different portions of the object of interest.

5. The method of claim 1 , further comprising guiding the robot through the environment by incrementally creating a sampling based motion planning framework.

6. An apparatus for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:

a memory; and

at least one processor coupled to the memory, the at least one processor configured:

to identify an object of interest;

to search for the object of interest in an environment comprising a plurality of objects;

to recognize and localize the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;

to construct a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;

to grow a sampling based structure around the 3D object map;

to assign a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;

to search the sampling based structure to determine a lowest cost sequence of edges; and

to guide the robot through the environment based on the lowest cost sequence of edges.

7. The apparatus of claim 6 , in which the at least one processor is further configured:

to compute a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and

to construct the 3D object map from the plurality of 3D depth maps.

8. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment based on texture information of the object of interest.

9. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment based on importance weights assigned to different portions of the object of interest.

10. The apparatus of claim 6 , in which the at least one processor is further configured to guide the robot through the environment by incrementally creating a sampling based motion planning framework.

11. An apparatus for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, comprising:

means for identifying an object of interest;

means for searching for the object of interest in an environment comprising a plurality of objects;

means for recognizing and localizing the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;

means for constructing a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;

means for growing a sampling based structure around the 3D object map;

means for assigning a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;

means for searching the sampling based structure to determine a lowest cost sequence of edges; and

means for guiding the robot through the environment based on the lowest cost sequence of edges.

12. The apparatus of claim 11 , further comprising:

means for computing a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and

means for constructing the 3D object map from the plurality of 3D depth maps.

13. The apparatus of claim 11 , further comprising means for guiding the robot through the environment based on texture information of the object of interest.

14. The apparatus of claim 11 , further comprising means for guiding the robot through the environment based on importance weights assigned to different portions of the object of interest.

15. The apparatus of claim 11 , further comprising means for guiding the robot through the environment by incrementally creating a sampling based motion planning framework.

16. A non-transitory computer readable medium having encoded thereon program code for guiding a robot equipped with a camera to facilitate three-dimensional (3D) reconstruction through sampling based planning, the program code executed by a processor and comprising:

program code to identify an object of interest;

program code to search for the object of interest in the environment comprising a plurality of objects;

program code to recognize and localize the object of interest in a plurality of two-dimensional (2D) images of the environment captured via the camera;

program code to construct a 3D object map based on the localized object in the plurality of 2D images, a depth variance associated with pixels of the 3D object map;

program code to grow a sampling based structure around the 3D object map;

program code to assign a cost to each edge of the sampling based structure based on the depth variance of pixels visible along a given edge;

program code to search the sampling based structure to determine a lowest cost sequence of edges; and

program code to guide the robot through the environment based on the lowest cost sequence of edges.

17. The non-transitory computer readable medium of claim 16 , further comprising:

program code to compute a plurality of 3D depth maps for the localized object based on the plurality of 2D images; and

program code to construct the 3D object map from the plurality of 3D depth maps.

18. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment based on texture information of the object of interest.

19. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment based on importance weights assigned to different portions of the object of interest.

20. The non-transitory computer readable medium of claim 16 , further comprising program code to guide the robot through the environment by incrementally creating a sampling based motion planning framework.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2016
From: AGHAMOHAMMADI, ALIAKBAR; NAJAFI SHOUSHTARI, SEYED HESAMEDDIN; TOWAL, REGAN BLYTHE
To: QUALCOMM INCORPORATED
Reel/Frame 040363/0870 →
Continuity (2)
Provisional Application 62286032 · Jan 22, 2016
Related Publication 20170213070A1 · Jul 27, 2017