IP Library › Granted Patent US 10,580,206
Granted Patent B2
US 10,580,206 · App. 15/726,017 · Granted Mar 3, 2020

Method and apparatus for constructing three-dimensional map

Inventors: Yuan Lin (Beijing, CN); Sili Chen (Beijing, CN); Yuhua Dong (Beijing, CN); Binbin Zheng (Beijing, CN); Yijie Yan (Beijing, CN); Zhongqin Wu (Beijing, CN)
Assignee: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
G06T17/05G06T7/55G06T17/00G06T2200/08G06T2207/10004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,580,206
App. No.
15/726,017
Filed
Oct 5, 2017
Granted
Mar 3, 2020
Kind
B2
Art Unit
2616
USPC
345/419
Abstract

The present disclosure discloses a method and apparatus for constructing a three-dimensional map. A specific embodiment of the method includes: acquiring multiple frames of a monocular image containing a target object collected by a monocular camera, and constructing a three-dimensional map containing the target object and other objects in the monocular image based on the monocular image; determining an absolute scale of the target object, and determining a ratio of a relative scale corresponding to the target object to the absolute scale; and adjusting a scale of each object in the three-dimensional map to obtain a three-dimensional map meeting a preset condition based on the ratio. The method and apparatus provided by the present disclosure achieves the construction of a world coordinate system with absolute scales in a three-dimensional map constructed based on a monocular image.

Claims (57)

1. A method for constructing a three-dimensional map, comprising:

acquiring a plurality of frames of a monocular image containing a target object collected by a monocular camera, and constructing a three-dimensional map containing the target object and other objects in the monocular image based on the monocular image, each object in the constructed three-dimensional map corresponding to a relative scale relative to a world coordinate system;

determining an absolute scale of the target object, and determining a ratio of the relative scale corresponding to the target object to the absolute scale, the absolute scale being an actual scale of the target object in the world coordinate system; and

adjusting a scale of each object in the three-dimensional map to obtain a three-dimensional map meeting a preset condition based on the ratio, the preset condition including: the each object in the three-dimensional map having the absolute scale;

wherein when the target object is a planar object, each type of the target object is preset with a corresponding absolute scale, the determining an absolute scale of the target object comprises: identifying a type of the target object, and using an absolute scale preset for the type of the target object as the absolute scale of the target object;

wherein the determining a ratio of the relative scale corresponding to the target object to the absolute scale comprises:

detecting an edge of the target object;

selecting points on the edge;

determining three-dimensional points in the three-dimensional map corresponding to the selected points on the edge;

calculating the relative scale corresponding to the target object based on coordinates of the three-dimensional points; and

determining the ratio of the relative scale corresponding to the target object to the absolute scale.

2. The method according to claim 1 , wherein when the target object is a non-planar object, before the acquiring the plurality of frames of a monocular image containing a target object collected by a monocular camera, the method further comprises:

performing a three-dimensional model construction for the target object to obtain a three-dimensional model of the target object, based on a characteristic point having a preset characteristic extracted from an image containing the target object, each three-dimensional point in the three-dimensional model having a coordinate in the world coordinate system, and the each three-dimensional point corresponding to a characteristic point and a preset characteristic of the characteristic point.

3. The method according to claim 2 , wherein the determining an absolute scale of the target object comprises:

detecting a profile of the target object;

extracting a characteristic point having a preset characteristic on the profile;

determining a three-dimensional point in the three-dimensional model of the target object corresponding to the extracted characteristic point, based on a matching relation between the preset characteristic of the extracted characteristic point and the preset characteristic corresponding to the three-dimensional point in the three-dimensional model of the target object; and

determining the absolute scale of the target object, based on a coordinate of the three-dimensional point in the three-dimensional model of the target object corresponding to the extracted characteristic point.

4. The method according to claim 3 , wherein the determining the ratio of the relative scale corresponding to the target object to the absolute scale comprises:

determining a three-dimensional point in the three-dimensional map corresponding to the extracted characteristic point;

determining the relative scale corresponding to the target object, based on the coordinate of the three-dimensional point in the three-dimensional map corresponding to the characteristic point; and

determining the ratio of the relative scale corresponding to the target object to the absolute scale.

5. An apparatus for constructing a three-dimensional map, comprising:

at least one processor; and

a memory storing instructions, which when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring a plurality of frames of a monocular image containing a target object collected by a monocular camera, and constructing a three-dimensional map containing the target object and other objects in the monocular image based on the monocular image, each object in the constructed three-dimensional map corresponding to a relative scale relative to a world coordinate system;

determining an absolute scale of the target object, and determining a ratio of the relative scale corresponding to the target object to the absolute scale, the absolute scale being an actual scale of the target object in the world coordinate system; and

adjusting a scale of each object in the three-dimensional map to obtain a three-dimensional map meeting a preset condition based on the ratio, the preset condition including: the each object in the three-dimensional map having the absolute scale;

wherein when the target object is a planar object, each type of the target object is preset with a corresponding absolute scale, the determining an absolute scale of the target object comprises: identifying a type of the target object, and using an absolute scale preset for the type of the target object as the absolute scale of the target object;

wherein the determining a ratio of the relative scale corresponding to the target object to the absolute scale comprises:

detecting an edge of the target object;

selecting points on the edge;

determining three-dimensional points in the three-dimensional map corresponding to the selected points on the edge;

calculating the relative scale corresponding to the target object based on coordinates of the three-dimensional points; and

determining the ratio of the relative scale corresponding to the target object to the absolute scale.

6. The apparatus according to claim 5 , wherein when the target object is a non-planar object, before acquiring the plurality of frames of a monocular image containing a target object collected by a monocular camera, the operations further comprise:

performing a three-dimensional model construction for the target object to obtain a three-dimensional model of the target object, based on a characteristic point having a preset characteristic extracted from an image containing the target object, each three-dimensional point in the three-dimensional model having a coordinate in the world coordinate system, and the each three-dimensional point corresponding to a characteristic point and a preset characteristic of the characteristic point.

7. The apparatus according to claim 6 , wherein the determining an absolute scale of the target object comprises:

detecting a profile of the target object;

extracting a characteristic point having a preset characteristic on the profile;

determining a three-dimensional point in the three-dimensional model of the target object corresponding to the extracted characteristic point, based on a matching relation between the preset characteristic of the extracted characteristic point and the preset characteristic corresponding to the three-dimensional point in the three-dimensional model of the target object; and

determining the absolute scale of the target object, based on a coordinate of the three-dimensional point in the three-dimensional model of the target object corresponding to the extracted characteristic point.

8. The apparatus according to claim 7 , wherein the determining the ratio of the relative scale corresponding to the target object to the absolute scale comprises:

determining a three-dimensional point in the three-dimensional map corresponding to the extracted characteristic point;

determining the relative scale corresponding to the target object, based on the coordinate of the three-dimensional point in the three-dimensional map corresponding to the characteristic point; and

determining the ratio of the relative scale corresponding to the target object to the absolute scale.

9. A non-transitory computer readable storage medium, storing a computer program thereon, the computer program, when executed by a processor, cause the processor to perform operations, the operations comprising:

acquiring a plurality of frames of a monocular image containing a target object collected by a monocular camera, and constructing a three-dimensional map containing the target object and other objects in the monocular image based on the monocular image, each object in the constructed three-dimensional map corresponding to a relative scale relative to a world coordinate system;

determining an absolute scale of the target object, and determining a ratio of the relative scale corresponding to the target object to the absolute scale, the absolute scale being an actual scale of the target object in the world coordinate system; and

adjusting a scale of each object in the three-dimensional map to obtain a three-dimensional map meeting a preset condition based on the ratio, the preset condition including: the each object in the three-dimensional map having the absolute scale;

wherein when the target object is a planar object, each type of the target object is preset with a corresponding absolute scale, the determining an absolute scale of the target object comprises: identifying a type of the target object, and using an absolute scale preset for the type of the target object as the absolute scale of the target object;

wherein the determining a ratio of the relative scale corresponding to the target object to the absolute scale comprises:

detecting an edge of the target object;

selecting points on the edge;

determining three-dimensional points in the three-dimensional map corresponding to the selected points on the edge;

calculating the relative scale corresponding to the target object based on coordinates of the three-dimensional points; and

determining the ratio of the relative scale corresponding to the target object to the absolute scale.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 5, 2017
From: LIN, YUAN; CHEN, SILI; DONG, YUHUA; ZHENG, BINBIN; YAN, YIJIE; WU, ZHONGQIN
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 043799/0609 →
Priority Claims (1)
CN 2017 1 0131578 · Mar 7, 2017 · national
Continuity (1)
Related Publication 20180261005A1 · Sep 13, 2018
Cited By (1)
US 12,260,575