IP Library Granted Patent US 11,709,073
Granted Patent B2
US 11,709,073 · App. 17/097,393 · Granted Jul 25, 2023

Techniques for collaborative map construction between an unmanned aerial vehicle and a ground vehicle

Inventor: Mingyu Wang (Guangdong, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
G01C21/3841B64C39/024G01C21/3815G01C21/3852G01C21/3889B64U2101/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,709,073
App. No.
17/097,393
Granted
Jul 25, 2023
Kind
B2
Abstract

Techniques are disclosed for collaborative map construction using multiple vehicles. Such a system may include a ground vehicle including a first computing device and a first scanning sensor, and an aerial vehicle including a second computing device and a second scanning sensor. The ground vehicle can obtain a first real-time map based on first scanning data using the first scanning sensor, and transmit a first real-time map and position information to the aerial vehicle. The aerial vehicle can receive the first real-time map and the position information from the first computing device, obtain a second real-time map based on second scanning data collected using the second scanning sensor, and obtain a third real-time map based on the first real-time map and the second real-time map.

Claims (63)

1. A system for collaborative map construction, comprising:

a ground vehicle including a first computing device;

a first scanning sensor coupled to the ground vehicle;

an aerial vehicle including a second computing device;

a second scanning sensor coupled to the aerial vehicle;

the first computing device including at least one processor and a ground vehicle mapping manager, the ground vehicle mapping manager including first instructions which, when executed by the processor, cause the ground vehicle mapping manager to:

obtain a first real-time map of environment in which the ground vehicle is located from a front-view perspective based on first scanning data using the first scanning sensor; and

transmit the first real-time map and position information to the aerial vehicle;

the second computing device including at least one processor and an aerial vehicle mapping manager, the aerial vehicle mapping manager including second instructions which, when executed by the processor, cause the aerial vehicle mapping manager to:

receive the first real-time map and the position information from the first computing device;

obtain a second real-time map of the environment from an overhead perspective based on second scanning data collected using the second scanning sensor; and

generate a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,

wherein the third real-time map has precision higher than the second real-time map.

2. The system of claim 1 , wherein the first real-time map is a higher precision map than the second real-time map.

3. The system of claim 1 , wherein to obtain the third real-time map based on the first real-time map and the second real-time map, the second instructions, when executed, further cause the aerial vehicle mapping manager to:

determine an overlapping portion of the first real-time map and the second real-time map; and

merge the first real-time map and the second real-time map using the overlapping portion.

4. The system of claim 1 , wherein the first scanning sensor includes a LiDAR sensor and the second scanning sensor includes a visual sensor.

5. The system of claim 4 , wherein the first real-time map is constructed based on point cloud data obtained from the first scanning sensor, and the second real-time map is constructed based on visual data obtained from the second scanning sensor.

6. The system of claim 5 , wherein to obtain the first real-time map based on the first scanning data collected from a view angle of the ground vehicle using the first scanning sensor, the first instructions, when executed, further cause the ground vehicle mapping manager to:

obtain the second real-time map from the aerial vehicle;

convert coordinates in the second real-time map to a coordinate system to match the first real-time map;

determine an overlapping portion of the first real-time map and the second real-time map in the coordinate system; and

transmit with the overlapping portion to the aerial vehicle.

7. The system of claim 1 , wherein the second instructions, when executed, further cause the aerial vehicle mapping manager to:

determine available resources associated with the second computing device are below a threshold value;

send a request to the first computing device to generate the third real-time map, the request including the second real-time map; and

receive the third real-time map from the first computing device.

8. A method for collaborative map construction, comprising:

receiving, by an aerial vehicle, a first real-time map of environment in which a ground vehicle is located from a front-view perspective and position information from the ground vehicle, wherein the first real-time map is based on first scanning data collected using a first scanning sensor coupled to the ground vehicle;

obtaining a second real-time map of the environment from an overhead perspective based on second scanning data collected using a second scanning sensor coupled to the aerial vehicle; and

generating a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,

wherein the third real-time map has precision higher than the second real-time map.

9. The method of claim 8 , wherein receiving, by the aerial vehicle, the first real-time map from the ground vehicle, further comprises:

receiving position information of the aerial vehicle from the ground vehicle.

10. The method of claim 8 , wherein the position information includes global navigation satellite system (GLASS) received from the aerial vehicle.

11. The method of claim 8 , wherein the position information includes a real-time relative position determined using the first scanning sensor.

12. The method of claim 8 , wherein the ground vehicle is an autonomous vehicle.

13. The method of claim 8 , wherein the ground vehicle is configured to obtain third scanning data from a third scanning sensor coupled to the ground vehicle, the first scanning sensor including a LiDAR sensor and the third scanning sensor including an imaging sensor, and generate the first real-time map based on the first scanning data and the third scanning data using a Simultaneous Localization and Mapping (SLAM) algorithm.

14. The method of claim 8 , further comprising:

transmitting the second scanning data to the ground vehicle, wherein the ground vehicle is configured to generate the first real-time map based on the first scanning data and the second scanning data using a Simultaneous Localization and Mapping (SLAM) algorithm, wherein the first scanning sensor includes a LiDAR sensor and the second scanning sensor includes an imaging sensor.

15. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by one or more processors, cause the one or more processors to:

receive, by an aerial vehicle, a first real-time map of environment in which a ground vehicle is located from a front-view perspective and position information from the ground vehicle, wherein the first real-time map is based on first scanning data collected using a first scanning sensor coupled to the ground vehicle;

obtain a second real-time map of the environment from an overhead perspective based on second scanning data collected using a second scanning sensor coupled to the aerial vehicle; and

generate a third real-time map of the environment from the overhead perspective based on the first real-time map and the second real-time map,

wherein the third real-time map has precision higher than the second real-time map.

16. The non-transitory computer readable storage medium of claim 15 , wherein the first real-time map is constructed based on point cloud data obtained from the first scanning sensor, and the second real-time map is constructed based on visual data obtained from the second scanning sensor.

17. The non-transitory computer readable storage medium of claim 16 , wherein to receive, by the aerial vehicle, the first real-time map from the ground vehicle, wherein the first real-time map is based on the first scanning data collected using the first scanning sensor coupled to the ground vehicle, the instructions, when executed, further cause the one or more processors to:

transmit the second real-time map to the ground vehicle, wherein the ground vehicle is configured to:

convert coordinates in the second real-time map to a coordinate system to match the first real-time map;

determine an overlapping portion of the first real-time map and the second real-time map in the coordinate system; and

transmit with the overlapping portion to the aerial vehicle.

18. The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, further cause the one or more processors to:

determine available resources associated with the aerial vehicle are below a threshold value;

send a request to the ground vehicle to generate the third real-time map, the request including the second real-time map; and

receive the third real-time map from the ground vehicle.

19. The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, further cause the one or more processors to:

receive the third real-time map from the aerial vehicle;

generate control data based on the third real-time map to navigate the aerial vehicle; and

transmit the control data to the aerial vehicle.

20. The non-transitory computer readable storage medium of claim 15 , wherein the instructions, when executed, further cause the one or more processors to:

generate control data based on the third real-time map to navigate the aerial vehicle.

21. The system of claim 1 , wherein the position information transmitted from the ground vehicle to the aerial vehicle is position information of the aerial vehicle.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 17, 2020
From: WANG, MINGYU
To: SZ DJI TECHNOLOGY CO., LTD.
Reel/Frame 054452/0618 →
Continuity (2)
Continuation PCTCN2019077504 · Mar 8, 2019
Related Publication 20210131821A1 · May 6, 2021
Cited By (1)
US 12,222,218