IP Library Granted Patent US 11,022,984
Granted Patent B2
US 11,022,984 · App. 16/264,239 · Granted Jun 1, 2021

Automatic terrain evaluation of landing surfaces, and associated systems and methods

Inventors: Zhenyu Zhu (Shenzhen, CN); Honghui Zhang (Shenzhen, CN); You Zhou (Shenzhen, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
G05D1/042B64C39/024B64D45/08B64D47/08G06K9/0063G06K9/00201G06T7/593B64C2201/123B64C2201/127B64C2201/18G05D1/0038G06K9/6202G06T2207/10012G06T2207/10028G06T2207/10032G06T2207/30181
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,022,984
App. No.
16/264,239
Granted
Jun 1, 2021
Kind
B2
Abstract

Automatic terrain evaluation of landing surfaces, and associated systems and methods are disclosed herein. A representative method includes receiving a request to land a movable object and, in response to the request, identifying a target landing area on a landing surface based on at least one image of the landing surface obtained by the movable object. The method can further include directing the movable object to land at the target landing area.

Claims (57)

1. A computer-implemented method for automatically identifying characteristics of a landing surface, comprising:

receiving a request to land a movable object;

in response to the request, identifying a target landing area on a landing surface based on at least one image of the landing surface obtained by the movable object; and

directing the movable object to land at the target landing area,

wherein identifying the target landing area comprises retrieving depth information of individual pixels of the at least one image of the landing surface, the individual pixels corresponding to portions of the landing surface.

2. The method of claim 1 , further comprising:

generating a depth map including the depth information of the individual pixels, to represent heights of the corresponding portions of the landing surface.

3. The method of claim 1 , wherein the at least one image includes at least two images, and wherein the method further comprises:

performing a stereo matching on the at least two images of the landing surface to generate depth information of matching pixels of the at least two images.

4. The method of claim 1 , further comprising:

identifying a plurality of candidate areas within a pixel frame of the at least one image of the landing surface;

for an individual candidate area, identifying a best fit plane for pixels within the candidate area and determining a cost value associated with the best fit plane based on a cost function; and

selecting at least one of the candidate areas as corresponding to the target landing area, based on the cost values.

5. The method of claim 4 , wherein the cost value associated with the best fit plane denotes a degree of deviation of pixels from the best fit plane.

6. The method of claim 4 , wherein the cost function is based on a sum of squares of deviations of pixels from the best fit plane.

7. The method of claim 4 , further comprising:

determining whether the best fit planes are tilted by comparing normal vectors of the best fit planes with a gravity vector; and

discarding candidate areas with best fit planes that are tilted.

8. The method of claim 4 , further comprising:

discarding candidate areas with cost functions that are larger than a threshold value.

9. The method of claim 8 , further comprising:

if all candidate areas are discarded because the cost functions are larger than the threshold value:

directing the movable object to travel to another location; and

capturing images of another landing surface at the other location.

10. The method of claim 4 , further comprising:

adjusting the cost functions to include cost function information for nearby candidate areas.

11. The method of claim 4 , wherein the process of selecting comprises:

selecting a candidate area with the lowest cost function of the individual candidate areas.

12. The method of claim 4 , wherein the process of determining a plurality of candidate areas comprises:

scanning the pixel frame with a sliding window to generate the plurality of candidate areas, wherein a size of the sliding window is equal to or larger than a size of the movable object.

13. The method of claim 4 , wherein a shape of the candidate areas conforms to a shape of the movable object.

14. An apparatus, comprising:

a structure;

at least one motor carried by the structure; and

a controller programmed with instructions that, when executed:

receive a request to land the structure;

in response to the request, identify a target landing area on a landing surface based on at least one image of the landing surface obtained on board the structure; and

direct the structure to land at the target landing area using the at least one motor;

wherein identifying the target landing area comprises retrieving depth information of individual pixels of the at least one image of the landing surface, the individual pixels corresponding to portions of the landing surface.

15. The apparatus of claim 14 , wherein the structure comprises at least a portion of an unmanned aerial vehicle (UAV), a manned aircraft, an autonomous car, a self-balancing vehicle, or a robot.

16. The apparatus of claim 14 , wherein the instructions, when executed:

capture new images of the landing surface as the apparatus descends toward the target landing area;

update a depth map to include depth information of individual pixels of the new images of the landing surface, wherein the individual pixels correspond to portions of the landing surface; and

determine based on the depth map whether the target landing area continues to satisfy a landing condition as the apparatus descends.

17. An unmanned aerial vehicle (UAV) apparatus, comprising:

a plurality of motors;

a plurality of propellers, with individual propellers carried by corresponding individual motors;

at least one camera; and

a controller programmed with instructions that, when executed:

identify a target landing area on a landing surface based on at least one image of the landing surface captured by the at least one camera;

direct the UAV apparatus to land using the motors at the target landing area;

wherein identifying the target landing area comprises retrieving depth information of individual pixels of the at least one image of the landing surface, the individual pixels corresponding to portions of the landing surface.

18. The UAV apparatus of claim 17 , wherein the at least one camera includes two bottom-viewing cameras; and

wherein the instructions, when executed:

simultaneously capture two images of the landing surface using the two bottom-viewing cameras.

19. The UAV apparatus of claim 17 , wherein the instructions, when executed:

capture two images of the landing surface using the at least one camera at two different points in time, wherein the two images of the landing surface are captured from different perspectives.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 25, 2019
From: ZHU, ZHENYU; ZHANG, HONGHUI; ZHOU, YOU
To: SZ DJI TECHNOLOGY CO., LTD.,
Reel/Frame 049000/0124 →
Continuity (2)
Continuation PCTCN2016093767 · Aug 6, 2016
Related Publication 20190163206A1 · May 30, 2019
Cited By (2)
US 12,578,703 US 12,693,133