IP Library › Granted Patent US 12,015,757
Granted Patent B2
US 12,015,757 · App. 17/310,718 · Granted Jun 18, 2024

Obstacle detection method and apparatus and unmanned aerial vehicle

Inventor: Xin Zheng (Guangdong, CN)
Assignee: AUTEL ROBOTICS CO., LTD.
H04N13/254B64C39/024B64D47/00G01J1/4204G06V20/58H04N13/239H04N13/296B64U2101/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,015,757
App. No.
17/310,718
Granted
Jun 18, 2024
Kind
B2
Abstract

Embodiments of the present invention relate to an obstacle detection method and apparatus and an unmanned aerial vehicle. The unmanned aerial vehicle includes a binocular photographing component and a laser texture component. The method includes: determining to start the laser texture component; starting the laser texture component, to emit a laser texture; obtaining a binocular view that is collected by the binocular photographing component and that includes the laser texture, and setting the binocular view that includes the laser texture as a target binocular view; and performing obstacle detection based on the target binocular view. In the above technical solutions, according to the embodiments of the present invention, precision of binocular stereo matching can be improved without changing an original binocular matching algorithm and structure, thereby improving precision of obstacle detection. In addition, the unmanned aerial vehicle can perform binocular sensing while flying at night.

Claims (98)

1. An obstacle detection method, applied to an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises a binocular photographing component and a laser texture component, and the method comprises:

determining to start the laser texture component;

starting the laser texture component, to emit a laser texture;

obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and

performing obstacle detection based on the target binocular view;

wherein the determining to start the laser texture component comprises:

determining that a detection zone of the laser texture component has an obstacle;

wherein the determining that a detection zone of the laser texture component has an obstacle comprises:

obtaining an initial binocular view currently collected by the binocular photographing component:

determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and

when the distance is less than or equal to a detection distance of the laser texture component determining that the detection zone of the laser texture component has an obstacle;

wherein the determining to start the laser texture component further comprises:

determining that a photographing scene of the binocular photographing component is a weak texture scene;

wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:

obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;

performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;

collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and

when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.

2. The method according to claim 1 , wherein the determining to start the laser texture component further comprises:

determining that a photographing scene of the binocular photographing component is a repeated texture scene.

3. The method according to claim 2 , wherein the determining that a photographing scene of the binocular photographing component is a repeated texture scene comprises:

obtaining an initial binocular view currently collected by the binocular photographing component;

performing stereo matching on the initial binocular view, and obtaining a smallest cost value C 1i and a second smallest cost value C 2i that correspond to each matching block, wherein i indicates an i-th matching block; and

determining that the photographing scene of the binocular photographing component is the repeated texture scene if there is a matching block i that satisfies the following formula:

C 1i /C 2i > K , wherein

A value range of K is: 0.5≤K<1.

4. The method according to claim 1 , wherein the determining that luminance of an environment of the unmanned aerial vehicle is less than a preset value comprises:

an initial view of initial binocular views currently collected by the binocular photographing component; and

determining that the luminance of the environment of the unmanned aerial vehicle is less than the preset value according to an average gray scale value of pixels in the initial view.

5. An obstacle detection apparatus, applicable to an unmanned aerial vehicle, wherein the unmanned aerial vehicle comprises a binocular photographing component and a laser texture component, and the obstacle detection apparatus comprises:

a memory, configured to store a computer-executable obstacle detection program; and

a processor, configured to invoke the computer-executable obstacle detection program to implement:

determine to start the laser texture component;

start the laser texture component, to emit a laser texture;

obtain a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and

perform obstacle detection based on the target binocular view, the processor is further configured to determine that a detection zone of the laser texture component has an obstacle;

obtain an initial binocular view currently collected by the binocular photographing component;

determine a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and

when the distance is less than or equal to a detection distance of the laser texture component, determine that the detection zone of the laser texture component has an obstacle;

determine that a photographing scene of the binocular photographing component is a weak texture scene;

obtain an initial view of initial binocular views currently collected by the binocular component;

perform a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;

collect statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and

when the quantity N is less than or equal to a quantity threshold Nth, determine that the photographing scene of the binocular photographing component is the weak texture scene.

6. The apparatus according to claim 5 , wherein

the processor is further configured to determine that a photographing scene of the binocular photographing component is a repeated texture scene.

7. The apparatus according to claim 6 , wherein the processor is further configured to:

obtain an initial binocular view currently collected by the binocular photographing component;

perform stereo matching on the initial binocular view, and obtain a smallest cost value C 1i and a second smallest cost value C 2i that correspond to each matching block, wherein i indicates an ith matching block; and

determine that the photographing scene of the binocular photographing component is the repeated texture scene if there is a matching block i that satisfies the following formula:

C 1i /C 2i >K , wherein

a value range of K is: 0.5≤K<1.

8. The apparatus according to claim 5 , wherein the processor is further configured to:

obtain an initial view of initial binocular views currently collected by the binocular photographing component; and

determine that the luminance of the environment of the unmanned aerial vehicle is less than the preset value according to an average grayscale value of pixels in the initial view.

9. An unmanned aerial vehicle, comprising:

a vehicle body;

a vehicle arm, connected to the vehicle body;

a power apparatus, disposed on the vehicle arm;

a binocular photographing component, disposed on the vehicle body, and configured to collect a binocular view in a movement direction of the unmanned aerial vehicle;

a laser texture component, disposed on the vehicle body, and configured to emit a laser texture sensible by the binocular photographing component;

a processor, disposed inside the vehicle body, and being in communication connection with each of the binocular photographing component and the laser texture component; and

a memory, being in communication connection with the processor, wherein

the memory stores instructions executable by the processor, and the instructions are executed by the processor to enable the processor to execute:

determining to start the laser texture component;

starting the laser texture component; to emit a laser texture;

obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and

performing obstacle detection based on the target binocular view;

wherein starting the laser texture component to emit a laser texture further comprises:

determining that a detection zone of the laser texture component has an obstacle;

wherein the determining that a detection zone of the laser texture component has an obstacle comprises:

obtaining an initial binocular view currently collected by the binocular photographing component;

determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view; and

when the distance is less than or equal to a detection distance of the laser texture component, determining that the detection zone of the laser texture component has an obstacle;

wherein the determining to start the laser texture component further comprises:

determining that a photographing scene of the binocular photographing component is a weak texture scene;

wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:

obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;

performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;

collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and

when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.

10. A non-transitory computer-readable storage medium, storing computer-executable instructions, the computer-executable instructions being configured to cause an unmanned aerial vehicle to execute:

determining to start the laser texture component;

starting the laser texture component, to emit a laser texture;

obtaining a binocular view that is collected by the binocular photographing component and that comprises the laser texture, and setting the binocular view that comprises the laser texture as a target binocular view; and

performing obstacle detection based on the target binocular view wherein starting the laser texture component to emit a laser texture further comprises:

determining that a detection zone of the laser texture component has an obstacle;

wherein the determining that a detection zone of the laser texture component has an obstacle comprises:

obtaining an initial binocular view currently collected by the binocular photographing component;

determining a distance between the unmanned aerial vehicle and an obstacle closest to the unmanned aerial vehicle based on the initial binocular view and

when the distance is less than or equal to a detection distance of the laser texture component, determining that the detection zone of the laser texture component has an obstacle;

wherein the determining to start the laser texture component further comprises:

determining that a photographing scene of the binocular photographing component is a weak texture scene;

wherein the determining that a photographing scene of the binocular photographing component is a weak texture scene comprises:

obtaining an initial view of at least one initial binocular views currently collected by the binocular photographing component;

performing a gradient operation on the initial view, to obtain a gradient value of each pixel in the initial view;

collecting statistics on a quantity N of pixels whose gradient values are less than or equal to a gradient threshold; and

when the quantity N is less than or equal to a quantity threshold Nth, determining that the photographing scene of the binocular photographing component is the weak texture scene.

Assignments (1)
EMPLOYMENT AGREEMENT Recorded Aug 20, 2021
From: ZHENG, XIN
To: AUTEL ROBOTICS CO., LTD.
Reel/Frame 057249/0240 →
Priority Claims (1)
CN 201910149492.3 · Feb 28, 2019 · national
Continuity (1)
Related Publication 20220191460A1 · Jun 16, 2022