IP Library › Granted Patent US 11,526,998
Granted Patent B2
US 11,526,998 · App. 16/728,448 · Granted Dec 13, 2022

Methods and system for infrared tracking

Inventors: Chao Weng (Shenzhen, CN); Wei Zhang (Shenzhen, CN); Mingxi Wang (Shenzhen, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
G06T7/246B64C39/024G05D1/0202G05D1/12G06T5/50G06T7/223B64C2201/123B64C2201/127B64C2201/146G06T2207/10036G06T2207/10048G06T2207/20221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,526,998
App. No.
16/728,448
Filed
Dec 27, 2019
Granted
Dec 13, 2022
Kind
B2
Art Unit
3667
USPC
701/3
Abstract

A computer-implemented method for tracking includes obtaining an infrared image and a visible image from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), obtaining a combined image based on the infrared image and the visible image, identifying a target in the combined image, and generating control signals for tracking the identified target using the imaging device.

Claims (59)

1. A method for tracking, comprising:

obtaining, from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), an infrared image and a visible image;

obtaining a combined image based on the infrared image and the visible image, including:

applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image; and

controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;

wherein:

the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;

the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and

the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

2. The method of claim 1 , wherein obtaining the combined image further comprises:

processing the infrared image to extract infrared features;

processing the visible image to extract the visible features;

matching the infrared image and the visible image based on the infrared features and the visible features; and

applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

3. The method of claim 2 , wherein the infrared features comprise an infrared outer contour and the visible features comprise a visible outer contour, and wherein matching the infrared image and the visible image comprises aligning the infrared outer contour with the visible outer contour.

4. The method of claim 1 , wherein identifying the target in the combined image comprises obtaining target information from a remote terminal and identifying the target based on the target information.

5. The method of claim 1 , wherein generating the control signals for tracking the identified target comprises determining whether to control the UAV, the carrier, or the imaging device based on a current configuration of the UAV, the carrier, or the imaging device.

6. The method of claim 1 , wherein generating the control signals for tracking the identified target comprises detecting a deviation of the target from a predetermined configuration and generating the control signals for substantially correcting the deviation.

7. An unmanned aerial vehicle (UAV), comprising:

a carrier;

a memory that stores one or more computer-executable instructions; and

one or more processors configured to access the memory and execute the computer-executable instructions to perform a method comprising:

obtaining, from an imaging device supported by the carrier, an infrared image and a visible image;

obtaining a combined image based on the infrared image and the visible image, including:

applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image;

identifying a target in the combined image; and

controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;

wherein:

the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;

the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and

the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

8. The UAV of claim 7 , wherein obtaining the combined image comprises:

processing the infrared image to extract infrared features;

processing the visible image to extract the visible features;

matching the infrared image and the visible image based on the infrared features and the visible features; and

applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

9. The UAV of claim 8 , wherein the infrared features comprise an infrared outer contour and the visible features comprise a visible outer contour, and wherein matching the infrared image and the visible image comprises aligning the infrared outer contour with the visible outer contour.

10. The UAV of claim 7 , wherein generating the control signals for tracking the identified target comprises determining whether to control the UAV, the carrier, or the imaging device based on a current configuration of the UAV, the carrier, or the imaging device.

11. The UAV of claim 7 , wherein generating the control signals for tracking the identified target comprises detecting a deviation of the target from a predetermined configuration and generating the control signals for substantially correcting the deviation.

12. A tracking system, comprising:

a memory that stores one or more computer-executable instructions; and

one or more processors configured to access the memory and execute the computer-executable instructions to perform a method comprising:

obtaining, from an imaging device supported by a carrier of an unmanned aerial vehicle (UAV), an infrared image and a visible image;

obtaining a combined image based on the infrared image and the visible image, including:

applying, based on a first parameter and a second parameter, at least one of visible features of the visible image to the infrared image to generate the combined image;

identifying a target in the combined image; and

controlling at least one of the UAV, the carrier, or the imaging device to track the identified target;

wherein:

the visible features of the visible image are sorted in a rank of prominence from high to low or low to high, prominence of a feature being positively correlated to at least one of a size of the feature or obviousness of an edge of the feature;

the first parameter controls selecting, from the visible features of the visible image, N visible features of highest prominence in the rank as the visible features to be applied to the infrared image, a value of N being positively correlated to a value of the first parameter; and

the second parameter controls an intensity of the visible features to be applied to the infrared image, the intensity of the visible features to be applied to the infrared image being positively correlated to a value of the second parameter.

13. The system of claim 12 , wherein obtaining the combined image comprises:

processing the infrared image to extract infrared features;

processing the visible image to extract the visible features;

matching the infrared image and the visible image based on the infrared features and the visible features; and

applying the at least one of the visible features to the infrared image to generate the combined image further based on the matching.

14. The system of claim 13 , wherein the infrared features comprise an infrared outer contour and the visible features comprise a visible outer contour, and wherein matching the infrared image and the visible image comprises aligning the infrared outer contour with the visible outer contour.

15. The system of claim 12 , wherein generating the control signals for tracking the identified target comprises determining whether to control the UAV, the carrier, or the imaging device based on a current configuration of the UAV, the carrier, or the imaging device.

16. The system of claim 12 , wherein generating the control signals for tracking the identified target comprises detecting a deviation of the target from a predetermined configuration and generating the control signals for substantially correcting the deviation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 27, 2019
From: WENG, CHAO; ZHANG, WEI; WANG, MINGXI
To: SZ DJI TECHNOLOGY CO., LTD.
Reel/Frame 051376/0959 →
Continuity (2)
Continuation PCTCN2017109364 · Nov 3, 2017
Related Publication 20200143545A1 · May 7, 2020