IP Library › Granted Patent US 10,754,025
Granted Patent B2
US 10,754,025 · App. 16/013,305 · Granted Aug 25, 2020

Radar and camera-based data fusion

Inventors: Sayed Zeeshan Asghar (Singapore, SG); Batuhan Okur (Singapore, SG)
Assignee: RAPSODO PTE. LTD.
G01S13/867G01S13/58G01S13/723G06K9/00536G06K9/6289G06T7/248G06T7/74H04N5/232G06T2207/30221G06T2207/30241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,754,025
App. No.
16/013,305
Granted
Aug 25, 2020
Kind
B2
Abstract

A method including detecting an object within a field of view of a radar using a radar signal; tracking movement of the object through the field of view of the radar; triggering a camera to capture a plurality of images of the object based on the movement of the object; detecting the object in the plurality of images; combining data of the radar signal with data of the camera to estimate a position of the object; identifying a radar signal track generated by the motion of the object based on the combined data; and estimating a trajectory of the object based on identifying the radar signal track.

Claims (74)

1. A method, comprising:

detecting an object within a field of view of a radar using a radar signal;

tracking movement of the object through the field of view of the radar;

triggering a camera to capture a plurality of images of the object based on the movement of the object;

detecting the object in the plurality of images;

combining data of the radar signal with data of the camera to estimate a position of the object;

identifying a radar signal track generated by the motion of the object based on the combined data;

estimating a trajectory of the object based on identifying the radar signal track.

2. The method of claim 1 , wherein triggering the camera further comprises:

determining a pre-determined amount of time has passed since an onset of tracking the movement of the object; and

triggering the camera to capture the plurality of images based on a passage of the pre-determined amount of time.

3. The method of claim 1 , wherein triggering the camera further comprises:

determining an amplitude of a radar signal associated with tracking the movement of the object; and

triggering the camera to capture the plurality of images based on determining the amplitude of the radar signal.

4. The method of claim 1 , wherein triggering the camera further comprises:

determining a pattern of a radar signal associated with tracking the motion of the object;

matching the pattern of the radar signal to a previously known pattern stored in memory; and

triggering the camera to capture the plurality of images based on matching the pattern of the radar signal with the previously known pattern.

5. The method of claim 1 , wherein triggering the camera further comprises:

detecting the movement of a second object associated with the object; and

triggering the camera to capture the plurality of images based on detecting the movement of the second object.

6. The method of claim 1 , wherein identifying the radar signal track further comprises:

generating a Doppler radar signal corresponding to a radial velocity of the object.

7. The method of claim 1 , wherein estimating the position of the object further comprises:

determining an elevation and an azimuth angle of the object in three-dimensional spherical coordinates.

8. The method of claim 7 , further comprising:

converting the position of the object from three-dimensional spherical coordinates to Cartesian coordinates.

9. The method of claim 1 , wherein estimating the trajectory of the object further comprises:

determining an object speed, a vertical launch angle, a horizontal launch angle, or a combination thereof.

10. The method of claim 1 , further comprising:

determining a point of interest associated with the object, the object having an irregular profile, wherein triggering the camera further comprises:

triggering the camera and a second camera to capture the plurality of images of the object.

11. The method of claim 1 , wherein detecting the object further comprises:

detecting the object located closest to a radar system associated with the field of view of the radar.

12. A system, comprising:

a processor;

a radar device in electronic communication with the processor;

a camera in electronic communication with the processor;

memory in electronic communication with the processor;

instructions stored in the memory, the instructions being executable by the processor to:

detect an object within a field of view of a radar using a radar signal;

track movement of the object through the field of view of the radar;

trigger a camera to capture a plurality of images of the object based on the movement of the object;

detect the object in the plurality of images;

combine data of the radar signal with data of the camera to estimate a position of the object;

identify a radar signal track generated by the motion of the object based on the combined data;

estimate a trajectory of the object based on identifying the radar signal track.

13. The system of claim 12 , wherein when the processor triggers the camera, the instructions are further executable to:

determine a pre-determined amount of time has passed since an onset of tracking the movement of the object; and

trigger the camera to capture the plurality of images based on a passage of the pre-determined amount of time.

14. The system of claim 12 , wherein when the processor triggers the camera, the instructions are further executable to:

determine an amplitude of a radar signal associated with tracking the movement of the object; and

trigger the camera to capture the plurality of images based on determining the amplitude of the radar signal.

15. The system of claim 12 , wherein when the processor triggers the camera, the instructions are further executable to:

determine a pattern of a radar signal associated with tracking the movement of the object;

match the pattern of the radar signal to a previously known pattern stored in memory; and

trigger the camera to capture the plurality of images based on matching the pattern of the radar signal with the previously known pattern.

16. The system of claim 12 , wherein when the processor triggers the camera, the instructions are further executable to:

detect the movement of a second object associated with the object; and

trigger the camera to capture the plurality of images based on detecting the movement of the second object.

17. The system of claim 12 , wherein when the processor identifies the radar signal track, the instructions are further executable to:

generate a Doppler radar signal corresponding to a radial velocity of the object.

18. The system of claim 12 , wherein when the processor estimates the position of the object, the instructions are further executable to:

determine an elevation and an azimuth angle of the object in three-dimensional spherical coordinates.

19. The system of claim 18 , wherein the instructions are further executable by the processor to cause the system to:

convert the position of the object from three-dimensional spherical coordinates to Cartesian coordinates.

20. A non-transitory computer-readable medium storing computer-executable code, the code executable by a processor to cause a system to:

detect an object within a field of view of a radar using a radar signal;

track movement of the object through the field of view of the radar;

trigger a camera to capture a plurality of images of the object based on the movement of the object;

detect the object in the plurality of images;

combine data of the radar signal with data of the camera to estimate a position of the object;

identify a radar signal track generated by the motion of the object based on the combined data;

estimate a trajectory of the object based on identifying the radar signal track.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 20, 2018
From: ASGHAR, SAYED ZEESHAN; OKUR, BATUHAN
To: RAPSODO PTE. LTD.
Reel/Frame 046397/0394 →
Continuity (1)
Related Publication 20190391254A1 · Dec 26, 2019
Cited By (2)
US 12,478,834 US 12,616,891