IP Library Granted Patent US 9,734,399
Granted Patent B2
US 9,734,399 · App. 14/247,398 · Granted Aug 15, 2017

Context-aware object detection in aerial photographs/videos using travel path metadata

Inventors: Arturo Flores (La Jolla, CA); Kyungnam Kim (Oak Park, CA); Yuri Owechko (Newbury Park, CA)
Assignee: THE BOEING COMPANY
G06K9/00651B64D47/08G06K9/00744G06K9/6215G06T7/136G06T7/194G06T7/38G06T2207/10016G06T2207/10032G06T2207/20224
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,734,399
App. No.
14/247,398
Granted
Aug 15, 2017
Kind
B2
Abstract

A system and a method for real-time detection of a moving object on a travel path in a geographical area is provided. The system and method may also be used to track such a moving object in real-time. The system includes an image capturing device for capturing successive images of a geographical area, a geographical reference map comprising contextual information of the geographical area, and a processor configured to calculate differences between successive images to detect, in real-time, a moving object on the travel path. The method includes capturing successive images of the geographical area using the image capturing device, geo-registering at least some of the successive images relative to the geographical reference map, and calculating differences between the successive images to detect, in real-time, an object.

Claims (45)

1. A method for real-time detection of an object, the method comprising:

capturing, using an image capturing device, successive images of a geographical area containing at least one travel path;

geo-registering at least some of the successive images relative to a geographical reference map comprising contextual information of the geographical area, the contextual information comprising travel path metadata associated with the at least one travel path;

calculating differences between the successive images to thereby detect an object in real-time;

calculating, using the travel path metadata as additional context for the detected object, an accumulated pixel squared difference image between the geographical reference map and the successive images; and

applying a predefined sensitivity threshold to the accumulated pixel squared difference image to thereby produce a binary image.

2. The method of claim 1 , further comprising:

receiving registration information via a user interface,

wherein a first image of the successive images is manually geo-registered using the received registration information.

3. The method of claim 1 , further comprising:

acquiring the geographical reference map by obtaining images of the geographical area including the at least one travel path.

4. The method of claim 1 , wherein the at least one travel path comprises a road.

5. The method of claim 1 , wherein the image capturing device is positioned on an aerial structure.

6. The method of claim 1 , wherein capturing the successive images of the geographical area comprises:

capturing at least one video sequence including the geographical area; and

separating the video sequence into the successive images.

7. The method of claim 1 , wherein geo-registering the at least some of the successive images relative to the geographical reference map is performed using planar homography.

8. The method of claim 1 , further comprising:

using the contextual information to suppress false detections of one or more moving objects outside of the at least one travel path.

9. The method of claim 1 , further comprising:

using the contextual information to detect a moving object on the at least one travel path.

10. The method of claim 1 , wherein geo-registering at least some of the successive images relative to a geographical reference map comprises:

performing a first geo-registration between the at least some of the successive images; and

performing a second geo-registration between at least one of the at least some successive images relative to the reference map, wherein the second geo-registration operates to reduce errors introduced during the first geo-registration.

11. The method of claim 1 , wherein the detected object comprises a moving object, the method further comprising tracking the position of the moving object.

12. A system for real-time detection of a moving object on a travel path in a geographical area, the system comprising:

an image capturing device for capturing successive images of the geographical area;

a geographical reference map comprising contextual information of the geographical area, the contextual information comprising travel path metadata associated with the travel path; and

a processor configured to:

calculate differences between the successive images to thereby detect, in real-time, a moving object on the travel path;

calculate, using the travel path metadata as additional context for the detected moving object, an accumulated pixel squared difference image between the geographical reference map and the successive images; and

applying a predefined sensitivity threshold to the accumulated pixel squared difference image to thereby produce a binary image.

13. The system of claim 12 , wherein said image capturing device is a video camera.

14. The system of claim 13 , wherein said detected moving object is a vehicle and said travel path is a road.

15. The system of claim 12 , wherein the image capturing device is mounted on an aerial structure.

16. The system of claim 15 , wherein the airborne structure comprises an aerial platform on an unmanned vehicle.

17. The system of claim 12 , wherein the geographical reference map is housed in a server remote from the processor.

18. The system of claim 12 , wherein the processor is further configured to track the position of the detected moving object.

19. A method for real-time detection of a moving object, the method comprising:

capturing, using an image capturing device, a plurality of successive images of a geographical area including a travel path;

geo-registering at least a portion of the plurality of successive images with a geographical reference map comprising contextual information associated with the geographical area, the contextual information comprising travel path metadata associated with the travel path;

calculating differences between the plurality of successive images to thereby detect a moving object in real-time; and

selecting a sensitivity threshold value based on whether the detected moving object is determined to be within the travel path,

wherein a lesser sensitivity threshold value is selected when the detected moving object is within the travel path, and a greater sensitivity threshold value is selected when the detected moving object is outside the travel path, and

wherein the sensitivity threshold value is selected to provide a predetermined false alarm rate.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 8, 2014
From: FLORES, ARTURO; KIM, KYUNGNAM; OWECHKO, YURI
To: THE BOEING COMPANY
Reel/Frame 032628/0050 →
Continuity (1)
Related Publication 20150286868A1 · Oct 8, 2015