IP Library Granted Patent US 11,288,519
Granted Patent B2
US 11,288,519 · App. 15/996,354 · Granted Mar 29, 2022

Object counting and classification for image processing

Inventor: Stefan Schulte (Marke, BE)
Assignee: FLIR Belgium BVBA
G06K9/00785G06K9/00771G06K9/4604G06K9/6218G06K9/6267G06T7/13G06T7/248G06K2209/23G06T7/90G06T2207/10048G06T2207/30236G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,288,519
App. No.
15/996,354
Granted
Mar 29, 2022
Kind
B2
Abstract

Systems and methods according to one or more embodiments are provided for classifying and counting objects that pass through monitored zones in a field of view of an imaging device. An imaging device receives a series of images of a target scene having at least one monitored zone. An image processor extracts a target line of image values from each image, and adds the target line to a first spatio-temporal image. An edge detector and a cluster analysis module analyzes the first spatio-temporal image to identify objects associated with the at least one zone. An object classifier classifies in object and stores tracking information.

Claims (44)

1. A method comprising:

receiving a series of images of a target scene having at least one monitored zone;

for a scene target line which is a line in a predefined position in the target scene, extracting, from each of the images, an image target line of image values, each image target line being an image of the scene target line;

wherein the scene target line extends along its length transversely to an expected direction of travel of moving objects crossing the scene target line, each of the scene target line and the image target lines having a length greater than width;

storing the extracted image target lines in a first spatio-temporal image; and

analyzing the first spatio-temporal image to identify the moving objects associated with the at least one monitored zone, the objects crossing the scene target line, the objects moving away from the scene target line upon crossing the scene target line;

wherein said analyzing comprises:

detecting one or more edges in the first spatio-temporal image, at least one edge being defined by a plurality of the extracted image target lines; and

classifying each said moving object based on the extracted image target lines and the one or more detected edges in the first spatio-temporal image.

2. The method of claim 1 , further comprising capturing the series of images of the target scene by an imaging device, wherein the imaging device comprises a thermal camera.

3. The method of claim 1 , wherein the at least one monitored zone is a fixed subset of the field of view through which the objects are monitored.

4. The method of claim 1 , wherein:

the at least one monitored zone is a plurality of monitored zones, and the scene target line extends across the monitored zones transversely to the monitored direction of travel of objects moving through the monitored zones and crossing the scene target line;

said analyzing the first spatio-temporal image comprises identifying the moving objects associated with each of the monitored zones; and

said classifying each said moving object comprises classifying each said moving object as associated with the respective zone.

5. The method of claim 4 , wherein each monitored zone comprises a traffic lane, and the scene target line extends across the traffic lanes and spans a width of the traffic lanes.

6. The method of claim 5 wherein said classifying comprises counting moving objects of interest that cross the scene target line.

7. The method of claim 1 , wherein the detecting the one or more edges comprises detecting edges between the most recent image target line and a prior image target line.

8. The method of claim 7 , further comprising creating a feature vector of the detected edges and adding the feature vector to a second spatio-temporal image.

9. The method of claim 8 , further comprising applying a cluster algorithm to the second spatio-temporal image to identify clusters of edges associated with an object.

10. The method of claim 1 , wherein classifying each object comprises identifying a vehicle type.

11. A system comprising:

an image capture device configured to generate a series of images of a target scene including at least one zone;

an image processor configured to extract an image target line from each image of the series of images and add the image target line to a first spatio-temporal image, wherein each image target line is an image of a scene target line in a predefined position in the target scene, the scene target line extending along its length across a path of objects moving through the at least one zone, the path crossing the scene target line and then moving away from the scene target line, each of the scene target line and the image target lines having a length greater than width;

a cluster analysis module configured to analyze the first spatio-temporal image to identify an object moving along said path;

an edge detector configured to detect one or more edges in the first spatio-temporal image, at least one edge being defined by a plurality of the extracted image target lines; and

an object classifier configured to classify the object based on the extracted image target lines and the at least one edge in the first spatio-temporal image.

12. The system of claim 11 , wherein the imaging device comprises a thermal camera and/or a visible light camera.

13. The system of claim 11 , wherein:

the at least one zone is a plurality of zones, and the scene target line extends substantially across the zones and is oriented transversely to a direction of travel of objects through the zones;

said analyzing the first spatio-temporal image comprises identifying the moving objects associated with each of the zones; and

said classifying each said moving object comprises classifying each said moving object as associated with the respective zone.

14. The system of claim 13 , wherein each zone comprises a traffic lane, and the scene target line extends across the traffic lanes and spans a width of the traffic lanes.

15. The system of claim 14 wherein the object classifier is further configured to count identified vehicle types passing through each zone.

16. The system of claim 11 wherein said at least one edge is an edge between adjacent target lines in the spatio-temporal image, and the edge detector is configured to generate an associated feature vector incorporating the detected edge information, and store the feature vector in a second spatio-temporal image.

17. The system of claim 16 wherein the cluster analysis module is further configured to analyze the feature vector to detect objects in the second spatio-temporal image.

18. The system of claim 11 further comprising a memory configured to store object tracking information associated with each zone.

19. The system of claim 11 wherein the object classifier is further configured to identify a vehicle type associated with objects traveling through a monitored zone.

20. A system comprising:

an image capture device configured to generate a series of images of a target scene including at least one zone;

an image processor configured to extract an image target line from each image of the series of images and add the image target line to a first spatio-temporal image, wherein each image target line is an image of a scene target line in a predefined position in the target scene, the scene target line extending along its length across a path of objects moving through the at least one zone, the path crossing the scene target line and then moving away from the scene target line, each of the scene target line and the image target lines having a length greater than width;

a cluster analysis module configured to analyze the first spatio-temporal image to identify moving objects crossing the scene target line;

an edge detector configured to detect one or more edges in the first spatio-temporal image, at least one edge being defined by a plurality of the extracted image target lines; and

an object classifier configured to classify the objects based on the extracted image target lines and the at least one edge in the first spatio-temporal image.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 1, 2025
From: FLIR BELGIUM BVBA
To: RAYMARINE UK LIMITED
Reel/Frame 071149/0656 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 10, 2021
From: FLIR BELGIUM BVBA
To: FLIR SYSTEMS TRADING BELGIUM BVBA
Reel/Frame 056505/0694 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2020
From: FLIR BELGIUM BVBA
To: FLIR SYSTEMS TRADING BELGIUM BVBA
Reel/Frame 051528/0623 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2018
From: SCHULTE, STEFAN
To: FLIR BELGIUM BVBA
Reel/Frame 045979/0928 →
Continuity (2)
Provisional Application 62527935 · Jun 30, 2017
Related Publication 20190005336A1 · Jan 3, 2019