IP Library Granted Patent US 10,474,906
Granted Patent B2
US 10,474,906 · App. 15/468,339 · Granted Nov 12, 2019

High dynamic range video of fast moving objects without blur

Inventor: Shalender Singh (Milpitas, CA)
Assignee: Echelon Corporation
G06K9/00785H04N5/2351H04N5/2356H04N5/77H04N9/045H04W48/18
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,474,906
App. No.
15/468,339
Granted
Nov 12, 2019
Kind
B2
Abstract

Fast moving objects may be monitored by a video unit under a variety of lighting conditions. In a bootstrap phase, a model database is established for each of a plurality of sampling points, each model being a combination of color of the video image, respective optimum exposure, and respective lighting condition. Then in an image capture phase the video unit compares captured color and optimum exposure of the sampling point, with the models of color and respective optimum exposure in the model database, to find a best match lighting condition model in the model database. The video unit then adjusts an exposure setting to the most frequently occurring value for exposure among the other sampling points, if lighting conditions for the other sampling points are within a threshold range of the lighting condition of the best match model in the model database.

Claims (61)

1. A method for video monitoring of fast moving objects under a variety of lighting conditions, comprising:

establishing, by a video unit during a bootstrap phase, a plurality of sampling points for a viewing scene that includes a background object;

capturing, by the video unit during the bootstrap phase, at each sampling point, a plurality of video images under different lighting conditions, each video image having a color under a respective lighting condition at a respective optimum exposure;

storing, by the video unit during the bootstrap phase, in a model database either in the video unit or in a cloud database, for each sampling point, at least one model that is a combination of the color of at least one of the video images, its respective optimum exposure, and its respective lighting condition;

capturing, by the video unit during an image capture phase, for at least one of the sampling points, a video image having a captured color under current lighting conditions and an optimum exposure;

comparing, by the video unit during the image capture phase, for the least one of the sampling points, the captured color and the optimum exposure, with the at least one model of color and respective optimum exposure in the model database, to find a respective lighting condition of a best match model in the model database;

adjusting, by the video unit during the image capture phase, an exposure setting of the video unit, to a most frequently occurring value for exposure among other ones of the plurality of sampling points, if lighting conditions for the other ones of the plurality of sampling points are within a threshold range of the lighting condition of the best match model in the model database; and

if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the model database, then identify that a foreground object has moved over the background object where the subset of the other ones of the plurality of sampling points are located.

2. The method of claim 1 , further comprising:

storing, by the video unit during the image capture phase, in the model database, for the least one of the sampling points, a new model that is a combination of the captured color, its optimum exposure, and the lighting condition of the best match model in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection, if lighting conditions for the other ones of the plurality of sampling points are not within a threshold range of the lighting condition of the best match model in the model database.

3. The method of claim 1 , further comprising:

if lighting conditions for a subset of the other ones of the plurality of sampling points, are not within a larger threshold range than said threshold range of the lighting condition of the best match model in the model database, then create a new lighting condition using the respective exposure and color at each of the subset of the other ones of the plurality of sampling points, and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

4. The method of claim 1 , further comprising:

count the foreground object as a vehicle moving over a highway as a the background object.

5. The method of claim 1 , further comprising:

if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the mode database, then perform a non-optical exposure compensation on the subset of the other ones of the plurality of sampling points and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

6. A computer program product comprising computer executable program code recorded on a computer readable, non-transitory storage medium, for video monitoring of fast moving objects under a variety of lighting conditions, comprising:

code for establishing, by a video unit during a bootstrap phase, a plurality of sampling points for a viewing scene that includes a background object;

code for capturing, by the video unit during the bootstrap phase, at each sampling point, a plurality of video images under different lighting conditions, each video image having a color under a respective lighting condition at a respective optimum exposure;

code for storing, by the video unit during the bootstrap phase, in a model database either in the video unit or in a cloud database, for each sampling point, at least one model that is a combination of the color of at least one of the video images, its respective optimum exposure, and its respective lighting condition;

code for capturing, by the video unit during an image capture phase, for at least one of the sampling points, a video image having a captured color under current lighting conditions and an optimum exposure;

code for comparing, by the video unit during the image capture phase, for the least one of the sampling points, the captured color and the optimum exposure, with the at least one model of color and respective optimum exposure in the model database, to find a respective lighting condition of a best match model in the model database;

code for adjusting, by the video unit during the image capture phase, an exposure setting of the video unit, to a most frequently occurring value for exposure among other ones of the plurality of sampling points, if lighting conditions for the other ones of the plurality of sampling points are within a threshold range of the lighting condition of the best match model in the model database; and

code for, if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the model database, then identifying, by the video unit during the image capture phase, that a foreground object has moved over the background object where the subset of the other ones of the plurality of sampling points are located.

7. The computer program product of claim 6 , further comprising:

code for storing, by the video unit during the image capture phase, in the model database, for the least one of the sampling points, a new model that is a combination of the captured color, its optimum exposure, and the lighting condition of the best match model in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection, if lighting conditions for the other ones of the plurality of sampling points are not within a threshold range of the lighting condition of the best match model in the model database.

8. The computer program product of claim 6 , further comprising:

code for, if lighting conditions for a subset of the other ones of the plurality of sampling points, are not within a larger threshold range than said threshold range of the lighting condition of the best match model in the model database, then creating, by the video unit during the image capture phase, a new lighting condition using the respective exposure and color at each of the subset of the other ones of the plurality of sampling points, and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

9. The computer program product of claim 6 , further comprising:

code for counting, by the video unit during the image capture phase, the foreground object as a vehicle moving over a highway as a the background object.

10. The computer program product of claim 6 , further comprising:

code for, if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the mode database, then performing, by the video unit during the image capture phase, a non-optical exposure compensation on the subset of the other ones of the plurality of sampling points and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

11. An apparatus for video monitoring of fast moving objects under a variety of lighting conditions, comprising:

a video unit located within viewing distance of a highway, including a video camera, video processor, a processor and memory including computer program code configured to cause the video processor to process a video stream from the video camera;

the video processor configured to establish during a bootstrap phase, a plurality of sampling points for a viewing scene that includes a background object;

the video processor configured to capture during the bootstrap phase, at each sampling point, a plurality of video images under different lighting conditions, each video image having a color under a respective lighting condition at a respective optimum exposure;

the video processor configured to store during the bootstrap phase, in a model database either in the video unit or in a cloud database, for each sampling point, at least one model that is a combination of the color of at least one of the video images, its respective optimum exposure, and its respective lighting condition;

the video processor configured to capture during an image capture phase, for at least one of the sampling points, a video image having a captured color under current lighting conditions and an optimum exposure;

the video processor configured to compare during the image capture phase, for the least one of the sampling points, the captured color and the optimum exposure, with the at least one model of color and respective optimum exposure in the model database, to find a respective lighting condition of a best match model in the model database;

the video processor configured to adjust during the image capture phase, an exposure setting of the video unit, to a most frequently occurring value for exposure among other ones of the plurality of sampling points, if lighting conditions for the other ones of the plurality of sampling points are within a threshold range of the lighting condition of the best match model in the model database; and

if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the mode database, then the video processor configured to identify that a foreground object has moved over the background object where the subset of the other ones of the plurality of sampling points are located.

12. The apparatus of claim 11 , further comprising:

the video processor configured to store during the image capture phase, in the model database, for the least one of the sampling points, a new model that is a combination of the captured color, its optimum exposure, and the lighting condition of the best match model in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection, if lighting conditions for the other ones of the plurality of sampling points are not within a threshold range of the lighting condition of the best match model in the model database.

13. The apparatus of claim 11 , further comprising:

if lighting conditions for a subset of the other ones of the plurality of sampling points, are not within a larger threshold range than said threshold range of the lighting condition of the best match model in the model database, then the video processor configured to create a new lighting condition using the respective exposure and color at each of the subset of the other ones of the plurality of sampling points, and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

14. The apparatus of claim 11 , further comprising:

the video processor configured to count the foreground object as a vehicle moving over a highway as a the background object.

15. The apparatus of claim 11 , further comprising:

if lighting conditions for a subset of the other ones of the plurality of sampling points, do not match any model in the mode database, then the video processor configured to perform a non-optical exposure compensation on the subset of the other ones of the plurality of sampling points and store these as new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection.

16. A method for operating a model database for moving object detection of fast moving objects under a variety of lighting conditions, comprising:

establishing, by a video processor during a bootstrap phase, a plurality of sampling points for a viewing scene that includes a background object;

capturing, by the video processor during the bootstrap phase, at each sampling point, a plurality of video images under different lighting conditions, each video image having a color under a respective lighting condition at a respective optimum exposure;

storing, by the video processor during the bootstrap phase, in a model database either in the video processor or in a cloud database, for each sampling point, at least one model that is a combination of the color of at least one of the video images, its respective optimum exposure, and its respective lighting condition;

storing, by the video processor during an image capture phase, new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection; and

if lighting conditions for a subset of the plurality of sampling points, do not match any model in the model database, then identify that a foreground object has moved over a background object where the subset of the plurality of sampling points is located.

17. A computer program product comprising computer executable program code recorded on a computer readable, non-transitory storage medium, for operating a model database for moving object detection of fast moving objects under a variety of lighting conditions, comprising:

code for establishing, by a video processor during a bootstrap phase, a plurality of sampling points for a viewing scene that includes a background object;

code for capturing, by the video processor during the bootstrap phase, at each sampling point, a plurality of video images under different lighting conditions, each video image having a color under a respective lighting condition at a respective optimum exposure;

code for storing, by the video processor during the bootstrap phase, in a model database either in the video processor or in a cloud database, for each sampling point, at least one model that is a combination of the color of at least one of the video images, its respective optimum exposure, and its respective lighting condition;

code for storing, by the video processor during an image capture phase, new models in the model database, as a dynamic, self-learning process that improves the accuracy of moving object detection; and

code for, if lighting conditions for a subset of the plurality of sampling points, do not match any model in the model database, then identify that a foreground object has moved over a background object where the subset of the plurality of sampling points is located.

Assignments (3)
RELEASE OF SECURITY INTEREST Recorded Sep 24, 2019
From: OBSIDIAN AGENCY SERVICES, INC., AS COLLATERAL AGENT
To: ECHELON CORPORATION
Reel/Frame 050480/0865 →
SECURITY INTEREST Recorded Oct 8, 2018
From: ECHELON CORPORATION
To: OBSIDIAN AGENCY SERVICES, INC., AS COLLATERAL AGENT
Reel/Frame 047205/0018 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2017
From: SINGH, SHALENDER
To: ECHELON CORPORATION
Reel/Frame 041721/0223 →
Continuity (1)
Related Publication 20180276482A1 · Sep 27, 2018