IP Library Granted Patent US 10,366,482
Granted Patent B2
US 10,366,482 · App. 15/211,378 · Granted Jul 30, 2019

Method and system for automated video image focus change detection and classification

Inventors: Roger B. Greenway, Jr. (Franklin, TN); Jonathan N. Thompson (Houston, TX); James H. Whitcomb, Jr. (Houston, TX); Stephen B. Greenway (Hot Spring, AR)
Assignee: PANASONIC CORPORATION OR NORTH AMERICA
G06T7/0002G06K9/6267H04N17/002G06T2207/10016G06T2207/30232
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,366,482
App. No.
15/211,378
Granted
Jul 30, 2019
Kind
B2
Abstract

A system includes a processor and a memory for evaluating a focus status of a video camera. The system receives a video image from a video server in accordance with a predetermined schedule. A selected focus classification procedure is performed on the received video image. The focus state of the video image of the video camera is classified in accordance with the focus classification procedure and determining from the video image whether the video camera is in a state of focus. A warning count is cleared when the video image is determined to be in a state of focus and an alarm indication is set when the warning count exceeds a predetermined threshold.

Claims (105)

1. A system for evaluating a focus status of a video camera, the system comprising:

a processor; and a memory storing instructions that, when executed by the processor, cause the processor to perform operations including:

receiving, at a receiver, a video image from a video server in accordance with a predetermined schedule;

determining a focus classification procedure to perform on the received video image; classifying the focus status of the video image in accordance with the focus classification procedure and determining, from the video image whether the video camera is in a state of focus; and

clearing a warning count when the video image is determined to be in a state of focus and setting an alarm indication when the warning count exceeds a predetermined threshold wherein the focus classification procedure comprises:

incrementally building an estimate of a background of a scene;

storing a reference image based on the estimate;

retrieving the reference image;

segmenting each of the video image and the reference image into a respective array of sub-images;

calculating a focus metric for each sub-image of the video image and the reference image;

comparing focus metrics between the sub-images of the video image and the sub-images of the reference image;

calculating a percent of sub-images that have a focus metric difference greater than a first parameter; and

determining whether the video camera is in the state of focus based on whether a percent of the image that is out of focus is greater than a second parameter;

wherein the focus metric may change for an out of focus image based on state of focus.

2. The system according to claim 1 , wherein the operations further include,

incrementing the warning count a first time when the video image is determined to be in a state of non-focus as a result of a first determination, and setting a warning status when the warning count is incremented.

3. The system according to claim 2 , wherein the operations further include,

clearing the incremented count when the video image is determined to be in the state of focus as a result of a second determination.

4. The system according to claim 2 , wherein the operations further include,

incrementing the warning count a second time when the video image is determined to be in the state of focus as a result of a second determination; and

setting the alarm indication when the warning count, incremented the second time, exceeds the predetermined threshold of at least two.

5. The system according to claim 1 , wherein the operations further include,

selecting one or more distinct focus classification procedures; and

classifying the focus status of an image of the video camera in accordance with the selected focus classification procedure.

6. The system according to claim 1 , wherein the operations further include:

selecting the focus classification procedure from a plurality of focus classification procedures; and

a second focus classification procedure of the plurality of focus classification procedures comprises:

segmenting the video image into an array of sub-images;

calculating at least one metric associated with a focus determination for each sub-image;

applying a blur to the video image to create a blurred image approximating what the video image would be if the video camera were in an out of focus state;

segmenting the blurred image in order to estimate how the at least one metric may change for an out of focused image;

storing the video image and the blurred image;

predicting an amount of blur for each sub-image;

comparing the predicted amount of blur to an acceptable amount of blur in the video image;

calculating a percent of sub-images that have a predicted blur greater than or equal to a first parameter, the percent representing a confidence that a whole of the video image is out of focus; and

determining whether the video camera is in the state of focus based on whether the percent of the video image considered out of focus is greater than a second parameter.

7. A method for evaluating a focus status of a video camera, the method comprising:

receiving, at a receiver a video image from a video server in accordance with a predetermined schedule;

determining a focus classification procedure to perform on the received video image;

classifying the focus status of the video image in accordance with the focus classification procedure and determining, from the video image, whether the video camera is in a state of focus; and

clearing a warning count when the video image is determined to be in a state of focus and setting an alarm indication when the warning count exceeds a predetermined threshold{circumflex over ( )} wherein the focus classification procedure comprises:

incrementally building an estimate of a background of a scene;

storing a reference image based on the estimate;

retrieving the reference image;

segmenting each of the video image and the reference image into a respective array of sub-images;

calculating a focus metric for each sub-image of the video image and the reference image;

comparing focus metrics between the sub-images of the video image and the sub-images of the reference image;

calculating a percent of sub-images that have a focus metric difference of greater than a first parameter; and

determining whether the video camera is in the state of focus based on whether a percent of the image that is out of focus is greater than a second parameter;

wherein the focus metric may change for an out of focus image based on state of focus.

8. The method according to claim 7 , wherein the operations further include,

incrementing the warning count a first time when the video image is determined to be in a state of non-focus as a result of a first determination, and setting a warning status when the warning count is incremented.

9. The method according to claim 8 , wherein the operations further include,

clearing the incremented count when the video image is determined to be in the state of focus as a result of a second determination.

10. The method according to claim 8 , wherein the operations further include,

incrementing the warning count a second time when the video image is determined to be in the state of focus as a result of a second determination; and

setting the alarm indication when the warning count, incremented the second time, exceeds the predetermined threshold of at least two.

11. The method according to claim 7 , wherein the operations further include,

selecting one or more of focus distinct classification; and

classifying the focus status of an image of the video camera in accordance with the selected focus classification procedure.

12. The method according to claim 7 , wherein the operations further include:

selecting the focus classification procedure from a plurality of focus classification procedures; and

a second focus classification procedure of the plurality of focus classification procedures comprises:

segmenting the video image into an array of sub-images;

calculating at least one metric associated with a focus determination for each sub-image;

applying a blur to the video image to create a blurred image approximating what the video image would be if the video camera were in an out of focus state;

segmenting the blurred image in order to estimate how the at least one metric may change for an out of focused image;

storing the video image and the blurred image;

predicting an amount of blur for each sub-image;

comparing the predicted amount of blur to an acceptable amount of blur in the video image;

calculating a percent of sub-images that have a predicted blur greater than or equal to a first parameter, the percent representing a confidence that a whole of the video image is out of focus; and

determining whether the video camera is in the state of focus based on whether the percent of the video image considered out of focus is greater than a second parameter.

13. A non-transitory computer readable storage medium that stores a set of executable instructions for evaluating a focus status of a video camera, the executable instructions, when executed by a processor, causing operations to be performed comprising:

receiving, at a receiver a video image from a video server in accordance with a predetermined schedule;

determining a focus classification procedure to perform on the received video image;

classifying the focus status of the video image in accordance with the focus classification procedure and determining, from the video image, whether the video camera is in a state of focus; and

clearing a warning count when the video image is determined to be in a state of focus and setting an alarm indication when the warning count exceeds a predetermined threshold wherein the focus classification procedure comprises:

incrementally building an estimate of a background of a scene;

storing a reference image based on the estimate;

retrieving the reference image;

segmenting each of the video image and the reference image into a respective array of sub-images;

calculating a focus metric for each sub-image of the video image and the reference image;

comparing focus metrics between the sub-images of the video image and the sub-images of the reference image;

calculating a percent of sub-images that have a focus metric difference of greater than a first parameter; and

determining whether the video camera is in the state of focus based on whether a percent of the image that is out of focus is greater than a second parameter;

wherein the focus metric may change for an out of focus image based on state of focus.

14. The non-transitory computer readable storage medium according to claim 13 , wherein the operations further include,

incrementing the warning count a first time when the video image is determined to be in a state of non-focus as a result of a first determination, and setting a warning status when the warning count is incremented.

15. The non-transitory computer readable storage medium according to claim 14 , wherein the operations further include,

clearing the incremented count when the video image is determined to be in the state of focus as a result of a second determination.

16. The non-transitory computer readable storage medium according to claim 14 , wherein the operations further include,

incrementing the warning count a second time when the video image is determined to be in the state of focus as a result of a second determination; and

setting the alarm indication when the warning count, incremented the second time, exceeds the predetermined threshold of at least two.

17. The non-transitory computer readable storage medium according to claim 13 , wherein the operations further include:

selecting the focus classification procedure from a plurality of focus classification procedures; and

a second focus classification procedure of the plurality of focus classification procedures comprises:

segmenting the video image into an array of sub-images;

calculating at least one metric associated with a focus determination for each sub-image;

applying a blur to the video image to create a blurred image approximating what the video image would be if the video camera were in an out of focus state;

segmenting the blurred image in order to estimate how the at least one metric may change for an out of focused image;

storing the video image and the blurred image;

predicting an amount of blur for each sub-image;

comparing the predicted amount of blur to an acceptable amount of blur in the video image;

calculating a percent of sub-images that have a predicted blur greater than or equal to a first parameter, the percent representing a confidence that a whole of the video image is out of focus; and

determining whether the video camera is in the state of focus based on whether the percent of the video image considered out of focus is greater than a second parameter.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 25, 2019
From: PANASONIC CORPORATION OF NORTH AMERICA
To: PANASONIC I-PRO SENSING SOLUTIONS CORPORATION OF AMERICA
Reel/Frame 051107/0194 →
CHANGE OF NAME Recorded Jun 3, 2019
From: VIDEO INSIGHT, INC.
To: PANASONIC CORPORATION OF NORTH AMERICA
Reel/Frame 049349/0024 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THIRD INVENTOR'S NAME PREVIOUSLY RECORDED AT REEL: 039168 FRAME: 0457. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 27, 2016
From: GREENWAY, ROGER B., JR.; THOMPSON, JONATHAN N.; WHITCOMB, JAMES H., JR.; GREENWAY, STEPHEN B.
To: VIDEO INSIGHT, INC.
Reel/Frame 041190/0975 →
CORRECTIVE ASSIGNMENT TO CORRECT THE FIRST INVENTORS NAME PREVIOUSLY RECORDED ON REEL 039168 FRAME 0457. ASSIGNOR(S) HEREBY CONFIRMS THE THE ASSIGNMENT. Recorded Sep 19, 2016
From: GREENWAY, ROGER B., JR.; THOMPSON, JONATHAN N.; WHITCOMB, JAMES H.; GREENWAY, STEPHEN B.
To: VIDEO INSIGHT, INC.
Reel/Frame 040071/0068 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 15, 2016
From: GREENWAY, BRYAN B., JR.; THOMPSON, JONATHAN N.; WHITCOMB, JAMES H., JR.; GREENWAY, STEPHEN B.
To: VIDEO INSIGHT, INC.
Reel/Frame 039168/0457 →
Continuity (2)
Provisional Application 62193795 · Jul 17, 2015
Related Publication 20170018065A1 · Jan 19, 2017