Method of using temporal context for image classification
View Patent ↗A method for improving scene classification of a sequence of digital images is disclosed herein. Such a method may include providing a sequence of images captured in temporal succession; (b) classifying each of the images individually based on information contained in the image alone to generate a first image classification; and (c) imposing a pre-determined temporal context model on the sequence of images to generate a final image classification for each image in the sequence.
1. A method for improving scene classification of a sequence of digital images comprising the steps of:
(a) providing a sequence of images captured in temporal succession, at least two pairs of consecutive images in the sequence of images having different elapsed times between their capture;
(b) classifying, with a programmed digital computer, each of the images individually based on information contained in the individual image to generate an initial content-based image classification for each of the images;
(c) generating, with a programmed digital computer, a final image classification for each image based at least on the respective initial content-based image classification and a pre-determined temporal context model that considers at least the temporal succession of the sequence of images; and
(d) storing the final image classifications in a computer readable storage medium,
wherein the classifying of step (b), and the final image classification classify images into one of a predetermined number of classes M, and
wherein M is greater than or equal to two.
2. The method as claimed in claim 1 wherein the information used in step (b) includes pixel information.
3. The method as claimed in claim 1 wherein the information used in step (b) includes capture-device-generated metadata information.
4. The method as claimed in claim 1 wherein the pre-determined temporal context model in step (c) is independent of elapsed time between consecutive images.
5. The method as claimed in claim 1 wherein the pre-determined temporal context model in step (c) is dependent on elapsed time between consecutive images in the sequence.
6. The method as claimed in claim 1 wherein the pre-determined temporal context model is a causal Hidden Markov Model dependent on a previous image.
7. The method as claimed in claim 1 wherein the pre-determined temporal context model is a non-casual model dependent on both a previous image and a subsequent image.
8. The method as claimed in claim 1 wherein the temporal context model is imposed using Viterbi algorithm.
9. The method as claimed in claim 1 wherein the temporal context model is imposed using a belief propagation algorithm.
10. The method as claimed in claim 1 wherein the pre-determined temporal context model in step (c) is dependent on elapsed time between consecutive images in the sequence, such that different elapsed times between a particular pair of consecutive images produces a different final image classification for a later-captured image of the particular pair of consecutive images.