Video monitoring system employing hierarchical hidden markov model (HMM) event learning and classification
A method and apparatus are disclosed for automatically learning and identifying events in image data using hierarchical HMMs to define and detect one or more events. The hierarchical HMMs include multiple paths that encompass variations of the same event. Hierarchical HMMs provide a framework for defining events that may be exhibited in various ways. Each event is modeled in the hierarchical HMM with a set of sequential states that describe the paths in a high-dimensional feature space. These models can then be used to analyze video sequences to segment and recognize each individual event to be recognized. The hierarchical HMM is generated during a training phase, by processing a number of images of the event of interest in various ways, typically observed from multiple viewpoints.
1. A method for representing an event in a computer vision system, comprising:
processing a number of images of said event;
initializing a hierarchical HMM with a start state and an end state;
iteratively training said hierarchical HMM to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
generating said hierarchical HMM to represent said event, said hierarchical HMM including a plurality of paths each representing a variation of said event.
2. The method of claim 1 , wherein said images include images of said event manifested in various ways.
3. The method of claim 1 , wherein said images include images of said event observed from multiple viewpoints.
4. The meted of claim 1 , wherein said generating further comprises splitting a state in said hierarchical HMM into two states if the state is not sufficiently representative to create a branch from the split state.
5. The method of claim 1 , wherein said generating further comprises reducing overlapping states in said hierarchical HMM model.
6. The method of claim 1 , wherein said variation of said event is a variation in terms of a speed of said event.
7. The method of claim 1 , wherein said variation of said event is a variation in terms of a direction of said event.
8. The method of claim 1 , wherein said variation of said event is a variation in terms of intermediate states of a body posture associated with said event.
9. The method of claim 1 , further comprising comparing an image sequence to said hierarchical HMM to recognize said event.
10. A method for representing an event in a computer vision system, comprising:
comparing a sequence of input images to a hierarchical HMM representing said event, said hierarchical HMM initiliazed with a start state and an end state and iteratively trained to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
detecting said event if said comparing step exceeds a predefined criteria.
11. The method of claim 10 , wherein said detecting further comprises finding a path in said hierarchical HMM that matches said sequence of input images.
12. The method of claim 10 , wherein said hierarchical HMM includes a plurality of paths each representing a variation of said event.
13. The method of claim 12 , wherein said variation of said event is a variation in terms of a speed of said event.
14. The method of claim 12 , wherein said variation of said event is a variation in terms of a direction of said event.
15. The method of claim 12 , wherein said variation of said event is a variation in terms of intermediate states of a body posture associated with said event.
16. The method of claim 10 , further comprising evaluating context information associated with said event.
17. A system for representing an event in a computer vision system, comprising:
a memory that stores computer-readable code; and
a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to:
process a number of images of said event;
initialize a hierarchical HMM with a start state and an end state;
iteratively train said hierarchical HMM to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
generate said hierarhical HMM to represent said event, said hierarchical HMM including a plurality of paths each representing a variation of said event.
18. The system of claim 17 , wherein said processor is further configured to split a state in said hierarchical HMM into two states if the state model is not sufficiently representative to create a branch from the split state.
19. The system of claim 17 , wherein said processor is further configured to reduce overlapping states in said hierarchical HMM model.
20. The system of claim 17 , wherein said processor is further configured to compare an image sequence to said hierarchical HMM to recognize said event.
21. A system for representing an event in a computer vision system, comprising:
a memory that stores computer-readable code; and
a processor operatively coupled to said memory, said processor configured to implement said computer-readable code, said computer-readable code configured to:
compare a sequence of input images to a hierarchical HMM representing said event, said hierarchical HMM initialized with a start state and an end state and iteratively trained to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
detect said event if said comparing exceeds a predefined criteria.
22. The system of claim 21 , wherein said processor is further configured to find a path in said hierarchical HMM that matches said sequence of input images.
23. The system of claim 21 , wherein said hierarchical HMM includes a plurality of paths each representing a variation of said event.
24. An article of manufacture for representing an event in a computer vision system, comprising:
a computer-readable medium having computer-readable code means embodied thereon, said computer-readable program code means comprising:
processing a number of images of said event;
initializing a hierarchical HMM with a start state and an end state;
iteratively training said hierarchical HMM to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
generating a hierarhical HMM to represent said event, said hierarchical HMM including a plurality of paths each representing a variation of said event.
25. An article of manufacture for representing an event in a computer vision system, comprising:
a computer-readable medium having computer-readable code means embodied thereon, said computer-readable program code means comprising:
comparing a sequence of input images to a hierarchical HMM representing said event, said hierarchical HMM initialized with a start state and an end state and iteratively trained to add additional states between the start and end states until an overall likelihood criterion is satisfied; and
detecting said event if said comparing step exceeds a predefined criteria.