Method and apparatus for automatic visual event detection
View Patent ↗Disclosed are methods and apparatus for automatic visual detection of events, for recording images of those events and retrieving them for display and human or automated analysis, and for sending synchronized signals to external equipment when events are detected. An event corresponds to a specific condition, among some time-varying conditions within the field of view of an imaging device, that can be detected by visual means based on capturing and analyzing digital images of a two-dimensional field of view in which the event may occur. Events may correspond to rare, short duration mechanical failures for which obtaining images for analysis is desirable. Events are detected by considering evidence obtained from an analysis of multiple images of the field of view, during which time moving mechanical components can be seen from multiple viewing perspectives.
1. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs, the event comprising a motion of a predetermined object;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.
2. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights;
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred; and
producing a signal that is synchronized with a time at which the event occurs.
3. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred, the event analysis comprising a text string representing an expression in a syntax substantially similar to the syntax of a conventional programming language.
4. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs, the field of view comprising no more than about 40,000 pixels;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.
5. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights;
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred; and
wherein the steps of the method are performed at a rate of not less than two hundred frames per second.
6. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting, responsive to an activity analysis of the plurality of event detection weights, a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights, the activity analysis comprising
entering an active state when the visual analysis computes an event detection weight that indicates sufficient evidence that the event is occurring; and
exiting the active state when the visual analysis computes a plurality of event detection weights that each indicate insufficient evidence that the event is occurring; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.
7. A method for automatic visual detection of an event, comprising:
using a human-machine interface to specify the event, the human-machine interface comprising a logic view;
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.
8. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs, the event comprising a motion of a predetermined object;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred.
9. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights;
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred; and
an output process that produces a signal that is synchronized with a time at which the event occurs.
10. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred, the event analysis process comprising a text string representing an expression in a syntax substantially similar to the syntax of a conventional programming language.
11. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs, the capture process comprising an imager comprising no more than about 40,000 pixels;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred.
12. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights;
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred; and
wherein the capture process, the visual analysis process, the selection process, and the event analysis process operate at a rate of not less than two hundred frames per second.
13. A system for automatic visual detection of an event, comprising:
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
an activity analysis process that selects, responsive to the plurality of event detection weights, a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights, the activity analysis process comprising an active state that is entered when the visual analysis process computes an event detection weight that indicates sufficient evidence that the event is occurring, and that is exited when the visual analysis process computes a plurality of event detection weights that each indicate insufficient evidence that the event is occurring; and
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred.
14. A system for automatic visual detection of an event, comprising:
a human-machine interface that specifies the event, the human-machine interface comprising a logic view;
a capture process that captures a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
a visual analysis process adapted to be carried out by at least one of a collection of digital hardware elements and a collection of computer software instructions residing on a non-transitory computer-readable medium that computes, responsive to the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
a selection process that selects a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
an event analysis process that determines, responsive to the subset of the plurality of event detection weights, whether the event has occurred.
15. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs, wherein each frame comprises intensity values from a particular integration time;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.
16. The method of claim 15 wherein the plurality of frames are captured using an imager with a global shutter.
17. A method for automatic visual detection of an event, comprising:
capturing a plurality of frames, each frame in the plurality of frames comprising an image of a two-dimensional field of view in which the event occurs;
computing, using a processor, responsive to a visual analysis of the plurality of frames, a plurality of event detection weights, each event detection weight of the plurality of event detection weights corresponding respectively to each frame of the plurality of frames and comprising evidence that the event is occurring in the field of view;
selecting, responsive to an activity analysis of the plurality of event detection weights, a plurality of event frames from the plurality of frames, the plurality of event frames corresponding to a subset of the plurality of event detection weights, wherein only those frames in which there is sufficient evidence that the event has occurred are selected as a result of the activity analysis, the activity analysis comprising,
entering an active state when the visual analysis computes an event detection weight that indicates sufficient evidence that the event is occurring; and
exiting the active state when the visual analysis computes a plurality of event detection weights that each indicate insufficient evidence that the event is occurring; and
determining, responsive to an event analysis of the subset of the plurality of event detection weights, whether the event has occurred.