Method for detection of temporal pattern anomalies in video streams

1. A method for detecting temporal pattern anomalies in a video stream, the method comprising:

receiving the video stream from a camera;

detecting an object in a current frame of the video stream and generating a processed current frame that contains the detected object;

determining a primary class of the detected object;

generating a feature representation of the processed current frame by a convolutional encoder;

clustering the feature representation m one or more primary clusters in a clustering space of the primary class, wherein the one or more primary clusters correspond to a sub-class of the primary class;

generating an information vector of the feature representation, that includes information regarding the primary class, the sub-class of the primary class, and one or more external factors associated with the feature representation;

clustering each information vector of the video stream into one or more secondary clusters, wherein each variable in the information vector is a clustering dimension; and

reporting a next frame as an anomaly when a corresponding information vector is positioned outside a secondary cluster of the one or more secondary clusters, of an information vector of a previous frame.

2. The method as claimed in claim 1 further comprising determining a cluster identification (ID) of the feature representation based on a corresponding sub-class of the primary class.

3. The method as claimed in claim 2 , wherein the information vector of a feature representation is a set containing a corresponding class number, a cluster identification (ID), a date, minute of day, a day, a month, a year, and a weather associated with corresponding image frame.

4. The method as claimed in claim 1 , wherein the camera is selected from at least one of: a traffic camera, a home doorbell camera, a body camera for soldiers or law enforcement, and a camera on an unmanned aerial vehicle (UAV).

5. The method as claimed in claim 1 , wherein the processed current frame is a cropped image of the current frame.

6. The method as claimed in claim 1 , wherein the processed current frame is a masked image of the current frame.

7. The method as claimed in claim 1 , wherein the convolutional encoder includes a series of convolutional layers to produce a feature representation of the processed current frame, wherein the feature representation is a tensor generated by a neural network that contains learned relevant information about the current frame.

8. The method as claimed in claim 1 further comprising detecting a negative anomaly in the video stream when a regular event ceases.

9. A system for detecting temporal pattern anomalies in a video stream, the system comprising:

an image input module configured to:

receive the video stream from a camera;

an object detection module configured to:

detect an object in a current frame of the video stream;

generate a processed current frame that contains the detected object; and

determine a primary class of the detected object;

a convolutional encoder configured to:

generate a feature representation of the processed current frame;

a primary clustering module configured to:

cluster the feature representation in one or more primary clusters in a clustering space of the primary class, wherein the one or more primary clusters correspond to a sub-class of the primary class;

a vector generation module configured to:

generate an information vector of the feature representation, that includes information regarding the primary class, the sub-class of the primary class, and one or more external factors associated with the feature representation; and

an anomaly clustering module configured to:

cluster each information vector of the video stream into one or more secondary clusters wherein each variable in the information vector is a clustering dimension; and

report a next frame as an anomaly when a corresponding information vector is positioned outside a secondary cluster of the one or more secondary clusters of an information vector of a previous frame.

10. The system as claimed in claim 9 , wherein the primary clustering module is configured to determine a cluster identification (ID) of the feature representation based on a corresponding sub-class of the primary class.

11. The system as claimed in claim 9 , wherein the information vector of a feature representation is a set containing a corresponding class number, a cluster identification (ID), a date, minute of day, a day, a month, a year, and a weather associated with corresponding image frame.

12. The system as claimed in claim 9 , wherein the camera is selected from at least one of: a traffic camera, a home doorbell camera, a body camera for soldiers or law enforcement, and a camera on an unmanned aerial vehicle (UAV).

13. The system as claimed in claim 9 , wherein the processed current frame is a cropped image of the current frame.

14. The system as claimed in claim 9 , wherein the processed current frame is a masked image of the current frame.

15. The system as claimed in claim 9 , wherein the convolutional encoder includes a series of convolutional layers to produce a feature representation of the processed current frame, wherein the feature representation is a tensor generated by a neural network that contains learned relevant information about the current frame.

16. The system as claimed in claim 9 , wherein the anomaly clustering module is further configured to detect a negative anomaly in the video stream when a regular event suddenly ceases.

17. A computer programmable product for detecting temporal pattern anomalies in a video stream, the computer programmable product comprising a set of instructions, the set of instructions when executed by a processor causes the processor to:

receive the video stream from a camera;

detect an object in a current frame of the video stream and generate a processed current frame that contains the detected object;

determine a primary class of the detected object;

generate a feature representation of the processed current frame by a convolutional encoder;

cluster the feature representation m one or more primary clusters in a clustering space of the primary class, wherein the one or more primary clusters correspond to a sub-classes of the primary class;

generate an information vector of the feature representation, that includes information regarding the primary class, the sub-class of the primary class, and one or more external factors associated with the feature representation;

cluster each information vector of the video stream into one or more secondary clusters, wherein each variable in the information vector is a clustering dimension; and

report a next frame as an anomaly when a corresponding information vector is positioned outside a secondary cluster, of the one or more secondary clusters of an information vector of a previous frame.

18. The computer programmable product as claimed in claim 17 , wherein the set of instructions when executed by a processor causes the processor to determine a cluster identification (ID) of the feature representation based on corresponding sub-class.

19. The computer programmable product as claimed in claim 17 , wherein the information vector of a feature representation is a set containing a corresponding class number, a cluster identification (ID), a date, minute of day, a day, a month, a year, and a weather associated with corresponding image frame.

20. The computer programmable product as claimed in claim 17 , wherein the set of instructions when executed by a processor detect a negative anomaly in the video stream when a regular event suddenly ceases.