Granted Patent B2

US 10,271,017 · App. 13/888,941 · Granted Apr 23, 2019

Abstract

Claims (71)

1. A method for generating an activity summary of a person in a commercial site, the method comprising:

receiving one or more videos from one or more image capture devices;

determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person;

generating a video-loop of the person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in the commercial site;

generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and

generating the activity summary of the person including the video-loop and the at least one action clip.

2. The method of claim 1 , wherein the suspicious action includes at least one of grasping an object, removing a component from the object and hiding the object.

3. The method of claim 1 , wherein generating the at least one action clip further comprises:

analyzing one or more images of the video-loop to determine the suspicious action performed by the person;

determining an image analysis score for each of the one or more images based on the analysis of the one or more images;

identifying a suspicious image from the one or more images based on the one or more image analysis scores; and

generating the at least one action clip including the identified suspicious image from the video-loop.

4. The method of claim 3 , wherein the at least one action clip begins with the identified suspicious image.

5. The method of claim 1 , wherein generating the action dip further comprises:

analyzing one or more sequences of images from the video-loop to determine one or more motion features associated with the person;

identifying a suspicious sequence of images from the one or more sequences of images based on the one or more motion features; and

generating the at least one action clip including the suspicious sequence of images from the video-loop.

6. The method of claim 1 , wherein generating the video-loop of the person further comprises:

receiving a first video of the one or more videos from a first image capture device of the one or more image capturing devices

identifying a first set of images including the person from the first video;

receiving a second video of the one or more videos from a second image capture device of the one or more image capturing devices;

identifying a second set of images including the person from the second video; and

generating the video-loop of the person by combining the first set of images and the second set of images.

7. The method of claim 1 , further comprising:

determining whether the person is approaching an exit of the commercial site; and

sending the activity summary for display in response to determining that the person is approaching the exit of the commercial site.

8. A system for generating an activity summary of a person in a commercial site, the system comprising:

at least one processor;

a tracking module stored in a memory and, executable by the at least one processor, the tracking module for receiving one or more videos from one or more image capture devices, determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person, and generating a video-loop of the person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in the commercial site;

an analysis module stored in the memory and executable by the at least one processor, the analysis module communicatively coupled to the tracking module for generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and

a summary generator stored in the memory and executable by the at least one processor, the summary generator communicatively coupled to the analysis module for generating the activity summary of the person including the video-loop and the at least one action clip.

9. The system of claim 8 , wherein the analysis module is further configured to:

analyze one or more images of the video-loop to determine the suspcious action performed by the person;

determine an image analysis score for each of the one or more images based on the analysis of the one or more images;

identify a suspicious image from the one or more images based on the one or more image analysis scores; and

generate the at least one action clip including the identified suspicious image from the video-loop.

10. The system of claim 8 , wherein the analysis module is further configured to:

analyze one or more sequences of images from the video-loop to determine one or more motion features associated with the person;

identify a suspicious sequence of images from the one or more sequences of images based on the one or more motion features; and

generate the at least one action clip including the suspicious sequence of images from the video-loop.

11. The system of claim 8 , wherein the tracking module is further configured to:

receive a first video of the one or more videos from a first image capture device of the one or more image capturing devices;

identify a first set of images including the person from the first video;

receive a second video of the one or more videos from a second image capture device of the one or more image capturing devices;

identify a second set of images including the person from the second video; and

generate the video-loop of the person by combining the first set of images and the second set of images.

12. The system of claim 8 , wherein the summary generator is further configured to determine whether the person is approaching an exit of the commercial site and send the activity summary for display in response to determining that the person is approaching the exit of the commercial site.

13. A computer program product comprising a non-transitory computer readable medium encoding instructions that, in response to execution by at least one processor, cause the processor to perform operations comprising:

receiving one or more videos from one or more image capture devices;

determining metadata associated with the person from the one or more videos, the metadata including location data of the person and an appearance descriptor that represents a spatial distribution of color corresponding to the person:

generating a video-loop of a person from the one or more videos by combining sets of images from the one or more videos based on a similarity of the metadata associated with the person, wherein the video-loop includes images of an entire trip, including all activities, of the person in a commercial site;

generating at least one action clip from the video-loop, wherein the at least one action clip includes a suspicious action performed by the person in the commercial site; and

generating an activity summary of the person including the video-loop and the at least one action clip.

14. The computer program product of claim 13 , further causing the processor to perform operations comprising:

analyzing one or more images of the video-loop to determine the suspcious action performed by the person;

determining an image analysis score for each of the one or more images based on the analysis of the one or more images;

identifying a suspicious image from the one or more images based on the one or more image analysis scores; and

generating the action dip including the identified suspicious image from the video-loop.

15. The computer program product of claim 14 , wherein the at least one action clip begins with the suspicious image.

16. The computer program product of claim 13 , further causing the processor to perform operations comprising:

analyzing one or more sequence of images from the video-loop to determine one or amore motion features associated with the person;

identifying a suspicious sequence of images from the one or more sequence of images based on the one or more motion features; and

generating the at least one action clip including the suspicious sequence of images from the video-loop.

17. The computer program product of claim 13 , further causing the processor to perform operations comprising:

determining whether the person is approaching an exit of the commercial site; and

sending the activity summary for display in response to determining that the person is approaching the exit of the commercial site.

18. The method of claim 1 , further comprising:

identifying one or more spatiotemporal interest points from the video-loop based on two-dimensional Gaussian smoothing and temporal Gabor filtering.

19. The method of claim 18 , further comprising:

analyzing a sequence of images represented by the one or more spatiotemporal interest points to determine shape features and motion features associated with the person.

20. The method of claim 19 , wherein a grasping action is determined as motion feature through an Adaboost algorithm or Fisher's linear discriminant algorithm.

Assignments (9)