Automatic video summarization

1. A device for automatic video summarization, the method comprising:

a storage device to hold a video;

a semantic classifier to generate a semantic model of the video from frames of the video;

a relevancy classifier to assign respective relevancy scores to the frames;

a multiplexer to:

initialize the semantic model with the respective relevancy scores including constructing a graph in which nodes correspond to the frames, node values correspond to respective relevancy scores of the frames, and edges are weighted by the inverse distance between frames established in the semantic model; and

process, iteratively, the semantic model to produce a set of sub-scenes, each iteration comprising the multiplexer to:

converge the semantic model following initialization;

select a sequence of frames with a highest relevancy score after converging; and

re-initialize the semantic model by fixing the relevancy scores for the selected sequence of frames.

2. The device of claim 1 , wherein generating the semantic model includes extracting features of the frames.

3. The device of claim 2 , wherein generating the semantic model includes generating a pseudo-semantic domain from the extracted features, wherein the pseudo-semantic domain is an n-dimensional space derived from the features.

4. The device of claim 3 , wherein generating the semantic model includes mapping the frames to the pseudo-semantic domain by, for each frame, deriving a coordinate in the pseudo-semantic domain, each element of the coordinate corresponding to a dimensionality of the pseudo-semantic domain and derived from a presence of a feature of the extracted features that is specific to the frame.

5. The device of claim 4 , wherein generating the semantic model includes creating a generative model from the frames mapped to the pseudo-semantic domain.

6. The device of claim 5 , wherein creating the generative model includes recursively identifying a set of key frames in the frames by adding a frame with a highest score to the set of frames, the score of a frame being the inverse of the sum of a square norm of the coordinate of the frame multiplied by a constant and divided by the square of the norm of the distance between the frame and a frame in the set of key frames for all members of the set of key frames.

7. The device of claim 1 , wherein assigning respective relevancy scores to the frames includes extracting behavioral indicators of a person who took the video from the video.

8. The device of claim 7 , wherein the behavioral indicators includes at least one of a lack of motion of a camera used to capture the video or an increased zoom of the camera used to capture the video to indicate an increased relevancy.

9. The device of claim 1 , comprising controller to:

present a user interface including the sub-scenes in an order in which they were produced;

receive a discard selection from a user for a sub-scene; and

discard the sub-scene from a video summarization.

10. A method of automatic summarization carry out by machine:

obtaining a video;

generating a semantic model of the video from frames of the video; assigning respective relevancy scores to the frames;

initializing the semantic model with the respective relevancy scores including constructing a graph in which nodes correspond to the frames, node values correspond to respective relevancy scores of the frames, and edges are weighted by the inverse distance between frames established in the semantic model: and

iteratively processing the semantic model to produce a set of sub-scenes, each iteration comprising:

converging the semantic model following initialization;

selecting a sequence of frames with a highest relevancy score after converging; and

re-initializing the semantic model by fixing the relevancy scores for the selected sequence of frames.

11. The method of claim 10 , wherein generating the semantic model includes extracting features of the frames.

12. The method of claim 11 , wherein generating the semantic model includes generating a pseudo-semantic domain from the extracted features, wherein the pseudo-semantic domain is an n-dimensional space derived from the features.

13. The method of claim 12 , wherein generating the semantic model includes mapping the frames to the pseudo-semantic domain by, for each frame, deriving a coordinate in the pseudo-semantic domain, each element of the coordinate corresponding to a dimensionality of the pseudo-semantic domain and derived from a presence of a feature of the extracted features that is specific to the frame.

14. The method of claim 10 , wherein assigning respective relevancy scores to the frames includes extracting behavioral indicators of a person who took the video from the video.

15. The method of claim 14 , wherein the behavioral indicators includes at least one of a lack of motion of a camera used to capture the video or an increased zoom of the camera used to capture the video to indicate an increased relevancy.

16. The method of claim 10 , comprising:

presenting a user interface including the sub-scenes in ark order in which they were produced;

receiving a discard selection from a user for a sub-scene; and

discarding the sub-scene from a video summarization.

17. At least one non-transitory computer-readable medium including instructions that, when executed by a machine, cause the machine to perform operations for automatic video summarization, the operations comprising:

obtaining a video;

generating a semantic model of the video from frames of the video; assigning respective relevancy scores to the frames;

initializing the semantic model with the respective relevancy scores including constructing a graph in which nodes correspond, to the frames, node values correspond to respective relevancy scores of the frames, and edges are weighted by the inverse distance between frames established in the semantic model; and

iteratively processing the semantic model to produce a set of sub-scenes, each iteration comprising:

conversing die semantic model following initialization;

selecting a sequence of frames with a highest relevancy score after converging; and

re-initializing the semantic model by fixing the relevancy scores for the selected sequence of frames.

18. The at least one machine readable medium of claim 17 , wherein generating the semantic model includes extracting features of the frames.

19. The at least one machine readable medium of claim 18 , wherein generating the semantic model includes generating a pseudo-semantic domain from the extracted features, wherein the pseudo-semantic domain is an n-dimensional space derived from the features.

20. The at least one machine readable medium of claim 19 , wherein generating the semantic model includes mapping the frames to the pseudo-semantic domain by, for each frame, deriving a coordinate in the pseudo-semantic domain, each element of the coordinate corresponding to a dimensionality of the pseudo-semantic domain and derived from a presence of a feature of the extracted features that is specific to the frame.

21. The at least one machine readable medium of claim 20 , wherein generating the semantic model includes creating a generative model from the frames mapped to the pseudo-semantic domain.

22. The at least one machine readable medium of claim 21 , wherein creating the generative model includes recursively identifying a set of key frames in the frames by adding a frame with a highest score to the set of frames, the score of a frame being the inverse of the sum of a square norm of the coordinate of the frame multiplied by a constant and divided by the square of the norm of the distance between the frame and a frame in the set of key frames for all members of the set of key frames.

23. The at least one machine readable medium of claim 17 , wherein assigning respective relevancy scores to the frames includes extracting behavioral indicators of a person who took the video from the video.

24. The at least one machine readable medium of claim 23 , wherein the behavioral indicators includes at least one of a lack of motion of a camera used to capture the video or an increased zoom of the camera used to capture the video to indicate an increased relevancy.

25. The at least one machine readable medium of claim 17 , where the instructions comprise:

presenting a user interface including the sub-scenes in an order in which they were produced;

receiving a discard selection from a user for a sub-scene; and

discarding the sub-scene from a video summarization.