IP Library Granted Patent US 8,786,785
Granted Patent B2
US 8,786,785 · App. 13/080,494 · Granted Jul 22, 2014

Video signature

Inventors: Seth Hendrickson (Bellevue, WA); George Randell Dong (Issaquah, WA); Saswata Mandal (Bellevue, WA); Charles Lawrence Zitnick, III (Seattle, WA)
Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,786,785
App. No.
13/080,494
Granted
Jul 22, 2014
Kind
B2
Abstract

Video signatures are generated when a small change in video information between consecutive frames N−1 and N is followed by a large change in video information between consecutive frames N and N+1. Information from frames N and/or N+1 is used to form a video signature.

Claims (46)

1. A method of finding a signature for a video, the method comprising:

dividing a frame N of the video into a plurality of cells;

calculating a video attribute of each cell of frame N;

dividing a frame N+1 into the plurality of cells;

calculating a video attribute of each cell of frame N+1;

dividing a frame N−1 of the video into a plurality of cells;

calculating a video attribute of each cell of frame N−1;

calculating a difference metric between frame N and frame N+1 that includes, for each cell of the plurality of cells, a difference between the video attribute of frame N and the video attribute of frame N+1;

calculating a difference metric between frame N and frame N−1 that includes, for each cell of the plurality of cells, a difference between the video attribute of frame N and the video attribute of frame N−1; and

generating a video signature that includes information from one or both of frame N and frame N+1 responsive to the difference metric between frame N and frame N+1 exceeding a first threshold and a difference metric between frame N and frame N−1 not exceeding a second threshold.

2. The method of claim 1 , wherein the plurality of cells includes M cells, and where an M-dimensional vector for each frame includes the video attribute for each of the plurality of M cells.

3. The method of claim 2 , wherein calculating the difference metric between frame N and frame N+1 includes calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N+1.

4. The method of claim 1 , where the video attribute is an average luminance for that cell.

5. The method of claim 1 , where the video attribute is an average luminance for that cell normalized with respect to an average luminance for all cells of that frame.

6. The method of claim 1 , where the video signature includes each of the video attributes of frame N concatenated with each of the video attributes of frame N+1.

7. The method of claim 1 , where dividing frame N of the video into the plurality of cells includes dividing the frame N into a four by four grid of cells.

8. The method of claim 1 , where frame N+1 immediately follows frame N and frame N−1 immediately precedes frame N.

9. The method of claim 1 , wherein frame N−1 is divided into M cells and an M-dimensional vector for frame N−1 includes the video attribute for each cell, and wherein calculating the difference metric between frame N and frame N−1 includes calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N−1.

10. The method of claim 1 , comprising associating a timestamp with the video signature.

11. The method of claim 1 , wherein the first threshold and the second threshold are equal.

12. A method for generating signatures of a video comprising:

dividing a frame N of the video into a grid of M cells comprising an average luminance value for each cell of frame N;

dividing a frame N+1 into the grid of M cells comprising an average luminance value for each cell of frame N+1;

dividing a frame N−1 into the grid of M cells comprising an average luminance for each cell of frame N−1;

calculating an M-dimensional vector for each of frame N, frame N+1, and frame N−1 that includes, for each different dimension, a luminance value of a different cell from that frame;

calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N+1;

calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N−1; and

generating a video signature that includes the M-dimensional vector for frame N concatenated with the M-dimensional vector of frame N+1 responsive to the Euclidean distance between frame N and frame N+1 exceeding a first threshold and the Euclidean distance between frame N and frame N−1 not exceeding a second threshold.

13. The method of claim 12 , wherein the average luminance value for each cell of a frame is normalized with respect to an average luminance for all cells of that frame.

14. The method of claim 12 , wherein the grid of M cells is a four by four grid.

15. The method of claim 12 , further comprising iteratively advancing to subsequent frames to search for additional signatures within the video.

16. The method of claim 12 , comprising associating a timestamp with the video signature.

17. A data-holding subsystem holding instructions executable by a logic subsystem to:

iteratively test a series of three consecutive frames within a video for a shot change, each test comprising:

dividing a frame N of the video into a grid comprising M cells;

calculating an M-dimensional vector for frame N that includes, for each different dimension of the M-dimensional vector, an average luminance value of a different cell from frame N;

dividing a frame N+1 of the video into a grid comprising M cells;

calculating an M-dimensional vector for frame N+1 that includes, for each different dimension of the M-dimensional vector, an average luminance value of a different cell from frame N+1;

dividing a frame N−1 of the video into a grid comprising M cells;

calculating an M-dimensional vector for frame N−1 that includes, for each different dimension of the M-dimensional vector, an average luminance value of a different cell from frame N−1;

calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N+1;

if the Euclidean distance between frame N and frame N+1 exceeds a first threshold, calculating a Euclidean distance between the M-dimensional vector for frame N and the M-dimensional vector for frame N−1;

generating a video signature that includes the M-dimensional vector for frame N concatenated with the M-dimensional vector of frame N+1 responsive to the Euclidean distance between frame N and frame N−1 not exceeding a second threshold, and iteratively test a next series of three consecutive frames within the video for a next shot change.

18. The data-holding subsystem of claim 17 , wherein the average luminance value for each cell of a frame is normalized with respect to an average luminance for all cells of that frame.

19. The data-holding subsystem of claim 17 , comprising associating a timestamp with the video signature.

20. The data-holding subsystem of claim 17 , wherein the grid is a four by four grid.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034544/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 6, 2011
From: HENDRICKSON, SETH; DONG, GEORGE RANDELL; MANDAL, SASWATA; ZITNICK, CHARLES LAWRENCE, III
To: MICROSOFT CORPORATION
Reel/Frame 026080/0015 →
Continuity (1)
Related Publication 20120257116A1 · Oct 11, 2012