IP Library › Granted Patent US 7,058,130

Granted Patent B2

US 7,058,130 · App. 10/015,040 · Granted Jun 6, 2006

Scene change detection

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 7,058,130
App. No.: 10/015,040
Granted: Jun 6, 2006
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

In one embodiment, a scene change detection component of a video encoder uses motion vectors to determine whether a scene change occurs in a video sequence. The scene change detection component uses field motion vectors determined by a motion estimator and compares the field motion vectors to a threshold to determine whether a scene change occurs. In another embodiment, if a scene change occurs, the video encoder can begin a new Group of Pictures (GOP).

Claims (118)

1. A method for determining a scene change in a video sequence during encoding of the video sequence, the method comprising:

receiving a first video frame, a second video frame and a third video frame;

determining a first set of motion vectors between the first video frame and the second video frame and a second set of motion vectors using the third video frame, the motion vectors havin magnitudes; and

comparing a ratio of the magnitudes of the first and second sets of motion vectors to a first threshold to determine whether a scene change has occurred, the occurrence of a scene change causing the first frame to be encoded as a different type of frame.

2. The method of claim 1 wherein the first video frame precedes the second video frame and the second video frame precedes the third video frame.

3. The method of claim 1 wherein the third video frame precedes the first and second video frames and the second video frame precedes the first video frame.

4. The method of claim 1 , wherein:

the first video frame comprises a first and second field;

the second video frame comprises a first and second field; and

the third video frame comprises a first and second field,

the first set of motion vectors comprises a first subset of motion vectors between the first field of the first video frame and the first field of the second video frame and a second subset of motion vectors between the second field of the first video frame and the second field of the second video frame; and

the second set of motion vectors comprises a first subset of motion vectors between the first field of the second video frame and the first field of the third video frame and a second subset of motion vectors between the second field of the second video frame and the second field of the third video frame.

5. The method of claim 4 , wherein comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and the first subset of the second set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the first set of motion vectors and the second subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the first video frame and second video frame if the first and second ratios are larger than the first threshold.

6. The method of claim 4 , wherein comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and the first subset of the second set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the first set of motion vectors and the second subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the first video frame and second video frame if the first and second ratios are larger than the first threshold and the sum of the magnitudes of members of the first subset of the first set of motion vectors is greater than a second threshold.

7. The method of claim 4 , wherein if second frame is a P-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold.

8. The method of claim 4 , wherein if the second frame is a P-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold and the sum of the magnitudes of members of the first subset of the second set of motion vectors is greater than a second threshold.

9. The method of claim 1 , wherein:

the first video frame comprises a first and second field;

the second video frame comprises a first and second field; and

the third video frame comprises a first and second field,

the second set of motion vectors comprises a first subset of motion vectors between the first field of the first video frame and the first field of the third video frame and a second subset of motion vectors between the second field of the first video frame and the second field of the third video frame.

10. The method of claim 9 , wherein if the first frame is an I-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and a sum of the first subset of the first set of motion vectors minus the second subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the first subset of the second set of motion vectors and the sum of the first subset of the second set of motion vectors minus the second subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the first video frame and second video frame if the first and second ratios are larger than the first threshold.

11. The method of claim 9 , wherein if the first frame is an I-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and the sum of the first subset of the first set of motion vectors minus the second subset of the first set of motion vectors to the first threshold;

determining there is a scene change between the first and second video frame if the first and second ratios are larger than the first threshold and if the sum of magnitudes of members of the first subset of the first set of motion vectors is greater than a second threshold.

12. The method of claim 9 , wherein if the second frame is a B-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold.

13. The method of claim 9 , wherein if the second frame is a B-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold and if a sum of members of the first subset of the second set of motion vectors is larger than a second threshold.

14. The method of claim 1 , wherein if a scene change is detected further comprising:

beginning a new group of pictures (GOP) at a point after the scene change.

15. The method of claim 14 , wherein if a scene change occurs at a frame following an I-frame further comprising:

converting the I-frame to a P-frame and converting a following P-frame to an I-frame.

16. The method of claim 1 , wherein the threshold is a heuristically determined value.

17. A video device to encode a video sequence comprising:

an input configured to receive a first video frame, a second video frame, and a third video frame; and

a processor configured to determine a first set of motion vectors between the first video frame and the second video frame and a second set of motion vectors using the third video frame, the motion vectors having magnitudes, and compare a ratio of the magnitudes of the first and second sets of motion vectors to a first threshold to determine whether a scene change has occurred, the occurrence of a scene change causing the first frame to be encoded as a different type of frame.

18. The video device of claim 17 , wherein:

the first video frame comprises a first and second field;

the second video frame comprises a first and second field;

the third video frame comprises a first and second field;

19. The video device of claim 18 wherein if second frame is a P-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold.

20. The video device of claim 18 , wherein if the second frame is a P-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the second set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the second subset of the first set of motion vectors to the first threshold; and

21. The video device of claim 17 , wherein:

the first video frame comprises a first and second field;

the second video frame comprises a first and second field;

the third video frame comprises a first and second field;

22. The video device of claim 17 , wherein if the first frame is an I-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

determining there is a scene change between the first video frame and second video frame if the first and second ratios are larger than the first threshold.

23. The video device of claim 17 , wherein if the first frame is an I-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

24. The video device of claim 17 , wherein if the first frame is a B-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the second subset of the first set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the first subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold.

25. The video device of claim 17 , wherein if the first frame is a B-frame, comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the second subset of the first set of motion vectors and the first subset of the first set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the second set of motion vectors and the first subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the second video frame and third video frame if the first and second ratios are larger than the first threshold and if a sum of members of the second subset of the first set of motion vectors is larger than a second threshold.

26. A computer readable medium storing executable computer program instructions which, when executed by a processor, cause the processor to perform a method to encode a video sequence comprising:

receiving a first video frame, a second video frame and a third video frame; determining a first set of motion vectors between the first video frame and the second video frame and a second set of motion vectors using the third video frame, the motion vectors having magnitudes; and

27. The computer readable medium as set forth in claim 26 , wherein the first video frame comprises a first and second field;

the second video frame comprises a first and second field; and

the third video frame comprises a first and second field,

28. The computer readable medium as set forth in claim 27 , wherein comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and the first subset of the second set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the first set of motion vectorsand the second subset of the second set of motion vectors to the first threshold; and

determining there is a scene change between the first video frame and second video frame if the first and second ratios are larger than the first threshold.

29. The computer readable medium as set forth in claim 27 , wherein comparing a ratio of the magnitudes of the first and second sets of motion vectors further comprises:

comparing a first ratio of the first subset of the first set of motion vectors and the first subset of the second set of motion vectors to the first threshold;

comparing a second ratio of the second subset of the first set of motion vectors and the second subset of the second set of motion vectors to the first threshold; and

30. An apparatus for encoding a video sequence comprising:

means for receiving a first video frame, a second video frame and a third video frame;

means for determining a first set of motion vectors between the first video frame and the second video frame and a second set of motion vectors using the third video frame, the motion vectors having magnitudes; and

means for comparing a ratio of the magnitudes of the first and second sets of motion vectors to a first threshold to determine whether a scene change has occurred, the occurrence of a scene change causing the first frame to be encoded as a different type of frame.

31. The apparatus of claim 30 , wherein if a scene change is detected further comprising:

beginning a new group of pictures (GOP) at a point afier the scene change.

32. The apparatus of claim 30 , wherein if a scene change occurs at a frame following an I-frame further comprising:

converting the I-frame to a P-frame and converting a following P-frame to an I-frame.

Assignments (2)

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2019

From: SONY CORPORATION

To: SATURN LICENSING LLC

Reel/Frame 048974/0222 →

ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2015

From: SONY ELECTRONICS INC.

To: SONY CORPORATION

Reel/Frame 036330/0420 →