IP Library Granted Patent US 9,953,681
Granted Patent B2
US 9,953,681 · App. 12/426,899 · Granted Apr 24, 2018

System and method for representing long video sequences

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,953,681
App. No.
12/426,899
Granted
Apr 24, 2018
Kind
B2
Abstract

Systems and procedures for transforming video into a condensed visual representation. An example procedure may include receiving video comprised of a plurality of frames. For each frame, the example procedure may create a first representation, reduced in one dimension, wherein a visual property of each pixel of the first representation is assigned by aggregating a visual property of the pixels of the frame having the same position in the unreduced dimension. The example procedure may further form a condensed visual representation including the first representations aligned along the reduced dimension according to an order of the frames in the video.

Claims (62)

1. A method comprising:

receiving video comprised of a plurality of frames, each frame comprised of a plurality of pixels arranged in horizontal rows and vertical columns;

for each frame, creating a first representation, reduced in a first dimension, wherein a visual property of each pixel of the first representation is assigned by aggregating a visual property of a plurality of pixels of the frame having a corresponding position in the unreduced dimension, wherein the first dimension comprises at least one of the horizontal rows and the vertical columns;

appending metadata to each first representation comprising representative parameters corresponding to the frame from which each first representation was created, wherein the metadata comprises an average color of the pixels in the frame from which the first representation was created;

forming a condensed visual representation comprising the first representation of each frame aligned along the first dimension according to an order of the plurality of frames in the video;

reducing, for each frame in the condensed visual representation, the first representation of each frame in a second dimension by grouping the pixels of each first representation into a predetermined number of blocks along the second dimension and replacing the pixels of each block with a pixel assigned by aggregating the visual property of each pixel in the block, wherein the second dimension comprises a different dimension than the first dimension;

detecting, based on the reduced condensed visual representation and metadata, at least one unexpected change in the video; and

inserting a flag marking the at least one unexpected change.

2. The method of claim 1 , wherein the predetermined number of blocks is determined based on one of a dimension of the frames, an amount of visual information contained in each frame, and a visual characteristic of the video.

3. The method of claim 1 , wherein an equal number of pixels is grouped into each block.

4. The method of claim 1 , wherein an unequal number of pixels is grouped into each block.

5. The method of claim 4 , wherein blocks containing pixels from a center of each first representation contain fewer pixels than blocks containing pixels from an outside of each first representation.

6. The method of claim 1 , wherein aggregating the visual property of each pixel in the block comprises averaging a color of each pixel in the block.

7. The method of claim 1 , wherein the first dimension is a horizontal dimension.

8. The method of claim 1 , wherein the second dimension is a vertical dimension.

9. The method of claim 1 , wherein each first representation is one pixel wide in the reduced dimension.

10. The method of claim 1 , wherein each first representation is created for a group of two or more frames.

11. The method of claim 1 , wherein the metadata comprises at least one of a standard deviation of a color of the pixels in the frame from which the first representation was created from the average color and a time stamp associated with the frame from which the first representation was created.

12. The method of claim 1 , wherein a tag is automatically generated identifying a first representation having a tagged property.

13. The method of claim 1 , wherein aggregating a visual property of the plurality of pixels of the frame comprises averaging a color of each pixel.

14. The method of claim 1 , wherein the predetermined number of blocks is received as a user selection.

15. A system comprising:

a processor;

an input device in communication with the processor, the input device configured to receive video comprised of a plurality of frames, each frame comprised of a plurality of pixels arranged in horizontal rows and vertical columns; and

an output device in communication with the processor, the output device configured to output a condensed visual representation;

a non-transitory, computer-readable storage medium in operable communication with the processor, wherein the computer-readable storage medium contains one or more programming instructions that, when executed, cause the processor to:

create a first representation reduced in a first dimension, for each frame in the video, wherein a visual property of each pixel of the first representation is assigned by aggregating a visual property of a plurality of pixels of the frame having a corresponding position in the unreduced dimension, wherein the first dimension comprises at least one of the horizontal rows and the vertical columns;

append metadata to each first representation comprising representative parameters corresponding to the frame from which each first representation was created, wherein the metadata comprises an average color of the pixels in the frame from which the first representation was created;

align the first representation of each frame along the first dimension according to an order of the plurality of frames in the video to form the condensed visual representation;

reduce, for each frame in the condensed visual representation, the first representation of each frame in a second dimension by grouping the pixels of each first representation into a predetermined number of blocks along the second dimension and replacing the pixels of each block with a pixel assigned by aggregating the visual property of each pixel in the block, wherein the second dimension comprises a different dimension than the first dimension;

detect, based on the reduced condensed visual representation and metadata, at least one unexpected change in the video; and

insert a flag marking the at least one unexpected change.

16. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to determine the predetermined number of blocks based on one of a dimension of the frames, an amount of visual information contained in each frame, a visual characteristic of the video.

17. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to group an equal number of pixels into each block.

18. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to group an unequal number of pixels into each block.

19. The system of claim 18 , wherein blocks containing pixels from a center of each first representation contain fewer pixels than blocks containing pixels from an outside of each first representation.

20. The system of claim 15 , wherein aggregating the visual property of each pixel in the block comprises averaging a color of each pixel in the block.

21. The system of claim 15 , wherein the first dimension is a horizontal dimension.

22. The system of claim 15 , wherein the second dimension is a vertical dimension.

23. The system of claim 15 , wherein each first representation is one pixel side in the reduced dimension.

24. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to create each first representation for a group of two or more frames.

25. The system of claim 15 , wherein the metadata comprises at least one of a standard deviation of a color of the pixels in the frame from which the first representation was created from the average color and a time stamp associated with the frame from which the first representation was created.

26. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to generate a tag identifying a first representation having a tagged property.

27. The system of claim 15 , wherein aggregating a visual property of the plurality of pixels of the frame comprises averaging a color of each pixel.

28. The system of claim 15 , wherein the computer-readable storage medium contains further one or more programming instructions that, when executed, cause the processor to receive a user selection indicating the predetermined number of blocks.

29. An article of manufacture comprising a non-transitory computer readable medium containing a plurality of machine-executable instructions, which, when executed by a computer, are configured to cause the computer to:

receive video comprised of a plurality of frames, each frame comprised of a plurality of pixels arranged in horizontal rows and vertical columns;

for each frame, create a first representation, reduced in a first dimension, wherein a visual property of each pixel of the first representation is assigned by aggregating a visual property of a plurality of pixels of the frame having a corresponding position in the unreduced dimension, wherein the first dimension comprises at least one of the horizontal rows and the vertical columns;

append metadata to each first representation comprising representative parameters corresponding to the frame from which each first representation was created, wherein the metadata comprises an average color of the pixels in the frame from which the first representation was created;

form a condensed visual representation comprising the first representation of each frame aligned along the first dimension according to an order of the plurality of frames in the video;

reduce, for each frame in the condensed visual representation, the first representation of each frame in a second dimension by grouping the pixels of each first representation into a predetermined number of blocks along the second dimension and replacing the pixels of each block with a pixel assigned by aggregating the visual property of each pixel in the block, wherein the second dimension comprises a different dimension than the first dimension;

detect, based on the reduced condensed visual representation and metadata, at least one unexpected change in the video; and

insert a flag marking the at least one unexpected change.

30. A system comprising:

means for receiving video comprised of a plurality of frames, each frame comprised of a plurality of pixels arranged in horizontal rows and vertical columns;

means for creating a first representation reduced in a first dimension, for each frame in the video, wherein a visual property of each pixel of the first representation is assigned by aggregating a visual property of a plurality of pixels of the frame having a corresponding position in the unreduced dimension, wherein the first dimension comprises at least one of the horizontal rows and the vertical columns;

means for appending metadata to each first representation comprising representative parameters corresponding to the frame from which each first representation was created, wherein the metadata comprises an average color of the pixels in the frame from which the first representation was created;

means for aligning the first representation of each frame along the first dimension according to an order of the plurality of frames in the video to form the condensed visual representation;

means for reducing, for each frame in the condensed visual representation, the first representation of each frame in a second dimension by grouping the pixels of each first representation into a predetermined number of blocks along the second dimension and replacing the pixels of each block with a pixel assigned by aggregating the visual property of each pixel in the block, wherein the second dimension comprises a different dimension than the first dimension;

means for outputting the condensed visual representation;

means for detecting, based on the reduced condensed visual representation and metadata, at least one unexpected change in the video; and

means for inserting a flag marking the at least one unexpected change.

Assignments (6)
CHANGE OF NAME Recorded Mar 31, 2026
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 075303/0717 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO CORPORATION
To: TIVO LLC
Reel/Frame 069083/0230 →
CHANGE OF NAME Recorded Oct 1, 2024
From: TIVO LLC
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 069083/0311 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 3, 2020
From: VISIBLE WORLD, LLC
To: TIVO CORPORATION
Reel/Frame 054588/0461 →
CHANGE OF NAME Recorded Oct 11, 2018
From: VISIBLE WORLD INC.
To: VISIBLE WORLD, LLC
Reel/Frame 047216/0534 →