IP Library › Granted Patent US 8,774,274
Granted Patent B2
US 8,774,274 · App. 13/348,262 · Granted Jul 8, 2014

Compressing and decompressing multiple, layered, video streams employing multi-directional spatial encoding

Inventors: Simon Winder (Seattle, WA); Matthew T. Uyttendaele (Seattle, WA); Charles Lawrence Zitnick, III (Seattle, WA); Richard Szeliski (Redmond, WA); Sing Bing Kang (Redmond, WA)
Assignee: Microsoft Corporation
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,774,274
App. No.
13/348,262
Granted
Jul 8, 2014
Kind
B2
Abstract

A process for compressing and decompressing non-keyframes in sequential sets of contemporaneous video frames making up multiple video streams where the video frames in a set depict substantially the same scene from different viewpoints. Each set of contemporaneous video frames has a plurality frames designated as keyframes with the remaining being non-keyframes. In one embodiment, the non-keyframes are compressed using a multi-directional spatial prediction technique. In another embodiment, the non-keyframes of each set of contemporaneous video frames are compressed using a combined chaining and spatial prediction compression technique. The spatial prediction compression technique employed can be a single direction technique where just one reference frame, and so one chain, is used to predict each non-keyframe, or it can be a multi-directional technique where two or more reference frames, and so chains, are used to predict each non-keyframe.

Claims (20)

1. A computer-implemented process for compressing video data comprising sequential sets of contemporaneous video frames wherein the video frames in a set depict substantially the same scene from different view points, said process comprising:

using a computer to perform the following process actions:

designating a plurality of keyframes for each set of contemporaneous frames; and

for each set of contemporaneous video frames in time sequence order, compressing each video frame that is not a keyframe using a combined chaining and spatial prediction compression technique, wherein chaining refers to each non-keyframe being compressed using one or more frames that have viewpoints immediately adjacent the non-keyframe as reference frames for carrying out the spatial prediction compression technique, regardless of whether they are keyframes or non-keyframes, and wherein whenever a non-keyframe is employed as a reference to encode another non-keyframe the non-keyframe employed as a reference is itself encoded using one or more other frames, exclusive of the non-keyframe using the non-keyframe employed as a reference, which have viewpoints immediately adjacent that of the non-keyframe reference as reference frames for carrying out a spatial prediction compression of the frame.

2. The process of claim 1 , wherein the spatial prediction compression technique is one of (i) a single direction technique wherein just one reference frame is used to predict each non-keyframe, or (ii) a multi-directional technique wherein two or more reference frames are used to predict each non-keyframe.

3. The process of claim 1 , wherein a chain of video frames is defined as an anchoring keyframe followed by two or more non-keyframes links such that each frame in the chain has a viewpoint that is immediately adjacent that of any preceding or succeeding frame in the chain, and wherein the process action of compressing each non-keyframe using a combined chaining and spatial prediction compression technique comprises the actions of:

identifying one or more candidate video frame chains which can be employed in compressing the non-keyframe under consideration; and

selecting the candidate chain if only one chain is employed in compressing the non-keyframe under consideration, or a prescribed number of chains if more than one chain is employed in compressing the non-keyframe under consideration, which once compressed is estimated to collectively produce the least amount of data.

4. The process of claim 1 , wherein a chain of video frames is defined as an anchoring keyframe followed by two or more non-keyframes links such that each frame in the chain has a viewpoint that is immediately adjacent that of any preceding or succeeding frame in the chain, and wherein the process action of compressing each non-keyframe using a combined chaining and spatial prediction compression technique comprises an action of selecting a prescribed number of video frame chains which can be employed in compressing the non-keyframe under consideration, wherein each chain has a different non-keyframe acting as the non-keyframe used as a reference in compressing the non-keyframe under consideration, and wherein for each non-keyframe in the chain, the non-keyframe represents the frame having a viewpoint that is the closest to the frame preceding the non-keyframe in the chain, or one of the closest if there are two or more frames with viewpoints equidistant to the frame preceding the non-keyframe in the chain.

5. The process of claim 1 , further comprising an action of decompressing each non-keyframe within each set of contemporaneous frames using a combined reverse chaining and spatial prediction decompression technique.

6. The process of claim 5 , wherein a chain of video frames is defined as an anchoring keyframe followed by two or more non-keyframes links such that each frame in the chain has a viewpoint that is immediately adjacent that of any preceding or succeeding frame in the chain, and wherein the data associated with each compressed non-keyframe comprises the identity of each frame in each chain employed in compressing the non-keyframe as well as the frame sequence of each chain, and wherein the process action of decompressing each non-keyframe within each set of contemporaneous frames using a combined reverse chaining and spatial prediction decompression technique, comprises the action of:

accessing the data associated with the non-keyframe under consideration that is being decompressed; and

for each chain employed in compressing the non-keyframe under consideration,

(i) decompressing the anchoring keyframe of the chain using a decompression scheme applicable to the compression method used to compress the keyframe,

(ii) decompressing the non-keyframe in the chain that had the keyframe as a reference frame using the spatial prediction decompression technique applicable to the spatial prediction compression technique used to compress the non-keyframe,

(iii) decompressing the non-keyframe in the chain that had the previously-decompressed non-keyframe as a reference frame using the spatial prediction decompression technique applicable to the spatial prediction compression technique used to compress the non-keyframe,

(iv) determining if the last-decompressed non-keyframe is said non-keyframe under consideration, and

(v) whenever it is determined that the last-decompressed non-keyframe is not the non-keyframe under consideration, repeating actions (iii) and (iv).

7. The process of claim 1 , wherein the spatial prediction decompression technique is one of (i) a single direction technique wherein just one reference frame is used to decode each non-keyframe, or (ii) a multi-directional technique wherein two or more reference frames are used to decode each non-keyframe.

8. A computer-readable storage medium, which does not consist of a signal, and having computer-executable instructions stored thereon for performing the process actions recited in claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 9, 2014
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 034544/0541 →
Continuity (3)
Division 11182144 · Jul 15, 2005
Continuation In Part 10910077 · Aug 3, 2004
Related Publication 20120114037A1 · May 10, 2012