IP Library Granted Patent US 11,991,376
Granted Patent B2
US 11,991,376 · App. 17/226,510 · Granted May 21, 2024

Switchable scalable and multiple description immersive video codec

Inventors: Jill Boyce (Portland, OR); Basel Salahieh (Santa Clara, CA)
Assignee: Intel Corporation
H04N19/39H04N19/172H04N19/182
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,991,376
App. No.
17/226,510
Granted
May 21, 2024
Kind
B2
Abstract

Techniques related to immersive video coding are discussed. Such techniques include encoding an immersive video bitstream by applying scalable video coding to some portions of the immersive video and multiple description coding to other portions of the immersive video and decoding and/or rendering the immersive video bitstream using selective scalable video coding and multiple description coding responsive to indicators in the bitstream.

Claims (32)

1. A decoder system comprising:

a memory to store at least a portion of an immersive video of a bitstream comprising a plurality of pixel values each associated with an immersive video view, the bitstream comprising a first indicator indicating a switch between scalable coding and multiple descriptor coding to be used in a first portion of the immersive video and a second indicator indicating a switch between scalable coding and multiple descriptor coding to be used in a second portion of the immersive video; and

processor circuitry coupled to the memory, the processor circuitry to:

generate, in response to the first indicator indicating multiple descriptor coding, the first portion of the immersive video by merging first and second decoded pixel samples in the first portion; and

generate, in response to the second indicator indicating scalable coding, the second portion of the immersive video by summing third and fourth decoded pixel samples in the second portion.

2. The system of claim 1 , wherein the processor circuitry to generate the second portion of the immersive video comprises the processor circuitry to sum the third and fourth decoded pixel samples and remove an offset.

3. The system of claim 1 , wherein the first indicator comprises an indicator for one of a video sequence, a picture, an access unit, an atlas, or a patch.

4. The system of claim 1 , wherein the first indicator comprises a flag coded as a single bit or coded using arithmetic coding.

5. The system of claim 1 , wherein the third pixel sample corresponds to a low frequency layer and the fourth pixel sample corresponds to a high frequency layer of the first portion of the immersive video.

6. The system of claim 1 , wherein the first and second portions comprise one of a patch of immersive video, an immersive video atlas, or an immersive video view.

7. The system of claim 1 , wherein the first portion of the immersive video is generated by one of weighted averaging the first and second decoded pixel samples, simple averaging of the first and second decoded pixel samples, spatially merging the first and second decoded pixel samples, or temporally merging the first and second decoded pixel samples.

8. The system of claim 1 , wherein the third decoded pixel sample comprises a pixel value and the fourth decoded pixel sample comprises a residual value.

9. The system of claim 1 , wherein the third pixel sample corresponds to a near layer of the of the first portion of the immersive video and the fourth pixel sample corresponds to a far layer of the of the first portion of the immersive video.

10. The system of claim 1 , the processor circuitry to:

generate a third portion of the immersive video based only on single pixel samples for each pixel of the third portion.

11. The system of claim 1 , the processor circuitry to:

present at least part of the first portion and at least part of the second video portions to a user via an immersive video display device.

12. A method comprising:

receiving an immersive video of a bitstream comprising a plurality of pixel values each associated with an immersive video view, the bitstream comprising a first coding indicator indicating a switch between scalable coding and multiple descriptor coding to be used in a first portion of the immersive video and a second indicator indicating a switch between scalable coding and multiple descriptor coding to be used in a second portion of the immersive video;

generating, in response to the first indicator indicating multiple descriptor coding, the first portion of the immersive video by merging first and second decoded pixel samples corresponding to the first portion; and

generating, in response to the second indicator indicating scalable coding, the second portion of the immersive video by summing third and fourth decoded pixel samples corresponding to the second portion.

13. The method of claim 12 , wherein generating the second portion of the immersive video comprises summing the third and fourth decoded pixel samples and removing an offset.

14. The method of claim 12 , wherein the first indicator comprises an indicator for one of a video sequence, a picture, an access unit, an atlas, or a patch.

15. The method of claim 12 , wherein the first indicator comprises a flag coded as a single bit or coded using arithmetic coding.

16. The method of claim 12 , wherein the third pixel sample corresponds to a low frequency layer and the fourth pixel sample corresponds to a high frequency layer of the first portion of the immersive video.

17. At least one non-transitory machine readable medium comprising a plurality of instructions that, in response to being executed on a computing device, cause the computing device to generate immersive video by:

receiving an immersive video of a bitstream comprising a plurality of pixel values each associated with an immersive video view, the bitstream comprising a first indicator indicating a switch between scalable coding and multiple description coding to be used in a first portion of the immersive video and a second indicator indicating a switch between scalable coding and multiple description coding to be used in a second portion of the immersive video;

generating, in response to the first scalable or multiple descriptor coding indicator indicating multiple descriptor coding, the first portion of the immersive video by merging first and second decoded pixel samples corresponding to the first portion; and

generating, in response to the second scalable or multiple descriptor coding indicator indicating scalable coding, the second portion of the immersive video by summing third and fourth decoded pixel samples corresponding to the second portion.

18. The machine readable medium of claim 17 , wherein generating the second portion of the immersive video comprises summing the third and fourth decoded pixel samples and removing an offset.

19. The machine readable medium of claim 17 , wherein the first indicator comprises an indicator for one of a video sequence, a picture, an access unit, an atlas, or a patch.

20. The machine readable medium of claim 17 , wherein the first indicator comprises a flag coded as a single bit or coded using arithmetic coding.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 20, 2021
From: BOYCE, JILL; SALAHIEH, BASEL
To: INTEL CORPORATION
Reel/Frame 057240/0019 →
Continuity (2)
Provisional Application 63007859 · Apr 9, 2020
Related Publication 20210258590A1 · Aug 19, 2021
Cited By (1)
US 12,597,170