IP Library Granted Patent US 11,457,198
Granted Patent B2
US 11,457,198 · App. 16/960,948 · Granted Sep 27, 2022

Adaptive frame packing for 360-degree video coding

Inventors: Philippe Hanhart (La Conversion, CH); Yuwen He (San Diego, CA); Yan Ye (San Diego, CA)
Assignee: VID SCALE, Inc.
H04N13/161H04N19/172H04N19/186H04N19/593H04N19/597
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,457,198
App. No.
16/960,948
Granted
Sep 27, 2022
Kind
B2
Abstract

A video coding device may be configured to periodically select the frame packing configuration (e.g., face layout and/or face rotations parameters) associated with a RAS, The device may receive a plurality of pictures, which may each comprise a plurality of faces. The pictures may be grouped Into a plurality of RASs. The device may select a frame packing configuration with the lowest cost for a first RAS. For example, the cost of a frame packing configuration may be determined based on the first picture of the first RAS. The device may select a frame packing configuration for a second RAS. The frame packing configuration for the first RAS may be different than the frame packing configuration for the second RAS. The frame packing configuration for the first RAS and the frame packing configuration for the second RAS may be signaled in the video bitstream.

Claims (35)

1. A device comprising:

a processor configured to:

obtain a plurality of pictures grouped into a plurality of random access segments (RASs), a picture comprising a plurality of faces, wherein a frame packing configuration corresponding to the plurality of pictures grouped into the plurality of RASs is updated periodically;

obtain a first frame packing configuration that indicates a face layout and a face rotation for a first RAS;

obtain, for a second RAS, a second frame packing configuration that is different than the first frame packing configuration for the first RAS; and

include a first indication of the first frame packing configuration for the first RAS and a second indication of the second frame packing configuration for the second RAS in a video bitstream.

2. The device of claim 1 , wherein to obtain the first frame packing configuration for the first RAS, the processor is configured to:

determine a gradient for one or more faces comprised in a first picture of the first RAS;

determine potential costs associated with coding the first picture in a plurality of frame packing configurations based on the gradient for the one or more faces, wherein the plurality of frame packing configurations indicates a position and a rotation for the one or more faces comprised in the first picture; and

obtain a frame packing configuration having a lowest cost from the plurality of frame packing configurations.

3. The device of claim 2 , wherein the first picture of the first RAS is an intra coded picture.

4. The device of claim 2 , wherein the gradient for a face comprised in the first picture of the first RAS is a dominant gradient.

5. The device of claim 2 , wherein the gradient for the face comprised in the first picture of the first RAS is computed on luma samples.

6. A method comprising:

obtaining a plurality of pictures grouped into a plurality of random access segments (RASs), a picture comprising a plurality of faces, wherein a frame packing configuration corresponding to the plurality of pictures grouped into the plurality of RASs is updated periodically;

obtaining a first frame packing configuration that indicates a face layout and a face rotation for a first RAS;

obtaining, for a second RAS, a second frame packing configuration that is different than the first frame packing configuration for the first RAS; and

including a first indication of the first frame packing configuration for the first RAS and a second indication of the second frame packing configuration for the second RAS in a video bitstream.

7. The method of claim 6 , wherein obtaining the first frame packing configuration for the first RAS comprises:

determining a gradient for one or more faces comprised in a first picture of the first RAS;

determining potential costs associated with coding the first picture in a plurality of frame packing configurations based on the gradient for the one or more faces, wherein the plurality of frame packing configurations indicates a position and a rotation for the one or more faces comprised in the first picture; and

obtaining a frame packing configuration having a lowest cost from the plurality of frame packing configurations.

8. The method of claim 7 , wherein the first picture of the first RAS is an intra coded picture.

9. The method of claim 7 , wherein the gradient for a face comprised in the first picture of the first RAS is a dominant gradient.

10. The method of claim 7 , wherein the gradient for a face comprised in the first picture of the first RAS is computed on luma samples.

11. A device comprising:

a processor configured to:

obtain a plurality of indications of a plurality of frame packing configurations corresponding to a plurality of pictures grouped into a plurality of random access segments (RAS), wherein a picture comprises a plurality of faces;

periodically change a frame packing configuration from the plurality of frame packing configurations corresponding to the plurality of pictures grouped into the plurality of RASs; and

obtain a location of face boundaries associated with the plurality of faces in the picture based on the periodically changed frame packing configuration.

12. The device of claim 11 , wherein to periodically change the frame packing configuration from the plurality of frame packing configurations corresponding to the plurality of pictures grouped into the plurality of RASs, the processor is configured to cycle through the plurality of frame packing configurations.

13. The device of claim 11 , wherein the plurality of frame packing configurations indicates a position and a rotation for a face associated with the corresponding plurality of RAS.

14. The device of claim 11 , wherein the obtained location of face boundaries comprises locations of continuous and discontinuous face boundaries associated with the plurality of faces in the picture.

15. The device of claim 1 , wherein the first indication indicates the first frame packing configuration associated with the first RAS, and wherein the second indication indicates the second frame packing configuration associated with the second RAS.

16. The device of claim 6 , wherein the first indication indicates the first frame packing configuration associated with the first RAS, and wherein the second indication indicates the second frame packing configuration associated with the second RAS.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 5, 2024
From: HANHART, PHILIPPE; HE, YUWEN; YE, YAN
To: VID SCALE, INC.
Reel/Frame 069135/0452 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: VID SCALE, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 068284/0031 →
Cited By (2)
US 12,278,948 US 12,382,015