IP Library › Granted Patent US 11,558,608
Granted Patent B2
US 11,558,608 · App. 17/512,162 · Granted Jan 17, 2023

On split prediction

Inventors: Yue Li (San Diego, CA); Li Zhang (San Diego, CA); Jizheng Xu (San Diego, CA)
Assignee: LEMON INC.
H04N19/105G06T9/002G06T9/40G06V10/95H04N19/119H04N19/159H04N19/176H04N19/96
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,608
App. No.
17/512,162
Granted
Jan 17, 2023
Kind
B2
Abstract

A method implemented by a coding apparatus. The method includes obtaining probabilities of split types being implemented when partitioning a picture, and skipping one or more of the split types based on the probabilities obtained when a coding block is partitioned during a conversion between a video media file and a bitstream. A corresponding apparatus and non-transitory computer readable medium are also provided.

Claims (28)

1. A method implemented by a video coding apparatus, comprising:

obtaining probabilities of split types being implemented when partitioning a picture;

determining which of the split types are below a predetermined threshold based on the probabilities; and

skipping one or more of the split types based on the probabilities obtained when a coding block of the picture is partitioned during a conversion between a video media file and a bitstream, wherein the one or more of the split types determined to be below the predetermined threshold is skipped when the coding block is being partitioned.

2. The method of claim 1 , wherein obtaining probabilities of split types being implemented further comprises deriving probabilities of split types being implemented based on probabilities of split type boundaries coinciding with subblock boundaries of the coding block.

3. The method of claim 1 , wherein the split types comprise a non-split type, a quad tree split type, a binary tree split type, and a ternary tree split type.

4. The method of claim 2 , wherein a probability of a j-th subblock boundary from the subblock boundaries is represented as {p j Non , p j QT , p j BT , p j TT }, where p represents the probability of the split type, where Non represents a non-split split type, QT represents a quad tree split type, BT represents a binary tree split type, TT represents a ternary tree split type, where j is an element of {1, 2, 3, . . . K), and where K represents a total number of the subblock boundaries in the coding block.

5. The method of claim 2 , wherein the probabilities of split type boundaries coinciding with subblock boundaries are based on an a×b probability vector, where a represents a total number of split types for a subblock of the coding block, and where b represents a total number of the subblock boundaries in the coding block.

6. The method of claim 2 , wherein the probabilities of split types being implemented is based on all of the subblock boundaries in the coding block, a subset of the subblock boundaries in the coding block, an average of probabilities, or a weighted average of probabilities.

7. The method of claim 1 , wherein one probability from the probabilities of the split types is derived based on avg{p i 1 ST , p i 2 ST , p i 3 ST . . . , p i x ST }, where p represents the probability of the split type, where i represents a subblock boundary from the subblock boundaries, where ST represents one of a quad tree split type, a horizontal binary tree split type, a vertical binary tree split type, a horizontal ternary tree split type, and a vertical ternary tree split type, and wherein x represents a number of the subblock boundaries in the coding block.

8. The method of claim 1 , wherein obtaining probabilities of split types being implemented comprises receiving an input from a convolutional neural network identifying all possible partitioning structures for the coding block considering a predetermined number of split types and a predetermined depth.

9. The method of claim 8 , further comprising deriving a probability distribution for all of the possible partitioning structures based on the input, wherein the probability distribution includes a probability of each of the split types being implemented.

10. The method of claim 9 , wherein the input identifying all possible partitioning structures for the coding block is based on a sum of possible splits for a no-split type, a quad tree split type, a horizontal binary tree split type, a vertical binary tree split type, a horizontal ternary tree split type, and a vertical ternary tree split type.

11. The method of claim 1 , wherein obtaining probabilities of split types being implemented comprises receiving an input from a convolutional neural network, the input identifying split types for the coding block.

12. The method of claim 11 , further comprising determining which of the split types are above a predetermined threshold, and examining a partitioning result of each subblock in the coding block using the split types determined to be above the predetermined threshold.

13. An apparatus for coding video data comprising one or more processors and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor cause the processor to:

obtain probabilities of split types being implemented;

determine which of the split types are below a predetermined threshold based on the probabilities; and

skip one or more of the split types based on the probabilities obtained when a coding block is partitioned, wherein the one or more of the split types determined to be below the predetermined threshold is skipped when the coding block is being partitioned.

14. The apparatus of claim 13 , wherein the one or more processors are further configured to derive probabilities of split types being implemented based on probabilities of split type boundaries coinciding with subblock boundaries of the coding block.

15. The apparatus of claim 13 , wherein the one or more processors are further configured to receive an input from a convolutional neural network identifying all possible partitioning structures for the coding block considering a predetermined number of split types and a predetermined depth.

16. The apparatus of claim 13 , wherein the one or more processors are further configured to determine which of the split types are above a predetermined threshold, and examine a partitioning result of each subblock in the coding block using the split types determined to be above the predetermined threshold.

17. A non-transitory computer readable medium comprising a computer program product for use by a coding apparatus, the computer program product comprising computer executable instructions stored on the non-transitory computer readable medium that, when executed by one or more processors, cause the coding apparatus to:

obtain probabilities of split types being implemented;

determine which of the split types are below a predetermined threshold based on the probabilities; and

skip one or more of the split types based on the probabilities obtained when a coding block is partitioned, wherein the one or more of the split types determined to be below the predetermined threshold is skipped when the coding block is being partitioned.

18. The non-transitory computer readable medium of claim 17 , wherein the instructions further cause the coding apparatus to derive probabilities of split types being implemented based on probabilities of split type boundaries coinciding with subblock boundaries of the coding block.

19. The non-transitory computer readable medium of claim 18 , wherein the instructions further cause the coding apparatus to derive a probability distribution for all possible partitioning structures based on an input from a convolutional neural network identifying all possible partitioning structures for the coding block considering a predetermined number of split types and a predetermined depth, wherein the probability distribution includes a probability of each of the split types being implemented.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2022
From: BYTEDANCE INC.
To: LEMON INC.
Reel/Frame 059895/0552 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2022
From: LI, YUE; ZHANG, LI; XU, JIZHENG
To: BYTEDANCE INC.
Reel/Frame 058917/0648 →
Continuity (2)
Provisional Application 63106603 · Oct 28, 2020
Related Publication 20220132103A1 · Apr 28, 2022