OPTIMIZATION OF INTERFRAME PREDICTION ALGORITHMS BASED ON HETEROGENEOUS COMPUTING
In at least one embodiment, a motion estimation method may include dividing a first video frame to be estimated into a plurality of macroblocks, in which each of the macroblocks includes a plurality of sub-blocks. The method may further include determining a sampling pattern for each sub-block based on visual data of the sub-block, and determining a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block. The method may further include determining a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock, and determining a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
1 . A motion estimation method, comprising:
dividing a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks;
determining a sampling pattern for each sub-block based on visual data of the sub-block;
determining a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block;
determining a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock; and
determining a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
2 . The method of claim 1 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
3 . The method of claim 1 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
4 . The method of claim 3 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
5 . The method of claim 1 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
6 . The method of claim 1 , wherein the method utilizes the H.264 coding standard.
7 . The method of claim 1 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).
8 . The method of claim 1 , wherein the method utilizes the CUDA platform by Nvidia.
9 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:
dividing a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks;
determining a sampling pattern for each sub-block based on visual data of the sub-block;
determining a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block;
determining a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock; and
determining a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
10 . The non-transitory computer-readable medium of claim 9 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
11 . The non-transitory computer-readable medium of claim 9 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
12 . The non-transitory computer-readable medium of claim 11 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
13 . The non-transitory computer-readable medium of claim 9 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
14 . The non-transitory computer-readable medium of claim 9 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).
15 . An apparatus, comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
divide a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks,
determine a sampling pattern for each sub-block based on visual data of the sub-block,
determine a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block,
determine a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock, and
determine a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
16 . The apparatus of claim 15 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
17 . The apparatus of claim 15 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
18 . The apparatus of claim 17 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
19 . The apparatus of claim 15 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
20 . The apparatus of claim 15 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).
We claim:
1 . A motion estimation method, comprising:
dividing a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks;
determining a sampling pattern for each sub-block based on visual data of the sub-block;
determining a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block;
determining a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock; and
determining a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
2 . The method of claim 1 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
3 . The method of claim 1 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
4 . The method of claim 3 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
5 . The method of claim 1 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
6 . The method of claim 1 , wherein the method utilizes the H.264 coding standard.
7 . The method of claim 1 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).
8 . The method of claim 1 , wherein the method utilizes the CUDA platform by Nvidia.
9 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:
dividing a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks;
determining a sampling pattern for each sub-block based on visual data of the sub-block;
determining a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block;
determining a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock; and
determining a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
10 . The non-transitory computer-readable medium of claim 9 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
11 . The non-transitory computer-readable medium of claim 9 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
12 . The non-transitory computer-readable medium of claim 11 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
13 . The non-transitory computer-readable medium of claim 9 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
14 . The non-transitory computer-readable medium of claim 9 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).
15 . An apparatus, comprising:
a processor; and
a memory storing instructions that, when executed by the processor, configure the apparatus to:
divide a first video frame to be estimated into a plurality of macroblocks, wherein each of the macroblocks includes a plurality of sub-blocks,
determine a sampling pattern for each sub-block based on visual data of the sub-block,
determine a prediction motion vector for each sub-block by performing a pre-search based on the sampling pattern of the sub-block,
determine a search template for each macroblock based on the prediction motion vector of each sub-block within the macroblock, and
determine a prediction motion vector for each macroblock by performing a secondary search based on the search template of the macroblock.
16 . The apparatus of claim 15 , wherein each macroblock comprises 16×16 samples, and each sub-block comprises 4×4 samples.
17 . The apparatus of claim 15 , wherein a density of the sampling pattern for each sub-block is based on a visual complexity of the sub-block.
18 . The apparatus of claim 17 , wherein the sampling pattern for each sub-block comprises at least four sampling points, but no more than approximately half the total number of sampling points in the sub-block.
19 . The apparatus of claim 15 , wherein the search template for each macroblock comprises a direction, a step length, and a search domain size.
20 . The apparatus of claim 15 , wherein the pre-search and the secondary search are each performed by a graphics processing unit (GPU).