IP Library › Granted Patent US 12,328,428
Granted Patent B2
US 12,328,428 · App. 18/069,076 · Granted Jun 10, 2025

Upscale filtering of video data

Inventors: Vikrant Mahajan (Dinanagar, IN); Sandeep Nellikatte Srivatsa (Bangalore, IN); Ashish Mishra (Bhubaneswar, IN); Apoorva Nagarajan (Bangalore, IN); Divya Goswami (Delhi, IN)
Assignee: QUALCOMM INCORPORATED
H04N19/117H04N19/176H04N19/30
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,328,428
App. No.
18/069,076
Granted
Jun 10, 2025
Kind
B2
Abstract

A device for upscale filtering video data includes one or more processors configured to generate a number of upscaled pixels that is an integer multiple of one half of a size of a filter used to upscale filter the video data. The device may then store the remaining pixels to be used to upscale filter a subsequent, right-neighboring block. In this manner, the device may avoid generating upscaled pixels that will result in being unaligned with the block and that may not be output due to being unaligned.

Claims (115)

1. A method of upscale filtering video data, the method comprising:

determining a filter scale value representing one half of a number of support pixels to be used by an upscale filter to upscale filter a current block of video data;

determining an upscale ratio for the current block of video data;

calculating a size multiple value as a product of the upscale ratio and a difference between a size of the current block of video data and the filter scale value;

calculating a rounded value from the size multiple value such that the rounded value is a largest integer multiple of the filter scale value that does not exceed the size multiple value;

generating a number of upscaled pixels for each row of the current block of video data that is equal to the rounded value, without generating upscaled pixels for a number of pixels from a right edge of the current block equal to a difference between a product of the size of the current block and the upscale ratio and the rounded value;

storing values for the number of pixels from the right edge of the current block of video data for each row of the current block of video data; and

using the stored values for the number of pixels from the right edge of the current block to upscale filter pixels of a right-neighboring block of video data to the current block of video data.

2. The method of claim 1 , further comprising:

storing unprocessed values for pixels of the current block to the left of the number of pixels from the right edge of the current block; and

upscale filtering the number of pixels from the right edge of the current block using the stored unprocessed values for the pixels of the current block.

3. The method of claim 1 , wherein the filter scale value is a value of 4.

4. The method of claim 1 , wherein the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

5. The method of claim 1 , wherein the size of the current block of video data is one of 64 pixels or 128 pixels.

6. The method of claim 1 , wherein the rounded value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

7. The method of claim 1 , wherein the number of pixels from the right edge of the current block is one less than the filter scale value.

8. The method of claim 1 , wherein:

the number of support pixels is 8,

the filter scale value is 4,

the size of the current block of video data is 64,

the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2,

when the upscale ratio is 1.125, the rounded value is 64,

when the upscale ratio is 1.25, the rounded value is 72,

when the upscale ratio is 1.375, the rounded value is 80,

when the upscale ratio is 1.5, the rounded value is 88,

when the upscale ratio is 1.625, the rounded value is 96,

when the upscale ratio is 1.75, the rounded value is 104,

when the upscale ratio is 1.875, the rounded value is 112, or

when the upscale ratio is 2, the rounded value is 120, and

wherein the number of pixels from the right edge of the current block is 3.

9. The method of claim 1 , further comprising decoding an encoded block to form the current block of video data.

10. The method of claim 9 , further comprising decoding an encoded version of the right-neighboring block after generating the number of upscaled pixels for each row of the current block.

11. The method of claim 1 , further comprising:

encoding a raw block to form an encoded block; and

decoding the encoded block to form the current block of video data.

12. A device for upscale filtering video data, the device comprising:

a memory configured to store video data; and

one or more processors implemented in circuitry and configured to:

determine a filter scale value representing one half of a number of support pixels to be used by an upscale filter to upscale filter a current block of video data;

determine an upscale ratio for the current block of video data;

calculate a size multiple value as a product of the upscale ratio and a difference between the size of the current block of video data and the filter scale value;

calculate a rounded value from the size multiple value such that the rounded value is a largest integer multiple of the filter scale value that does not exceed the size multiple value;

generate a number of upscaled pixels for each row of the current block of video data that is equal to the rounded value, without generating upscaled pixels for a number of pixels from a right edge of the current block equal to a difference between a product of the size of the current block and the upscale ratio and the rounded value;

store values for the number of pixels from the right edge of the current block of video data for each row of the current block of video data; and

use the stored values for the number of pixels from the right edge of the current block to upscale filter pixels of a right-neighboring block of video data to the current block of video data.

13. The device of claim 12 , wherein the one or more processors are further configured to:

store unprocessed values for pixels of the current block to the left of the number of pixels from the right edge of the current block; and

upscale filter the number of pixels from the right edge of the current block using the stored unprocessed values for the pixels of the current block.

14. The device of claim 12 , wherein the filter scale value is a value of 4.

15. The device of claim 12 , wherein the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

16. The device of claim 12 , wherein the size of the current block of video data is one of 64 pixels or 128 pixels.

17. The device of claim 12 , wherein the rounded value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

18. The device of claim 12 , wherein the number of pixels from the right edge of the current block is one less than the filter scale value.

19. The device of claim 12 , wherein:

the number of support pixels is 8,

the filter scale value is 4,

the size of the current block of video data is 64,

the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2,

when the upscale ratio is 1.125, the rounded value is 64,

when the upscale ratio is 1.25, the rounded value is 72,

when the upscale ratio is 1.375, the rounded value is 80,

when the upscale ratio is 1.5, the rounded value is 88,

when the upscale ratio is 1.625, the rounded value is 96,

when the upscale ratio is 1.75, the rounded value is 104,

when the upscale ratio is 1.875, the rounded value is 112, or

when the upscale ratio is 2, the rounded value is 120, and

wherein the number of pixels from the right edge of the current block is 3.

20. The device of claim 12 , wherein the one or more processors are further configured to decode an encoded block to form the current block of video data.

21. The device of claim 20 , wherein the one or more processors are further configured to decode an encoded version of the right-neighboring block after generating the number of upscaled pixels for each row of the current block.

22. The device of claim 12 , wherein the one or more processors are further configured to:

encode a raw block to form an encoded block; and

decode the encoded block to form the current block of video data.

23. The device of claim 12 , wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

24. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:

determine a filter scale value representing one half of a number of support pixels to be used by an upscale filter to upscale filter a current block of video data;

determine an upscale ratio for the current block of video data;

calculate a size multiple value as a product of the upscale ratio and a difference between the size of the current block of video data and the filter scale value;

calculate a rounded value from the size multiple value such that the rounded value is a largest integer multiple of the filter scale value that does not exceed the size multiple value;

generate a number of upscaled pixels for each row of the current block of video data that is equal to the rounded value, without generating upscaled pixels for a number of pixels from a right edge of the current block equal to a difference between a product of the size of the current block and the upscale ratio and the rounded value;

store values for the number of pixels from the right edge of the current block of video data for each row of the current block of video data; and

use the stored values for the number of pixels from the right edge of the current block to upscale filter pixels of a right-neighboring block of video data to the current block of video data.

25. The non-transitory computer-readable storage medium of claim 24 , further comprising instructions that cause the processor to:

store unprocessed values for pixels of the current block to the left of the number of pixels from the right edge of the current block; and

upscale filter the number of pixels from the right edge of the current block using the stored unprocessed values for the pixels of the current block.

26. The non-transitory computer-readable storage medium of claim 24 , wherein the filter scale value is a value of 4.

27. The non-transitory computer-readable storage medium of claim 24 , wherein the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2.

28. The non-transitory computer-readable storage medium of claim 24 , wherein the size of the current block of video data is one of 64 pixels or 128 pixels.

29. The non-transitory computer-readable storage medium of claim 24 , wherein the rounded value is one of 64, 72, 80, 88, 96, 104, 112, or 120.

30. The non-transitory computer-readable storage medium of claim 24 , wherein the number of pixels from the right edge of the current block is one less than the filter scale value.

31. The non-transitory computer-readable storage medium of claim 24 , wherein:

the number of support pixels is 8,

the filter scale value is 4,

the size of the current block of video data is 64,

the upscale ratio is selected from a set of upscale ratios including 1.125, 1.25, 1.375, 1.5, 1.625, 1.75, 1.875, and 2,

when the upscale ratio is 1.125, the rounded value is 64,

when the upscale ratio is 1.25, the rounded value is 72,

when the upscale ratio is 1.375, the rounded value is 80,

when the upscale ratio is 1.5, the rounded value is 88,

when the upscale ratio is 1.625, the rounded value is 96,

when the upscale ratio is 1.75, the rounded value is 104,

when the upscale ratio is 1.875, the rounded value is 112, or

when the upscale ratio is 2, the rounded value is 120, and

wherein the number of pixels from the right edge of the current block is 3.

32. The non-transitory computer-readable storage medium of claim 24 , further comprising instructions that cause the processor to decode an encoded block to form the current block of video data.

33. The non-transitory computer-readable storage medium of claim 24 , wherein the one or more processors are further configured to:

encode a raw block to form an encoded block; and

decode the encoded block to form the current block of video data.

34. A device for upscale filtering video data, the device comprising:

means for determining a filter scale value representing one half of a number of support pixels to be used by an upscale filter to upscale filter a current block of video data;

means for determining an upscale ratio for the current block of video data;

means for calculating a size multiple value as a product of the upscale ratio and a difference between the size of the current block of video data and the filter scale value;

means for calculating a rounded value from the size multiple value such that the rounded value is a largest integer multiple of the filter scale value that does not exceed the size multiple value;

means for generating a number of upscaled pixels for each row of the current block of video data that is equal to the rounded value, without generating upscaled pixels for a number of pixels from a right edge of the current block equal to a difference between a product of the size of the current block and the upscale ratio and the rounded value;

means for storing values for the number of pixels from the right edge of the current block of video data for each row of the current block of video data; and

means for using the stored values for the number of pixels from the right edge of the current block to upscale filter pixels of a right-neighboring block of video data to the current block of video data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2023
From: MAHAJAN, VIKRANT; SRIVATSA, SANDEEP NELLIKATTE; MISHRA, ASHISH; NAGARAJAN, APOORVA; GOSWAMI, DIVYA
To: QUALCOMM INCORPORATED
Reel/Frame 062900/0987 →
Continuity (1)
Related Publication 20240214563A1 · Jun 27, 2024
References Cited (16)
US 10779276B2 · Ji et al. · 2020 [cited by applicant]
US 20100111023A1 · Pelletier et al. · 2010 [cited by applicant]
US 20130094568A1 · Hsu et al. · 2013 [cited by applicant]
US 20150156486A1 · Miyoshi · 2015 [cited by examiner]
US 20190394482A1 · Joshi · 2019 [cited by examiner]
US 20200145658A1 · Chang et al. · 2020 [cited by applicant]
US 20210297959A1 · Zhou et al. · 2021 [cited by applicant]
US 20210337523A1 · Shen et al. · 2021 [cited by applicant]
US 20220237741A1 · Yin et al. · 2022 [cited by applicant]
WO 2019237927A1 · 2019 [cited by applicant]
Correa M., et al., “AV1 and VVC Video Codecs: Overview on Complexity Reduction and Hardware Design”, IEEE Open Journal of Circuits and Systems, IEEE, vol. 2, Sep. 13, 2021, pp. 564-576, XP011878048, Section II.F. [cited by applicant]
El Mezeni D., et al., “Fast Guided Filter for Power-efficient Real-time 1080p Streaming Video Processing”, Journal of Real-time Image Processing, Springer, DE, vol. 17, No. 3, Jul. 4, 2018, pp. 511-525, XP037160569, Par… [cited by applicant]
International Search Report and Written Opinion—PCT/US2023/081588—ISA/EPO—Feb. 28, 2024. [cited by applicant]
Zummach E., et al., “Efficient Hardware Design for the AV1 CDEF Filter Targeting 4K UHD Videos”, 2019 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, Oct. 12, 2020, 5 Pages, XP033933390, The Whole Do… [cited by applicant]
ITU-T H.265: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, High Efficiency Video Coding, The International Telecommunication Union, Jun. 2019, 696 Pages. [cited by applicant]
ITU-T H.266: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, Versatile Video Coding, The International Telecommunication Union, Aug. 2020, 516 pages. [cited by applicant]