IP Library › Granted Patent US 12,262,060
Granted Patent B2
US 12,262,060 · App. 18/113,345 · Granted Mar 25, 2025

Systems and methods for signaling neural network post-filter patch size information in video coding

Inventors: Sachin G. Deshpande (Camas, WA); Ahmed Cheikh Sidiya (Morgantown, WV)
Assignee: Sharp Kabushiki Kaisha
H04N19/80H04N19/70H04N19/85
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,262,060
App. No.
18/113,345
Granted
Mar 25, 2025
Kind
B2
Abstract

A device may be configured to perform filtering based on information included in a neural network post-filter characteristics message. In one example, the neural network post-filter characteristics message includes a syntax element indicating whether the post-processing filter accepts as input a patch size having a width equal to horizontal sample counts indicated by a syntax element and a height equal to vertical sample counts indicated by a syntax element, or a patch size having a width equal to a multiple of horizontal sample counts indicated by a syntax element and a height equal to a multiple of vertical sample counts indicated by a syntax element.

Claims (13)

1. A method of performing neural network filtering for video data, the method comprising:

receiving a neural network post-filter characteristics message;

parsing a first syntax element from the neural network post-filter characteristics message indicating whether (i) a second syntax element and a third syntax element are present in the neural network post-filter characteristics message and a post-processing filter accepts a first patch size having a width equal to horizontal sample counts indicated by the second syntax element and a height equal to vertical sample counts indicated by the third syntax element, or (ii) a fourth syntax element and a fifth syntax element are present in the neural network post-filter characteristics message and the post-processing filter accepts a second patch size having a width equal to a multiple of horizontal sample counts indicated by the fourth syntax element and a height equal to a multiple of vertical sample counts indicated by the fifth syntax element; and

conditionally parsing the second syntax element and the third syntax element or the fourth syntax element and the fifth syntax element based on a value of the first syntax element.

2. A device comprising:

one or more processors configured to:

receive a neural network post-filter characteristics message;

parse a first syntax element from the neural network post-filter characteristics message indicating whether (i) a second syntax element and a third syntax element are present in the neural network post-filter characteristics message and a post-processing filter accepts a first patch size having a width equal to horizontal sample counts indicated by a the second syntax element and a height equal to vertical sample counts indicated by a the third syntax element, or (ii) a fourth syntax element and a fifth syntax element are present in the neural network post-filter characteristics message and the post-processing filter accepts a second patch size having a width equal to a multiple of horizontal sample counts indicated by the fourth syntax element and a height equal to a multiple of vertical sample counts indicated by the fifth syntax element; and

conditionally parse the second syntax element and the third syntax element or the fourth syntax element and the fifth syntax element based on a value of the first syntax element.

3. The device of claim 2 , wherein the device includes a video decoder.

4. A device comprising;

one or more processors configured to: signal a neural network post-filter characteristics message; signal a first syntax element in the neural network post-filter characteristics message indicating whether (i) a second syntax element and a third syntax element are present in the neural network post-filter characteristics message and a post-processing filter accepts a first patch size having a width equal to horizontal sample counts indicated by the second syntax element and a height equal to vertical sample counts indicated by the third syntax element, or (ii) a fourth syntax element and a fifth syntax element are present in the neural network post-filter characteristics message and the post-processing filter accepts a second patch size having a width equal to a multiple of horizontal sample counts indicated by the fourth syntax element and a height equal to a multiple of vertical sample counts indicated by the fifth syntax element; and conditionally signal the second syntax element and the third syntax element or the fourth syntax element and the fifth syntax element based on a value of the first syntax element.

5. The device of claim 4 , wherein the device includes a video encoder.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 23, 2023
From: DESHPANDE, S G.; CHEIKH SIDIYA, AHMED
To: SHARP KABUSHIKI KAISHA
Reel/Frame 062785/0024 →
Continuity (2)
Provisional Application 63438779 · Jan 12, 2023
Related Publication 20240244268A1 · Jul 18, 2024
References Cited (17)
US 20240221231A1 · Deshpande · 2024 [cited by examiner]
US 20240223764A1 · Wang · 2024 [cited by examiner]
European Opinion. (Year: 2023). [cited by examiner]
JVET-AA2006-v1 “Additional SEI messages for VSEI (Draft 2)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 27th Meeting, by teleconference, Jul. 13-22, 2022. [cited by applicant]
JVET-Z0082-v2 “AHG11: Content-adaptive neural network post-filter” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference, Apr. 20-29, 2022. [cited by applicant]
“Test Model of Incremental Compression of Neural Networks for Multimedia Content Description and Analysis (INCTM),” ISO/IEC JTC 1/SC 29/WG 04, N0179. Feb. 4, 2022. [cited by applicant]
JVET-Z0052-v1 “AHG9: NNR post-filter SEI message” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 26th Meeting, by teleconference, Apr. 20-29, 2022. [cited by applicant]
JVET-Y2006 “Additional SEI messages for VSEI (Draft 6)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 25th Meeting, by teleconference, Jan. 12-21, 2022. [cited by applicant]
Rec. ITU-T H.274 “Versatile supplemental enhancement information messages for coded video bitstreams” (Aug. 2020). [cited by applicant]
Rec. ITU-T H.273 “Coding-independent code point for video signal type identification” (Jul. 2021). [cited by applicant]
JVET-G1001 “Algorithm Description of Joint Exploration Test Model 7 (JEM 7)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 7th Meeting: Torino, IT, Jul. 13-21, 2017. [cited by applicant]
JVET-J1001-v2 “Verastile Video Coding (Draft 1)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, 10th Meeting: San Diego, US, Apr. 10-20, 2018. [cited by applicant]
JVET-T2001-v2 “Verastile Video Coding Editorial Refinements on Draft 10)” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, 20th Meeting, by teleconference, Oct. 7-16, 2020. [cited by applicant]
ITU-T H.265 “High efficiency video coding” (Dec. 2016). [cited by applicant]
ITU-T H.264 “Advanced video coding for generic audiovisual services” (Oct. 2016). [cited by applicant]
ISO/IEC FDIS 15938-17. “Information technology—Multimedia content description interface—Part 17: Compression of neural networks for multimedia content description and analysis” ISO/IEC 15938-17:2020(E). 2021. [cited by applicant]
“Information technology—MPEG video technologies—Part 7: Versatile supplemental enhancement information messages for coded video bitstreams, Amendment 1: Additional SEI messages” 28th Meeting of ISO/IEC JTC1/ SC29/WG5 9,… [cited by applicant]