Granted Patent B2

US 12,556,720 · App. 18/033,697 · Granted Feb 17, 2026

Inventors: Fabien Racape (San Francisco, CA); Lahiru Dulanjana Hewa Gamage (Davis, CA); Jean Begaint (Menlo Park, CA); Simon Feltman (Sunnyvale, CA)

Assignee: INTERDIGITAL VC HOLDINGS, INC.

H04N19/186H04N19/176H04N19/60

Abstract

Claims (39)

1 . A method, comprising:

coding a tensor comprising a video portion using a learned transform comprising a first set of constraints to generate a latent map, wherein metadata comprising trained parameters is included in said latent map;

quantizing and entropy coding the latent map;

entropy decoding the latent map;

processing the entropy decoded latent map under a second set of task-specific constraints, distinct from compression-related residual constraints, the processing comprising applying a convolutional neural network trained to perform semantic or perceptual analysis of video content; and

performing a task using the trained parameters with the processed latent map, the task comprising semantic or perceptual video processing distinct from bitstream reconstruction or residual prediction, including at least one of anomaly detection, video enhancement, object recognition, or augmented reality overlay.

2 . The method of claim 1 , wherein the second set of constraints comprises semantic segmentation constraints trained using pixel-level semantic labels of the video portion.

3 . The method of claim 1 , wherein the task comprises video super-resolution or perceptual enhancement, distinct from residual coding or entropy reconstruction.

4 . The method of claim 3 , wherein said processing further comprises weighting of samples.

5 . The method of claim 1 , wherein said processing comprises a temporal convolution across multiple consecutive frames in addition to spatial convolution across pixel locations.

6 . The method of claim 1 , wherein said processing comprises generating an augmented reality overlay or annotation of detected objects in the video portion.

7 . The method of claim 1 , wherein the second set of constraints is dynamically adapted based on feedback from the semantic or perceptual task output.

8 . A non-transitory computer readable medium containing data content generated according to the method of claim 1 , for playback using a processor.

9 . An apparatus, comprising:

a processor, configured to:

code a tensor comprising a video portion using a learned transform comprising a first set of constraints to generate a latent map, wherein metadata comprising trained parameters is included in said latent map;

quantizing and entropy coding the latent map;

entropy decoding the latent map;

process the entropy decoded latent map under a second set of task-specific constraints, distinct from compression-related residual constraints, the processing comprising applying a convolutional neural network trained to perform semantic or perceptual analysis; and

perform a task using the trained parameters with the processed latent map, the task comprising semantic or perceptual video processing distinct from bitstream reconstruction or residual prediction.

10 . The apparatus of claim 9 , further comprising a hardware accelerator configured to execute the convolutional neural network under the second set of semantic constraints.

11 . The apparatus of claim 9 , wherein the processor is further configured to store in memory distinct parameter sets for compression constraints and for semantic constraints, and selectively apply said sets to the latent map.

12 . The apparatus of claim 9 , wherein the processor is further configured to adapt the latent map for anomaly detection in video surveillance.

13 . The apparatus of claim 9 , wherein the processor is further configured to output a semantically enhanced video stream distinct from a decoded reconstruction of the original video stream.

14 . A method, comprising:

decoding a video bitstream comprising a tensor comprising a video portion using a learned transform comprising a first set of constraints, wherein metadata comprising trained parameters is included in said tensor; and

processing the decoded video bitstream with a two-dimensional convolution on each color component at each pixel location under a second set of semantic constraints, distinct from compression residual coding, the processing adapting the decoded video bitstream for a subsequent semantic or perceptual task including anomaly detection, super-resolution, video quality enhancement, or object classification.

15 . A non-transitory computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 14 .

16 . The method of claim 14 , wherein said processing operates on a window of samples centered at each sample location.

17 . The method of claim 14 , wherein said processing further comprises object recognition using convolutional features extracted under the second set of constraints.

18 . An apparatus, comprising:

a processor, configured to:

decode a video bitstream comprising a tensor comprising a video portion using a learned transform comprising a first set of constraints, wherein metadata comprising trained parameters is included in said tensor; and

process the decoded video bitstream with a two-dimensional convolution on each color component at each pixel location under a second set of semantic constraints distinct from compression residual coding, the processing adapting the decoded video bitstream for a subsequent semantic or perceptual task distinct from reconstruction or prediction coding.

19 . A device comprising:

an apparatus according to claim 18 ; and

at least one of (i) an antenna configured to receive a signal, the signal including the video block, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the video block, and (iii) a display configured to display an output representative of a video block.

20 . The apparatus of claim 18 , wherein said processing comprises convolution on each color component.

21 . The apparatus of claim 18 , wherein said processing comprises classification of video content for subsequent recommendation or retrieval tasks.

Assignments (2)

Continuity (2)

References Cited (10)