Granted Patent B2

US 12,634,433 · App. 18/250,949 · Granted May 19, 2026

Inventors: Feng Liu (Beaverton, OR); Wu-Chi Feng (Tigard, OR)

Assignee: PORTLAND STATE UNIVERSITY

H04N19/105H04N19/154H04N19/172H04N19/85

Abstract

Claims (33)

1 . A system, comprising:

a hybrid encoding system configured to selectively encode a portion of a video; and

a decoding system communicatively coupled to the hybrid encoding system and configured to, synthesize, with a deep machine learning model, a remainder of the video upon receiving the selectively encoded portion of the video from the hybrid encoding system, and selectively combine the synthesized remainder of the video with the portion of the video,

wherein the hybrid encoding system comprises a first processor and a first non-transitory memory, the first non-transitory memory configured with executable instructions that when executed by the first processor cause the first processor to:

compress two anchor frames in the video, wherein the two anchor frames include MPEG-encoded data from ground truth frames; and

generate, by inputting the two anchor frames into a second deep machine learning model identical to the deep machine learning model, at least one intermediate frame between the two anchor frames, wherein the at least one intermediate frame includes receiver-generated data.

2 . The system of claim 1 , wherein the first non-transitory memory is further configured with executable instructions that when executed by the first processor cause the first processor to:

compress a subset of the at least one intermediate frame; and

combine the two compressed anchor frames and the compressed subset of the at least one intermediate frame into a compressed video, the compressed video comprising the selectively encoded portion of the video.

3 . A system, comprising:

a hybrid encoding system configured to selectively encode a portion of a video; and

a decoding system communicatively coupled to the hybrid encoding system and configured to, synthesize, with a deep machine learning model, a remainder of the video upon receiving the selectively encoded portion of the video from the hybrid encoding system, and selectively combine the synthesized remainder of the video with the portion of the video;

wherein the hybrid encoding system comprises a first processor and a first non-transitory memory, the first non-transitory memory configured with executable instructions that when executed by the first processor cause the first processor to:

compress two anchor frames in the video;

generate, by inputting the two anchor frames into a second deep machine learning model identical to the deep machine learning model, at least one intermediate frame between the two anchor frames;

compress a subset of the at least one intermediate frame;

select the subset of the at least one intermediate frame based on an image quality metric of at least one macroblock of the at least one intermediate image frame below an image quality metric threshold; and

combine the two compressed anchor frames and the compressed subset of the at least one intermediate frame into a compressed video, the compressed video comprising the selectively encoded portion of the video.

4 . The system of claim 3 , wherein the image quality metric comprises a peak signal-to-noise ratio (PSNR) or a video multi-method assessment fusion (VMAF).

5 . The system of claim 3 , wherein the decoding system comprises a second processor and a second non-transitory memory, the second non-transitory memory configured with executable instructions that when executed by the second processor cause the second processor to:

decompress the at least two compressed anchor frames to obtain at least two decompressed anchor frames;

decompress the compressed subset of the at least one intermediate frame to obtain a decompressed subset of the at least one intermediate frame;

generate, by inputting the at least two decompressed anchor frames into the deep machine learning model, a synthesized image frame corresponding to the at least one intermediate frame between the at least two anchor frames;

reconstruct a hybrid image frame by combining the decompressed subset of the at least one intermediate frame with the synthesized image frame; and

output, to a display device, a video comprising the at least two decompressed anchor frames and the hybrid image frame.

6 . The system of claim 3 , wherein an anchor-frame distance between the two anchor frames is greater than two, and wherein the at least one intermediate frame between the two anchor frames comprises at least two intermediate frames between the two anchor frames.

7 . The system of claim 3 , wherein the deep machine learning model comprises a video frame synthesis neural network.

8 . The system of claim 1 , wherein the hybrid encoding system performs the MPEG-based encoding of the two anchor frames by using H.264/MPEG-4 or high efficiency video coding (HEVC).

9 . The system of claim 3 , wherein the first non-transitory memory is further configured with executable instructions that when executed by the first processor cause the first processor to transmit the compressed video to a decoding system.

10 . The system of claim 9 , wherein the decoding system is configured to:

decompress the two compressed anchor frames into two decompressed anchor frames and the compressed subset into a decompressed subset;

generate, by inputting the two decompressed anchor frames of the compressed video into a second deep machine learning model, at least one intermediate frame between the two decompressed anchor frames; and

combine the at least one intermediate frame between the two decompressed anchor frames with the decompressed subset of the at least one intermediate frame to generate a hybrid intermediate frame.

Assignments (1)

Continuity (2)

References Cited (21)