IP Library › Granted Patent US 11,729,349
Granted Patent B2
US 11,729,349 · App. 17/400,350 · Granted Aug 15, 2023

Method, electronic device, and computer program product for video processing

Inventors: Qiang Chen (Shanghai, CN); Pedro Fernandez Orellana (Shanghai, CN)
Assignee: EMC IP Holding Company LLC
H04N7/0125H04N21/238
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,729,349
App. No.
17/400,350
Granted
Aug 15, 2023
Kind
B2
Abstract

Illustrative embodiments include a method, an electronic device, and a computer program product for video processing. In the method, a first group of image frames in a first video having a first resolution is converted into a second group of image frames having a second resolution, the first resolution being higher than the second resolution; a second video having the second resolution is generated based on the second group of image frames; conversion parameters for the second video are determined based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and the conversion parameters and the second video are sent to a requester of the first video.

Claims (66)

1. A method for video processing, comprising:

converting a first group of image frames in a first video having a first resolution into a second group of image frames having a second resolution, the first resolution being higher than the second resolution;

generating a second video having the second resolution based on the second group of image frames;

determining conversion parameters for the second video based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and

sending the conversion parameters and the second video to a requester of the first video;

wherein the conversion parameters are determined responsive to the generation of the second video; and

wherein sending the conversion parameters and the second video to the requester of the first video comprises streaming the conversion parameters in combination with the second video to the requester.

2. The method according to claim 1 , wherein determining the conversion parameters comprises:

if a scene switch is detected in the first video, generating scene metadata, the scene metadata indicating a position of the scene switch in the first video; and

determining a first image frame subset of the first group of image frames and a second image frame subset of the second group of image frames based on the scene metadata, image frames in the first image frame subset and the second image frame subset having the same scene; and

determining the conversion parameters based on the image frames in the first image frame subset and the second image frame subset.

3. The method according to claim 2 , further comprising:

sending the scene metadata to the requester of the first video.

4. The method according to claim 1 , wherein determining the conversion parameters based on the first group of image frames and the second group of image frames comprises:

predicting, according to a conversion model, one image frame in the first group of image frames based on multiple image frames in the second group of image frames, the conversion model being used to convert an image having the second resolution into an image having the third resolution;

training the conversion model based on a difference between the predicted image frame and the one image frame in the first group of image frames; and

determining parameters of the trained conversion model as the conversion parameters.

5. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform the method of claim 1 .

6. A method for video processing, comprising:

receiving, from a provider of a first video having a first resolution, a second video having a second resolution converted from the first video and conversion parameters for the second video, the first resolution being higher than the second resolution, and the conversion parameters being determined based on a first group of image frames in the first video and a second group of image frames in the second video;

converting, using the conversion parameters, the second group of image frames into a third group of image frames having a third resolution, the third resolution being higher than the second resolution; and

generating a third video having the third resolution based on the third group of image frames to respond to a request for the first video;

wherein the conversion parameters are determined responsive to the generation of the second video; and

wherein the conversion parameters in combination with the second video are streamed from the provider of the first video.

7. The method according to claim 6 , wherein converting the second group of image frames into the third group of image frames comprises:

receiving scene metadata from the provider, the scene metadata indicating a position of a scene switch in the first video;

determining a second image frame subset having the same scene in the second group of image frames based on the scene metadata; and

converting, using the conversion parameters, the second image frame subset into a third image frame subset having the third resolution, as part of the third group of image frames.

8. The method according to claim 6 , wherein converting, using the conversion parameters, the second group of image frames into the third group of image frames comprises:

applying the conversion parameters to a conversion model, the conversion model being used to convert an image having the second resolution into an image having the third resolution; and

generating, according to the conversion model, one image frame in the third group of image frames based on multiple image frames in the second group of image frames.

9. A computer program product tangibly stored on a computer-readable medium and comprising machine-executable instructions, the machine-executable instructions, when executed, causing a machine to perform the method of claim 6 .

10. An electronic device, comprising:

a processor; and

a memory coupled to the processor, the memory having instructions stored therein that, when executed by the processor, cause the electronic device to execute actions comprising:

converting a first group of image frames in a first video having a first resolution into a second group of image frames having a second resolution, the first resolution being higher than the second resolution;

generating a second video having the second resolution based on the second group of image frames;

determining conversion parameters for the second video based on the first group of image frames and the second group of image frames, the conversion parameters being used to convert an image frame in the second group of image frames into an image frame having a third resolution, and the third resolution being higher than the second resolution; and

sending the conversion parameters and the second video to a requester of the first video;

wherein the conversion parameters are determined responsive to the generation of the second video; and

wherein sending the conversion parameters and the second video to the requester of the first video comprises streaming the conversion parameters in combination with the second video to the requester.

11. The electronic device according to claim 10 , wherein determining the conversion parameters comprises:

if a scene switch is detected in the first video, generating scene metadata, the scene metadata indicating a position of the scene switch in the first video; and

determining a first image frame subset of the first group of image frames and a second image frame subset of the second group of image frames based on the scene metadata, image frames in the first image frame subset and the second image frame subset having the same scene; and

determining the conversion parameters based on the image frames in the first image frame subset and the second image frame subset.

12. The electronic device according to claim 11 , wherein the actions further comprise:

sending the scene metadata to the requester of the first video.

13. The electronic device according to claim 10 , wherein determining the conversion parameters based on the first group of image frames and the second group of image frames comprises:

predicting, according to a conversion model, one image frame in the first group of image frames based on multiple image frames in the second group of image frames, the conversion model being used to convert an image having the second resolution into an image having the third resolution;

training the conversion model based on a difference between the predicted image frame and the one image frame in the first group of image frames; and

determining parameters of the trained conversion model as the conversion parameters.

14. An electronic device, comprising:

a processor; and

a memory coupled to the processor, the memory having instructions stored therein that, when executed by the processor, cause the electronic device to execute actions comprising:

receiving, from a provider of a first video having a first resolution, a second video having a second resolution converted from the first video and conversion parameters for the second video, the first resolution being higher than the second resolution, and the conversion parameters being determined based on a first group of image frames in the first video and a second group of image frames in the second video;

converting, using the conversion parameters, the second group of image frames into a third group of image frames having a third resolution, the third resolution being higher than the second resolution; and

generating a third video having the third resolution based on the third group of image frames to respond to a request for the first video;

wherein the conversion parameters are determined responsive to the generation of the second video; and

wherein the conversion parameters in combination with the second video are streamed from the provider of the first video.

15. The electronic device according to claim 14 , wherein converting the second group of image frames into the third group of image frames comprises:

receiving scene metadata from the provider, the scene metadata indicating a position of a scene switch in the first video;

determining a second image frame subset having the same scene in the second group of image frames based on the scene metadata; and

converting, using the conversion parameters, the second image frame subset into a third image frame subset having the third resolution as part of the third group of image frames.

16. The electronic device according to claim 14 , wherein converting, using the conversion parameters, the second group of image frames into the third group of image frames comprises:

applying the conversion parameters to a conversion model, the conversion model being used to convert an image having the second resolution into an image having the third resolution; and

generating, according to the conversion model, one image frame in the third group of image frames based on multiple image frames in the second group of image frames.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 12, 2021
From: CHEN, QIANG; FERNANDEZ ORELLANA, PEDRO
To: EMC IP HOLDING COMPANY LLC
Reel/Frame 057160/0150 →
Priority Claims (1)
CN 202110837055.8 · Jul 23, 2021 · national
Continuity (1)
Related Publication 20230027981A1 · Jan 26, 2023