IP Library › Granted Patent US 12,075,061
Granted Patent B2
US 12,075,061 · App. 17/955,740 · Granted Aug 27, 2024

Frame selection for streaming applications

Inventors: Aurobinda Maharana (Chinchwad, IN); Vignesh Ungrapalli (Udupi, IN); Ming-Yu Liu (San Jose, CA)
Assignee: Nvidia Corporation
H04N19/137H04N19/186H04N19/513
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,075,061
App. No.
17/955,740
Filed
Sep 29, 2022
Granted
Aug 27, 2024
Kind
B2
Art Unit
2482
USPC
375/240.16
Abstract

Systems and methods herein address reference frame selection in video streaming applications using one or more processing units to identify a frame of a sequence of frames as a blurred frame based at least in part on a first variance of motion (VoM) of the frame being less than or equal to an adaptive threshold that is based in part on a moving average of variance of motion (MAoV) determined using one or more reference frames.

Claims (51)

1. A system comprising:

one or more processing units to:

determine that a first variance of motion (VoM) corresponding to a frame of a video stream is above a VoM threshold; and

responsive to computation of a moving average of variance of motion (MAoV) using the first VoM and at least one initial VoM of a predefined blurred frame, identify the frame as a new blurred frame based in part on the first VoM of the frame being equal to or less than an adaptive threshold determined based at least in part on the MAoV.

2. The system of claim 1 , wherein the one or more processing units are further to compute the initial VoM and an initial MAoV based at least in part on content of the video stream.

3. The system of claim 1 , wherein the one or more processing units are further to:

maintain the adaptive threshold when a motion level for individual frames of the video stream is equal to or greater than a motion threshold or when a second VoM corresponding to one of the individual frames is equal to or less than the VoM threshold.

4. The system of claim 3 , wherein the one or more processing units are further to:

identify the one of the individual frames as a second new blurred frame in the video stream.

5. The system of claim 3 , wherein the one or more processing units are further to:

identify the second individual frame as a non-blurred frame in the video stream when the second VoM is above the adaptive threshold.

6. The system of claim 1 , wherein the first VoM corresponds to a Variance of Laplacian.

7. The system of claim 1 , wherein the one or more processing units are further to:

determine the at least one initial MAoV based in part on one or more historical frames, the one or more historical frames including the predefined blurred frame; and

average the at least one initial MAoV with the first VoM to determine the MAoV.

8. The system of claim 7 , wherein the one or more processing units are further to:

determine the one or more historical frames from a reference video stream having at least one similar aspect to the video stream, the at least one similar aspect including at least one of a similar feature, similar object, or similar background to the video stream.

9. The system of claim 1 , wherein the one or more processing units are further to determine the predefined blurred frame using an arbitrary value applied to the first VoM of the frame of the video stream.

10. The system of claim 1 , wherein the one or more processing units are further to determine luma and color difference (YUV) values corresponding to a plurality of frames including the frame of the video stream, wherein the first VoM is determined from the YUV values of the frame.

11. A method comprising:

determining that a first variance of motion (VoM) corresponding to a frame of a video stream is above a VoM threshold; and

responsive to computing a moving average of variance of motion (MAoV) using the first VoM and at least one initial VoM of a predefined blurred frame, identifying the frame as a new blurred frame based in part on the first VoM of the frame being equal to or less than an adaptive threshold determined based at least in part on the MAoV.

12. The method of claim 11 , further comprising:

computing the initial VoM and an initial MAoV based at least in part on content of the video stream.

13. The method of claim 11 , further comprising:

maintaining the adaptive threshold when a motion level for individual frames of the video stream is equal to or greater than a motion threshold or when a second VoM corresponding to one of the individual frames is equal to or less than the VoM threshold.

14. The method of claim 13 , further comprising:

identifying the one of the individual frames as a second new blurred frame in the video stream.

15. The method of claim 13 , further comprising:

identifying the second individual frame as a non-blurred frame in the video stream when the second VoM is above the adaptive threshold.

16. The method of claim 11 , wherein the first VoM corresponds to a Variance of Laplacian.

17. The method of claim 11 , further comprising:

determining the at least one initial MAoV based in part on one or more historical frames, the one or more historical frames including the predefined blurred frame; and

averaging the at least one initial MAoV with the first VoM to determine the MAoV.

18. The method of claim 17 , further comprising:

determining the one or more historical frames from a reference video stream having at least one similar aspect to the video stream, the at least one similar aspect including at least one of a similar feature, similar object, or similar background to the video stream.

19. A system comprising:

one or more processing units to identify a frame of a sequence of frames as a blurred frame based at least in part on a first variance of motion (VoM) of the frame being less than or equal to an adaptive threshold that is based in part on a moving average of variance of motion (MAoV) determined using one or more reference frames.

20. The system of claim 19 , wherein the system is comprised in at least one of:

a system for performing simulation operations;

a system for performing digital twin operations;

a system for performing light transport simulation;

a system for performing collaborative content creation for 3D assets;

a system for performing deep learning operations;

a system implemented using an edge device;

a system implemented using a robot;

a system for performing conversational AI operations;

a system for generating synthetic data;

a system incorporating one or more virtual machines (VMs);

a system implemented at least partially in a data center; or

a system implemented at least partially using cloud computing resources.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2022
From: MAHARANA, AUROBINDA; UNGRAPALLI, VIGNESH; LIU, MING-YU
To: NVIDIA CORPORATION
Reel/Frame 061256/0460 →
Continuity (1)
Related Publication 20240114144A1 · Apr 4, 2024
Cited By (1)
US 12,341,829