Pre-processing image frames based on camera statistics
The present disclosure relates to systems, methods, and computer-readable media for selectively identifying image frames from an input video to provide to an image processing model based on camera statistics. For example, systems disclosed herein include receiving an input video and associated camera statistics from a video capturing device. The systems disclosed herein further include identifying select image frames to provide to the image processing model based on the camera statistics and based on an application of the image processing model. The systems disclosed herein further include selectively identifying and providing camera statistics to the image processing model. By selectively providing data to the image processing model based on camera statistics, the systems disclosed herein can leverage capabilities of video capturing devices to significantly reduce the expense of processing resources when utilizing a variety of image processing models.
1 . A method, comprising:
receiving, from one or more video capturing devices, input video content and a set of camera statistics, wherein the set of camera statistics includes data obtained by the one or more video capturing devices in conjunction with generating the input video content;
identifying an application for an image processing model to perform with respect to the input video content;
based on the identified application for the image processing model:
pre-processing the input video content using the set of camera statistics to generate transformed video content; and
identifying a subset of camera statistics from the set of camera statistics that are relevant to the identified application for the image processing model; and
providing a first input and a second input to the image processing model, the first input including the transformed video content and the second input including the subset of camera statistics, wherein the image processing model comprises a deep learning model trained to generate an output based on video content input and associated camera statistics inputs.
2 . The method of claim 1 , wherein the deep learning model is trained based on training data including both training video content and associated camera statistics.
3 . The method of claim 1 , wherein pre-processing the input video content includes one or more of refining image frames, modifying color or brightness, or down-sampling resolution.
4 . The method of claim 1 , wherein pre-processing the input video content includes identifying a subset of the input video content based on content of interest included in the subset of the input video content.
5 . The method of claim 4 , wherein identifying the subset of the input video content includes identifying cropped portions of image frames of the input video content, and wherein providing the first input and the second input to the image processing model comprises providing the cropped portions of the image frames and the subset of camera statistics as inputs to the deep learning model.
6 . The method of claim 4 , wherein identifying the subset of the input video content includes identifying a subset of image frames from a plurality of image frames within which the content of interest appears, and wherein providing the first input and the second input to the image processing model comprises providing the subset of image frames and the subset of camera statistics as inputs to the deep learning model.
7 . The method of claim 4 , wherein identifying the subset of the input video content is based on the subset of camera statistics indicating the content of interest included in the subset of the input video content.
8 . The method of claim 1 , wherein the input video content comprises captured video footage that has been locally refined by the one or more video capturing devices based on the set of camera statistics.
9 . The method of claim 1 , wherein the deep learning model is implemented on one or more of a cloud computing system or a computing device that receives the input video content from the one or more video capturing devices.
10 . A system, comprising:
one or more processors;
memory in electronic communication with the at one or more processors; and
instructions stored in the memory, the instructions being executable by the one or more processors to:
receive, from one or more video capturing devices, input video content and a set of camera statistics, wherein the set of camera statistics includes data obtained by the one or more video capturing devices in conjunction with generating the input video content;
identify an application for an image processing model to perform with respect to the input video content;
based on the identified application for the image processing model:
pre-processing the input video content using the set of camera statistics to generate transformed video content; and
identify a subset of camera statistics from the set of camera statistics that are relevant to the identified application for an image processing model; and
provide a first input and a second input to the image processing model, the first input including the transformed video content and the second input including the subset of camera statistics, wherein the image processing model comprises a deep learning model trained to generate an output based on video content input and associated camera statistics.
11 . The system of claim 10 , wherein the deep learning model is trained based on training data including both training video content and associated camera statistics.
12 . The system of claim 10 , wherein pre-processing the input video content includes one or more of refining image frames, modifying color or brightness, or down-sampling resolution.
13 . The system of claim 10 , wherein pre-processing the input video content includes identifying a subset of the input video content based on content of interest included in the subset of the input video content.
14 . The system of claim 13 , wherein identifying the subset of the input video content includes identifying cropped portions of image frames of the input video content, and wherein providing the first input and the second input to the image processing model comprises providing the cropped portions of the image frames and the subset of camera statistics as inputs to the deep learning model.
15 . The system of claim 13 , wherein identifying the subset of the input video content includes identifying a subset of image frames from a plurality of image frames within which the content of interest appears, and wherein providing the first input and the second input to the image processing model comprises providing the subset of image frames and the subset of camera statistics as inputs to the deep learning model.
16 . The system of claim 13 , wherein identifying the subset of the input video content is based on the subset of camera statistics indicating the content of interest included in the subset of the input video content.
17 . The system of claim 10 , wherein the input video content comprises captured video footage that has been locally refined by the one or more video capturing devices based on the set of camera statistics.
18 . The system of claim 10 , wherein the deep learning model is implemented on one or more of a cloud computing system or a computing device that receives the input video content from the one or more video capturing devices.
19 . A method, comprising:
receiving, from a plurality of video capturing devices, input video content and a set of camera statistics, wherein the set of camera statistics includes data obtained by the plurality of video capturing devices in conjunction with generating a plurality of input video streams;
identifying an application for an image processing model to perform with respect to the input video content;
based on the identified application for the image processing model:
pre-processing the input video content using the set of camera statistics to generate transformed video content including transforming the plurality of input video streams; and
identifying a subset of camera statistics from the set of camera statistics that are relevant to the identified application for the image processing model; and
providing a first input and a second input to the image processing model, the first input including the transformed video content and the second input including the subset of camera statistics, wherein the image processing model comprises a deep learning model trained to generate an output based on video content input and associated camera statistics inputs, and wherein providing the first input and the second input involves providing at least some transformed input video streams from each of the plurality of video capturing devices to generate the output.
20 . The method of claim 19 , wherein pre-processing the input video content includes identifying a subset of input video content based on content of interest included in the subset of input video content, and wherein identifying the subset of input video content includes one or more of:
identifying cropped portions of image frames of the input video content; or
identifying a subset of image frames from a plurality of image frames within which the content of interest appears.