Selectively identifying data based on motion data from a digital video to provide as input to an image processing model

1 . A method, comprising:

generating decompressed digital video including a plurality of frames by decompressing compressed digital video, wherein the decompressing the compressed digital video comprises:

identifying motion data from the compressed digital video; and

generating pixel data indicating motion of content within respective frames of the plurality of frames based at least in part on the motion data;

identifying, using at least some of the motion data, plural scenes of the decompressed digital video corresponding to plural segments of frames among the plurality of frames;

selecting, based at least in part on results of the identifying the plural scenes, which of the plurality of frames to provide to an image processing model; and

providing, for the selected frames, the pixel data of the selected frames and at least some of the motion data as inputs to the image processing model, the image processing model having been trained to generate an output based on pixel data input and associated motion data input.

2 . The method of claim 1 , wherein the identifying the plural scenes is based on identified discontinuities in the motion data between the plural segments.

3 . The method of claim 1 , wherein the providing the pixel data as input to the image processing model comprises:

providing, as inputs to the image processing model, first frames of a first segment from the plural segments at a first frame rate; and

providing, as inputs to the image processing model, second frames of a second segment from the plural segments at a second frame rate lower than the first frame rate based on a difference between first motion data associated with the first segment and second motion data associated with the second segment.

4 . The method of claim 3 , further comprising determining the first frame rate and the second frame rate based on the first motion data associated with the first segment and the second motion data associated with the second segment, the first motion data indicating a higher metric of movement of content displayed within the first frames of the first segment than a corresponding metric of movement of content displayed within the second frames of the second segment.

5 . The method of claim 1 , wherein the image processing model comprises a deep learning model trained based on input frames for a plurality of input digital videos and associated motion data for the input frames.

6 . The method of claim 1 , wherein the image processing model is implemented on one or more of a cloud computing system or a computing device that received the compressed digital video and generated the decompressed digital video.

7 . The method of claim 1 , wherein the image processing model is trained to generate an output including a reduced clip of digital video based on the pixel data input and the associated motion data input.

8 . The method of claim 1 , wherein the image processing model is trained to generate an output including digital video having metadata tagged to one or more locations within the digital video based on the pixel data input and the associated motion data input.

9 . A system, comprising:

at least one processor;

memory in electronic communication with the at least one processor; and

instructions stored in the memory, the instructions being executable by the at least one processor to:

generate decompressed digital video including a plurality of frames by decompressing compressed digital video, wherein the decompressing the compressed digital video comprises:

identifying motion data from the compressed digital video; and

generating pixel data indicating motion of content within respective frames of the plurality of frames based at least in part on the motion data;

identify, using at least some of the motion data, plural scenes of the decompressed digital video corresponding to plural segments of frames among the plurality of frames;

select, based at least in part on results of the identifying the plural scenes, which of the plurality of frames to provide to an image processing model; and

provide, for the selected frames, the pixel data of the selected frames and at least some of the motion data as inputs to the image processing model, the image processing model having been trained to generate an output based on pixel data input and associated motion data input.

10 . The system of claim 9 , wherein the identifying the plural scenes is based on identified discontinuities in the motion data between the plural segments.

11 . The system of claim 9 , wherein the providing the pixel data as input to the image processing model comprises:

providing, as inputs to the image processing model, first frames of a first segment from the plural segments at a first frame rate; and

providing, as inputs to the image processing model, second frames of a second segment from the plural segments at a second frame rate lower than the first frame rate based on a difference between first motion data associated with the first segment and second motion data associated with the second segment.

12 . The system of claim 11 , further comprising instructions being executable by the at least one processor to determine the first frame rate and the second frame rate based on the first motion data associated with the first segment and the second motion data associated with the second segment, the first motion data indicating a higher metric of movement of content displayed within the first frames of the first segment than a corresponding metric of movement of content displayed within the second frames of the second segment.

13 . The system of claim 9 , wherein the image processing model comprises a deep learning model trained based on input frames for a plurality of input digital videos and associated motion data for the input frames.

14 . The system of claim 9 , wherein the image processing model is implemented on one or more of a cloud computing system or a computing device that received the compressed digital video and generated the decompressed digital video.

15 . The system of claim 9 , wherein the image processing model is trained to generate an output including a reduced clip of digital video based on the pixel data input and the associated motion data input.

16 . The system of claim 9 , wherein the image processing model is trained to generate an output including digital video having metadata tagged to one or more locations within the digital video based on the pixel data input and the associated motion data input.

17 . A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computing device to:

generate decompressed digital video including a plurality of frames by decompressing compressed digital video, wherein the decompressing the compressed digital video comprises:

identifying motion data from the compressed digital video; and

generating pixel data indicating motion of content within respective frames of the plurality of frames based at least in part on the motion data;

identify, using at least some of the motion data, plural scenes of the decompressed digital video corresponding to plural segments of frames among the plurality of frames;

select, based at least in part on results of the identifying the plural scenes, which of the plurality of frames to provide to an image processing model; and

provide, for the selected frames, the pixel data of the selected frames and at least some of the motion data as inputs to the image processing model, the image processing model having been trained to generate an output based on pixel data input and associated motion data input.

18 . The non-transitory computer readable medium of claim 17 ,

wherein the providing the pixel data as an input to the image processing model comprises:

providing, as inputs to the image processing model, first frames of a first segment from the plural segments at a first frame rate; and

providing, as inputs to the image processing model, second frames of a second segment from the plural segments at a second frame rate lower than the first frame rate based on a difference between first motion data associated with the first segment and second motion data associated with the second segment.

19 . The non-transitory computer readable medium of claim 18 , further comprising instructions that, when executed by the at least one processor, cause the computing device to determine the first frame rate and the second frame rate based on the first motion data associated with the first segment and the second motion data associated with the second segment, the first motion data indicating a higher metric of movement of content displayed within the first frames of the first segment than a corresponding metric of movement of content displayed within the second frames of the second segment.

20 . The non-transitory computer readable medium of claim 17 , wherein the identifying the plural scenes is based on identified discontinuities in the motion data between the plural segments.