Method, system and computer program product for automatically altering a video stream
There is provided a method for automatically altering a digital video stream including multiple video input frames, to automatically obtain output frames with a target composition, in which frame metadata relating to objects in one or more of the video input frames is analyzed on a frame-by-frame basis and used by a processor to automatically alter one or more output frames to be more similar to, or to match, the target composition, wherein cropping is performed in 3D. A related system and a related computer program product are also provided.
1. A method for automatically cropping input video frames of a digital video stream to obtain output video frames, wherein obtaining the output video frames comprises:
analyzing on a frame-by-frame basis frame metadata relating to objects in one or more of the input video frames; and
using of the frame metadata by a processor to automatically crop one or more of the input video frames based on a target frame composition, wherein the target frame composition defines at least one of a number, identity and/or position of objects for inclusion in at least one output video frame, wherein
the cropping is based on the detection of objects which have associated metrics, the metrics having three-dimensional space parameters, at least one of the three-dimensional space parameters relates to depth information corresponding to the objects in the one or more of the input video frames, and the cropping comprises cropping the one or more of the input video frames to selectively include at least one of the objects in the output video frame based on values of the three-dimensional space parameters associated with the at least one of the objects, respectively, and to exclude at least one further object from the output video frame based on values of the three-dimensional space parameters associated with the at least one further object.
2. The method according to claim 1 , wherein the depth information is depth sensor data.
3. The method according to claim 1 , wherein the cropping is based on detection of a single target object.
4. The method according to claim 1 , wherein the cropping is based on detection of a plurality of target objects.
5. The method according to claim 1 , wherein the target frame composition is based on a determined association between at least two objects for inclusion in the at least one output video frame.
6. The method according to claim 1 , wherein the cropping is based on a scene type metric, the scene type metric identifying a given input video frame as depicting a predefined category of scene.
7. The method according to claim 6 , wherein the scene type metric is based on a predefined rule describing speed of at least one object in the given input video frame.
8. The method according to claim 1 , comprising determining a bounding surface area in the one or more input video frames, wherein:
the bounding surface area of a given input video frame comprises all of the objects for inclusion in the at least one output frame; and
the cropping is based on the bounding surface area.
9. The method according to claim 8 , in which the bounding surface area is a regular polyhedron in three-dimensional space.
10. The method according to claim 1 , further comprising:
receiving, from a user and via a user interface, a selection of a given object in one or more of the input video frames, and:
responsive to the selection, setting the target frame composition to define said given object for inclusion in at least one output video frame.
11. A system comprising a processor and storage, configured to implement a method for automatically cropping input video frames of a digital video stream to obtain output video frames, wherein obtaining the output video frames comprises:
analyzing on a frame-by-frame basis frame metadata relating to objects in one or more of the input video frames; and
using of the frame metadata by a processor to automatically crop one or more of the input video frames based on a target frame composition, wherein the target frame composition defines at least one of a number, identity and/or position of objects for inclusion in at least one output video frame, wherein
the cropping is based on the detection of objects which have associated metrics, the metrics having three-dimensional space parameters, at least one of the three-dimensional space parameters relates to depth information corresponding to the objects in the one or more of the input video frames, and the cropping comprises cropping the one or more of the input video frames to selectively include at least one of the objects in the output video frame based on values of the three-dimensional space parameters associated with the at least one of the objects, respectively, and to exclude at least one further object from the output video frame based on values of the three-dimensional space parameters associated with the at least one further object.