Media rendering with orientation metadata
Systems and methods for cropping media for a particular orientation using a computing device are described. In some implementations, a method may first comprise receiving by a video preprocessor of the device, a first frame of media in a first orientation. A first region comprising a first feature within the first frame may be identified, by an image analyzer. A cropping calculator of the device may generate a score for the first region based on a characteristic of the first feature and determine that the score for the first region exceeds a threshold. An image processor of the device may then crop the first frame of the video, responsive to the determination that the score for the first region exceeds the threshold, to include the first region within a predetermined display area comprising a subset of the first frame in a second orientation.
1. A method, comprising:
receiving, by a mobile device, data of a first frame of a video;
extracting, by a video preprocessor of the mobile device from the data of the first frame, an identification of a first region of the first frame of the video;
cropping, by the video preprocessor of the mobile device, the first frame of the video to a cropped size centered on the identified first region of the first frame of the video, the cropped size based on an aspect ratio of the mobile device, a resolution of a display of the mobile device, and a first display orientation to display the first frame of the video, the first display orientation is one of a portrait orientation or a landscape orientation;
displaying, on the display of the mobile device, the cropped first frame of the video in the first display orientation;
receiving, by the mobile device, data of a second frame of the video;
extracting, by the video preprocessor of the mobile device from the data of the second frame, region data associated with a second region of the second frame of the video;
cropping, by the video preprocessor of the mobile device, the second frame of the video to the cropped size centered on the identified first region of the first frame and further based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and a second display orientation to display the second frame of the video, the second display orientation is different from the first display orientation and is another of the portrait orientation or the landscape orientation; and
displaying, on the display of the mobile device, the cropped second frame of the video in the second display orientation.
2. The method of claim 1 , further comprising:
receiving an indication of a change in orientation from the first display orientation to the second display orientation.
3. The method of claim 1 , wherein cropping the first frame of the video further comprises:
calculating frame boundaries to apply to the first frame of the video based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and the first display orientation; and
identifying cropping coordinates for a rectangle centered on the identified first region of the first frame of the video based on the calculated frame boundaries.
4. The method of claim 3 , wherein cropping the first frame of video further comprises identifying the cropping coordinates for the rectangle bounding all of the first region.
5. The method of claim 1 , wherein the region data is associated with a plurality of regions of the first frame of the video, and cropping the first frame of the video further comprises:
calculating frame boundaries to apply to the first frame of the video based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and the first display orientation; and
identifying cropping coordinates for a rectangle centered on one of the plurality of regions of the first frame of the video based on the calculated frame boundaries.
6. The method of claim 5 , further comprising:
determining the cropping coordinates for the rectangle are insufficient to bound all of the plurality of regions; and
wherein identifying the cropping coordinates for the rectangle centered on one of the plurality of regions comprises including a text image within the cropping coordinates for the rectangle.
7. The method of claim 1 , wherein the data comprises one or more regions of the first frame of the video with associated scores.
8. The method of claim 7 , wherein each score of the associated scores is associated with at least one feature of the respective region.
9. The method of claim 8 , wherein each score of the associated scores is a score generated based upon one or more characteristics of the at least one feature.
10. A system comprising:
one or more processors of a mobile device;
a network interface electrically connected to the one or more processors; and
a computer storage device electrically connected to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:
receive data of a first frame of a video;
extract, from the data of the first frame, an identification of a first region of the first frame of the video;
crop the first frame of the video to a cropped size centered on the identified first region of the first frame of the video, the cropped size based on an aspect ratio of the mobile device, a resolution of a display of the mobile device, and a first display orientation to display the first frame of the video, the first display orientation is one of a portrait orientation or a landscape orientation;
display, on the display, the cropped first frame of the video in the first display orientation;
receive data of a second frame of the video;
extract, from the data of the second frame, region data associated with a second region of the second frame of the video;
crop the second frame of the video to the cropped size centered on the identified first region of the first frame and further based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and a second display orientation to display the second frame of the video, the second display orientation is different from the first display orientation and is another of the portrait orientation or the landscape orientation; and
display, on the display, the cropped second frame of the video in the second display orientation.
11. The system of claim 10 , wherein the one or more processors are further configured to receive an indication of a change in orientation from the first display orientation to the second display orientation.
12. The system of claim 10 , wherein to crop the first frame of the video, the one or more processors are further configured to:
calculate frame boundaries to apply to the first frame of the video based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and the first display orientation; and
identify cropping coordinates for a rectangle centered on the identified first region of the first frame of the video based on the calculated frame boundaries.
13. The system of claim 12 , wherein to crop the first frame of video, the one or more processors are further configured to identify the cropping coordinates for the rectangle bounding all of the first region.
14. The system of claim 10 , wherein the region data is associated with a plurality of regions of the first frame of the video, and to crop the first frame of the video, the one or more processors is further configured to:
calculate frame boundaries to apply to the first frame of the video based on the aspect ratio of the mobile device, the resolution of the display of the mobile device, and the first display orientation; and
identify cropping coordinates for a rectangle centered on one of the plurality of regions of the first frame of the video based on the calculated frame boundaries.
15. The system of claim 14 , wherein the one or more processors is further configured to:
determine the cropping coordinates for the rectangle are insufficient to bound all of the plurality of regions; and
wherein to identify the cropping coordinates for the rectangle centered on one of the plurality of regions, the one or more processors are further configured to include a text image within the cropping coordinates for the rectangle.
16. The system of claim 10 , wherein the data comprises one or more regions of the first frame of the video with associated scores.
17. The system of claim 16 , wherein each score of the associated scores is associated with at least one feature of the respective region.
18. The system of claim 17 , wherein each score of the associated scores is a score generated based upon one or more characteristics of the at least one feature.