System and method for automatic detection of spherical video content
Systems and methods are described for identifying the video content as spherical video or non-spherical video in response to determining that frame scores and video scores satisfy a threshold level. For example, a plurality of image frames can be extracted from video content, classified in a dual stage process, and scored according to particular classification and scoring mechanisms.
1. A computer-implemented method to identify spherical video content, the method comprising:
receiving, at a video server, video content;
identifying a plurality of image frames in the video content as candidate image frames;
classifying the candidate image frames according to a plurality of image features, the classifying including assigning a frame score to each of the candidate image frames;
identifying a portion of the candidate image frames having a frame score satisfying a threshold condition pertaining to at least one of the plurality of image features; and
generating a video score for the video content by aggregating the frame scores for the portion of the candidate image frames having a frame score satisfying the threshold condition, wherein the video score is used to configure playback of the video content in response to determining that the video score satisfies a threshold video score.
2. The method of claim 1 , further comprising:
labeling the video content as spherical video content in response to determining that the video score satisfies a threshold video score.
3. The method of claim 1 , wherein the video score indicates a likelihood of spherical video image features being detected in at least one of the candidate image frames in the portion.
4. The method of claim 1 , wherein the classifying of the candidate image frames includes:
unwarping the candidate image frames to generate unwarped candidate image frames;
detecting a plurality of edges of each unwarped candidate image frame and determining whether portions of the detected plurality of edges are curved.
5. The method of claim 4 , further comprising assigning a frame score that satisfies the threshold condition, in response to determining an unwarped version of a candidate image frame in the candidate image frames includes more curved edges than straight edges.
6. The method of claim 1 , wherein the plurality of image features include at least an aspect ratio determined for each of the candidate image frames, the aspect ratio being used as an indication that video content that includes a respective candidate image includes spherical content, if the aspect ratio associated with the respective candidate image indicates an equal number of pixels per degree of view.
7. The method of claim 1 , wherein the plurality of image features include at least a determined variance level between columns of pixels in at least one candidate image frame.
8. A video management system including instructions stored on a non-transitory computer-readable storage medium, the system comprising:
at least one processor; and
a memory comprising instructions which, when executed by the at least one processor, cause the at least one processor to:
receive, at a video server, video content;
identify a plurality of image frames in the video content as candidate image frames;
classify the candidate image frames according to a plurality of image features, the classifying including assigning a frame score to each of the candidate image frames;
identify a portion of the candidate image frames having a frame score satisfying a threshold condition pertaining to at least one of the plurality of image features; and
generate a video score for the video content by aggregating the frame scores for the portion of the candidate image frames having a frame score satisfying the threshold condition, wherein the video score is used to configure playback of the video content in response to determining that the video score satisfies a threshold video score.
9. The video management system of claim 8 , wherein the at least one processor is further operable to:
label the video content as spherical video content in response to determining that the video score satisfies a threshold video score.
10. The video management system of claim 8 , wherein the video score indicates a likelihood of spherical video image features being detected in at least one of the candidate image frames in the portion.
11. The video management system of claim 8 , wherein the classifying of the candidate image frames includes:
unwarping the candidate image frames to generate unwarped candidate image frames;
detecting a plurality of edges of each unwarped candidate image frame and determining whether portions of the detected plurality of edges are curved.
12. The video management system of claim 11 , wherein the at least one processor is further operable to:
assign a frame score that satisfies the threshold condition, in response to determining an unwarped version of a candidate image frame in the candidate image frames includes more curved edges than straight edges.
13. The video management system of claim 8 , wherein the plurality of image features include at least an aspect ratio determined for each of the candidate image frames, the aspect ratio being used as an indication that video content that includes a respective candidate image includes spherical content, if the aspect ratio associated with the respective candidate image indicates an equal number of pixels per degree of view.
14. The video management system of claim 8 , wherein the plurality of image features include at least a determined variance level between columns of pixels in at least one candidate image frame.
15. A non-transitory recordable storage medium having recorded and stored thereon instructions that, when executed by a processor, perform actions of:
receiving image content;
identifying a plurality of image frames in the image content as candidate image frames;
classifying the candidate image frames according to a plurality of image features, the classifying including assigning a frame score to each of the candidate image frames;
identifying a portion of the candidate image frames having a frame score satisfying a threshold condition; and
generating an overall score for the image content by aggregating the frame scores for the portion of the candidate image frames having a frame score satisfying the threshold condition, wherein the overall score is used to configure playback of the image content in response to determining that the overall score satisfies a threshold score.
16. The non-transitory recordable storage medium of claim 15 , wherein the actions further include:
labeling the image content as spherical image content in response to determining that the overall score satisfies a predefined threshold score.
17. The non-transitory recordable storage medium of claim 15 , wherein the overall score indicates a likelihood of spherical image features being detected in at least one of the candidate image frames in the portion.
18. The non-transitory recordable storage medium of claim 15 , wherein the classifying of the candidate image frames includes:
unwarping the candidate image frames to generate unwarped candidate image frames;
detecting a plurality of edges in each unwarped candidate image frame and determining whether portions of the detected plurality of edges are curved.
19. The non-transitory recordable storage medium of claim 18 , wherein the actions further include:
assigning a frame score that satisfies the threshold condition, in response to determining an unwarped version of a candidate image frame in the candidate image frames includes more curved edges than straight edges.
20. The non-transitory recordable storage medium of claim 15 , wherein the plurality of image features include at least an aspect ratio determined for each of the candidate image frames, the aspect ratio being used as an indication that image content that includes a respective candidate image includes spherical content, if the aspect ratio associated with the respective candidate image indicates an equal number of pixels per degree of view.