Segmenting volumetric media content with rotational spaces
Cropping volumetric media is provided. Received volumetric video data is represented by a Bloch sphere. A user selected sub-volume is received to which the volumetric video data is to be cropped. CNNs having different strides are applied to sub-spheres defined within the Bloch sphere. The CNNs generate a correlation score for each sub-sphere relative to a user specified parameter. Best rotation matrices for the sub-spheres are determined to achieve a user specified angle of view, and the sub-sphere correlation scores are optimized according to the best rotation matrices. Cylindrical weights are applied to the sub-spheres according to the optimized correlation scores, and a best solution is selected for the sub-volume according to the correlation scores. The rotation matrices are reflected orthogonally onto the sub-volume, and the sub-volume is cropped from the volumetric video according to the sub-sphere with the highest correlation score to create the cropped image.
1 . A computer-implemented method for cropping volumetric media content, the method comprising:
receiving volumetric video data, wherein the volumetric video data is represented by a Bloch sphere;
receiving input of a user selected sub-volume to which the volumetric video data is to be cropped;
applying a number of convolutional neural networks (CNNs) to sub-spheres defined within the Bloch sphere, wherein the CNNs have different strides;
generating, by the CNNs, a correlation score for each sub-sphere relative to a user specified parameter;
determining best rotation matrices for the sub-spheres to achieve a user specified angle of view for a cropped image;
optimizing the sub-sphere correlation scores according to the best rotation matrices;
applying cylindrical weights to the sub-spheres according to the optimized correlation scores;
selecting a best solution for the user selected sub-volume according to the optimized sub-sphere correlation scores;
reflecting the best rotation matrices orthogonally onto the user selected sub-volume; and
cropping the user selected sub-volume from the volumetric video data according to the sub-sphere with the highest correlation score to create the cropped image.
2 . The method of claim 1 , wherein the sub-spheres represent video frames within the volumetric video data.
3 . The method of claim 1 , wherein each image change within a sub-sphere is processed by a different CNN.
4 . The method of claim 1 , wherein the user selected sub-volume comprises a predefined template.
5 . The method of claim 1 , wherein the user selected sub-volume comprises a user defined linear surface.
6 . The method of claim 1 , wherein the user selected sub-volume comprises a user defined non-linear surface.
7 . The method of claim 1 , further comprising identifying portions of the volumetric video data can be cropped according to historical cropping patterns.
8 . A system for cropping volumetric media content, the system comprising:
a storage device that stores program instructions;
one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to:
receive volumetric video data, wherein the volumetric video data is represented by a Bloch sphere;
receive input of a user selected sub-volume to which the volumetric video data is to be cropped;
apply a number of convolutional neural networks (CNNs) to sub-spheres defined within the Bloch sphere, wherein the CNNs have different strides;
generate, by the CNNs, a correlation score for each sub-sphere relative to a user specified parameter;
determine best rotation matrices for the sub-spheres to achieve a user specified angle of view for a cropped image;
optimize the sub-sphere correlation scores according to the best rotation matrices;
apply cylindrical weights to the sub-spheres according to the optimized correlation scores;
select a best solution for the user selected sub-volume according to the optimized sub-sphere correlation scores;
reflect the best rotation matrices orthogonally onto the user selected sub-volume; and
crop the user selected sub-volume from the volumetric video data according to the sub-sphere with the highest correlation score to create the cropped image.
9 . The system of claim 8 , wherein the sub-spheres represent video frames within the volumetric video data.
10 . The system of claim 8 , wherein each image change within a sub-sphere is processed by a different CNN.
11 . The system of claim 8 , wherein the user selected sub-volume comprises a predefined template.
12 . The system of claim 8 , wherein the user selected sub-volume comprises a user defined linear surface.
13 . The system of claim 8 , wherein the user selected sub-volume comprises a user defined non-linear surface.
14 . A computer program product for cropping volumetric media content, the computer program product comprising:
a computer readable storage medium having program instructions configured to cause one or more processors to:
receive volumetric video data, wherein the volumetric video data is represented by a Bloch sphere;
receive input of a user selected sub-volume to which the volumetric video data is to be cropped;
apply a number of convolutional neural networks (CNNs) to sub-spheres defined within the Bloch sphere, wherein the CNNs have different strides;
generate, by the CNNs, a correlation score for each sub-sphere relative to a user specified parameter;
determine best rotation matrices for the sub-spheres to achieve a user specified angle of view for a cropped image;
optimize the sub-sphere correlation scores according to the best rotation matrices;
apply cylindrical weights to the sub-spheres according to the optimized correlation scores;
select a best solution for the user selected sub-volume according to the optimized sub-sphere correlation scores;
reflect the best rotation matrices orthogonally onto the user selected sub-volume; and
crop the user selected sub-volume from the volumetric video data according to the sub-sphere with the highest correlation score to create the cropped image.
15 . The computer program product of claim 14 , wherein the sub-spheres represent video frames within the volumetric video data.
16 . The computer program product of claim 14 , wherein each image change within a sub-sphere is processed by a different CNN.
17 . The computer program product of claim 14 , wherein the user selected sub-volume comprises a predefined template.
18 . The computer program product of claim 14 , wherein the user selected sub-volume comprises a user defined linear surface.
19 . The computer program product of claim 14 , wherein the user selected sub-volume comprises a user defined non-linear surface.
20 . The computer program product of claim 14 , further comprising instructions for identifying portions of the volumetric video data can be cropped according to historical cropping patterns.