Probabilistic model to compress images for three-dimensional video
A method includes receiving head-tracking data that describe one or more positions of people while the people are viewing a three-dimensional video. The method further includes generating a probabilistic model of the one or more positions of the people based on the head-tracking data, wherein the probabilistic model identifies a probability of a viewer looking in a particular direction as a function of time. The method further includes generating video segments from the three-dimensional video. The method further includes, for each of the video segments: determining a directional encoding format that projects latitudes and longitudes of locations of a surface of a sphere onto locations on a plane, determining a cost function that identifies a region of interest on the plane based on the probabilistic model, and generating optimal segment parameters that minimize a sum-over position for the region of interest.
1. A method comprising:
receiving head-tracking data that describe one or more positions of people while the people are viewing a three-dimensional video;
generating a probabilistic model of the one or more positions of the people based on the head-tracking data, wherein the probabilistic model identifies a probability of a viewer looking in a particular direction as a function of time;
generating video segments from the three-dimensional video;
for each of the video segments:
determining a directional encoding format that projects latitudes and longitudes of locations of a surface of a sphere onto locations on a plane;
determining a cost function that identifies a region of interest on the plane based on the probabilistic model; and
generating optimal segment parameters that minimize a sum-over position for the region of interest; and
re-encoding the three-dimensional video to include the optimal segment parameters for each of the video segments and to modify portions of each of the video segments based on the probability of the viewer looking in the particular direction as the function of time.
2. The method of claim 1 , further comprising providing, based on the re-encoding of the three-dimensional video, a re-encoded video and the optimal segment parameters for each of the video segments to a viewing device, wherein the viewing device uses the optimal segment parameters for each of the video segments to un-distort the re-encoded video and texture the re-encoded video to the sphere to display the re-encoded video with the region of interest for each of the video segments displayed at a higher resolution than other regions in each of the video segments.
3. The method of claim 1 , wherein the re-encoding of the three-dimensional is performed in response to a threshold number of the people viewing the three-dimensional video.
4. The method of claim 1 , wherein the re-encoding of the three-dimensional video includes modifying portions of each of the video segments by blurring portions of each of the video segments based on the probability.
5. The method of claim 4 , wherein an intensity of a level of blur increases as the probability of the viewer looking in the particular direction decreases.
6. The method of claim 4 , further comprising identifying a region of low interest for each of the video segments,
wherein:
the blurring of the portions of each of the video segments includes blurring the region of low interest; and
the region of low interest is displayed at a lower resolution that other regions in each of the video segments.
7. The method of claim 1 , further comprising providing the re-encoded video and the optimal segment parameters for each of the video segments to a client device, wherein the client device uses the re-encoded video and the optimal segment parameters for each of the video segments to generate a two-dimensional video that automates head movement.
8. The method of claim 1 , wherein the probabilistic model is a heat map.
9. A system comprising:
a processor coupled to a memory;
a head tracking module stored in the memory and executable by the processor, the head tracking module configured to receive head-tracking data that describe one or more positions of people while the people are viewing a set of three-dimensional videos, generate a set of probabilistic models of the one or more positions of the people based on the head-tracking data, and estimate a first probabilistic model for a three-dimensional video, wherein the first probabilistic model identifies a probability of a viewer looking in a particular direction as a function of time and the three-dimensional video is not part of the set of three-dimensional videos;
a segmentation module stored in the memory and executable by the processor, the segmentation module configured to generate video segments from the three-dimensional video;
a parameterization module stored in the memory and executable by the processor, the parameterization module configured to, for each of the video segments:
determine a directional encoding format that projects latitudes and longitudes of locations of a surface of a sphere onto locations on a plane;
determine a cost function that identifies a region of interest on the plane based on the first probabilistic model; and
generate optimal segment parameters that minimize a sum-over position for the region of interest; and
an encoder module stored in the memory and executable by the processor, the encoder module configured to re-encode the three-dimensional video to include the optimal segment parameters for each of the video segments and modify of each of the video segments based on the probability of the viewer looking in the particular direction as the function of time.
10. The system of claim 9 , wherein the encoder module is further configured to provide, based on the re-encoding of the three-dimensional video, a re-encoded video and the optimal segment parameters for each of the video segments to a client device, wherein the client device uses the re-encoded video and the optimal segment parameters for each of the video segments to generate a two-dimensional video that automates head movement.
11. The system of claim 9 , wherein the first probabilistic model is a heat map.
12. The system of claim 9 , wherein the re-encoding of the three-dimensional video includes modifying portions of each of the video segments by blurring portions of each of the video segments based on the probability.
13. The system of claim 12 , wherein the encoder module is further configured to provide a recommendation of where to look by blurring every portion of the re-encoded video except for a location in the re-encoded video where a viewer is recommended to look.
14. The system of claim 12 , wherein the re-encoding of the three-dimensional is performed in response to a threshold number of the people viewing the three-dimensional video.
15. The system of claim 12 , further comprising a user interface module stored in the memory and executable by the processor, the user interface module configured to crop the region of interest for one or more video segments based on the optimal segment parameters to form one or more thumbnails of one or more cropped regions of interest and generate a timeline of the three-dimensional video with the one or more thumbnails.
16. A non-transitory computer readable storage medium storing instructions that, when executed by a processor, cause the processor to:
receive head-tracking data that describes one or more positions of people while the people are viewing a three-dimensional video;
generate a probabilistic model of the one or more positions of the people based on the head-tracking data, wherein the probabilistic model identifies a probability of a viewer looking in a particular direction as a function of time;
generate video segments from the three-dimensional video;
for each of the video segments:
determine a directional encoding format that projects latitudes and longitudes of locations of a surface of a sphere onto locations on a plane;
determine a cost function that identifies a region of interest on the plane based on the probabilistic model;
generate optimal segment parameters that minimize a sum-over position for the region of interest; and
identify a region of low interest; and
re-encode the three-dimensional video to include the optimal segment parameters for each of the video segments and to modify the region of low interest based on the probability of the viewer looking in the particular direction as the function of time.
17. The non-transitory computer readable storage medium of claim 16 , wherein the processor is further configured to provide, based on the re-encoding of the three-dimensional video, a re-encoded video and the optimal segment parameters for each of the video segments to a viewing device, wherein the viewing device uses the optimal segment parameters for each of the video segments to un-distort the re-encoded video and texture the re-encoded video to the sphere to display the re-encoded video with the region of interest for each of the video segments displayed at a higher resolution than other regions in each of the video segments and the region of low interest displayed at a lower resolution than other regions in each of the video segments.
18. The non-transitory computer readable storage medium of claim 16 , wherein the probabilistic model is a heat map.
19. The non-transitory computer readable storage medium of claim 16 , wherein the re-encoding of the three-dimensional video includes modifying portions of each of the video segments by blurring portions of each of the video segments based on the probability.
20. The non-transitory computer readable storage medium of claim 19 , wherein the re-encoding of the three-dimensional is performed in response to a threshold number of the people viewing the three-dimensional video.