System and method for data reduction based on scene content
A production system determines which areas or portions of a video file (e.g., for a scene) are static and which areas contain motion. Instead of steaming redundant image data for the static areas of the video, the production only sends image data for or updates the areas of each frame that contain motion to minimize an amount of data being streamed to a head mounted display (HMD) without compromising image quality.
1. A method comprising:
obtaining, by a production system, a video file for a three-dimensional (3D) video, the video file including a plurality of frames of the 3D video;
comparing, by a processor of the production system, a first frame to a final frame in the set of frames in the video file;
identifying, by the processor of the production system, one or more areas of motion in the set of frames of the video file based on comparing the first frame to the final frame in the set of frames, areas other than the one or more motions areas of the set of frames corresponding to static areas;
determining, for each of the one or more areas of motion in the set of frames, a rectangle encompassing each of the one or more areas of motion for at least the duration of the set of frames, wherein determining the rectangle encompassing each of the one or more areas of motion includes:
expanding the rectangle by a predetermined amount outside of the area of motion; and
applying a border feather around the expanded rectangle to provide a smooth transition between the area of motion and the static areas;
providing, to a head mounted display of a user, a still image corresponding to the static areas for each frame in the set of frames; and
providing, to the head mounted display, the image data for only the rectangle encompassing each of one or more motions areas for each frame of the set of frames including the border feather around each expanded rectangle, wherein the still image corresponding to the static areas of each frame in the set of frames is provided to the head mounted display once to prevent redundant static image data from being provided to the head mounted display.
2. The method of claim 1 , further comprising:
providing, to the head mounted display, a map with the image data for the one or more motions areas indicating a location for each motion area in the 3D video, the head mounted display of the user using the map to reconstruct the 3D video on the head mounted display.
3. The method of claim 1 , wherein the map further indicates a number of frames for which to incorporate the still image into the 3D video.
4. The method of claim 1 , wherein comparing the first frame to the final frame in the set of frames includes:
disregarding grain, noise, and motion in the video file that is below on a preset threshold amount of grain, noise, or motion.
5. A method comprising:
obtaining, by a production system, a video file for a three-dimensional (3D) video, the video file including a plurality of frames of the 3D video;
comparing, by a processor of the production system, a first frame to a final frame in the set of frames in the video file;
identifying, by the processor of the production system, one or more areas of motion in a set of frames of a video file for a three-dimensional (3D) video based on comparing the first frame to the final frame in the set of frames, areas other than the one or more motions areas of the set of frames corresponding to static areas;
providing, to a head mounted display of a user, a still image corresponding to the static areas for each frame in the set of frames;
providing, to the head mounted display, the image data for only the one or more motions areas for each frame of the set of frames; and
providing, to the head mounted display, a map with the image data for the one or more motions areas indicating a location for each motion area in the 3D video, the head mounted display of the user using the map to reconstruct the 3D video on the head mounted display, wherein the still image corresponding to the static areas of each frame in the set of frames is provided to the head mounted display once to prevent redundant static image data from being provided to the head mounted display.
6. The method of claim 5 , wherein the map further indicates a number of frames for which to incorporate the still image into the 3D video.
7. The method of claim 5 , wherein comparing the first frame to the final frame in the set of frames includes:
disregarding grain, noise, and motion in the video file that is below on a preset threshold amount of grain, noise, or motion.
8. The method of claim 5 , wherein identifying the one or more areas of motion in the set of frames of the video file further includes:
determining, for each of the one or more areas of motion in the set of frames, a minimum rectangle encompassing each of the one or more areas of motion for at least the duration of the set of frames.
9. The method of claim 8 , wherein identifying the one or more areas of motion in the set of frames of the video file further includes:
expanding the minimum rectangle encompassing each of the one or more areas of motion by a predetermined amount; and
applying a border feather around the expanded minimum rectangle to provide a smooth transition between the areas of motion and the still image.
10. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to:
obtain a video file for a three-dimensional (3D) video, the video file including a plurality of frames of the 3D video;
compare a first frame to a final frame in the set of frames in the video file;
identify one or more areas of motion in a set of frames of the video file based on comparing the first frame to the final frame in the set of frames, areas other than the one or more motions areas of the set of frames corresponding to static areas;
provide, to a head mounted display of a user, a still image corresponding to the static areas for each frame in the set of frames; and
provide, to the head mounted display, the image data for only the one or more motions areas for each frame of the set of frames, wherein the still image corresponding to the static areas of each frame in the set of frames is provided to the head mounted display once to prevent redundant static image data from being provided to the head mounted display.
11. The non-transitory computer-readable storage medium of claim 10 , further comprising instructions that, when executed by the processor, further cause the processor to:
provide, to the head mounted display, a map with the image data for the one or more motions areas indicating a location for each motion area in the 3D video, the head mounted display of the user using the map to reconstruct the 3D video on the head mounted display.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the map further indicates a number of frames for which to incorporate the still image into the 3D video.
13. The non-transitory computer-readable storage medium of claim 10 , wherein comparing the first frame to the final frame in the set of frames includes disregarding grain, noise, and motion in the video file that is below on a preset threshold amount of grain, noise, or motion.
14. The non-transitory computer-readable storage medium of claim 10 , wherein identifying the one or more areas of motion in the set of frames of the video file further includes determining, for each of the one or more areas of motion in the set of frames, a minimum rectangle encompassing each of the one or more areas of motion for at least the duration of the set of frames.
15. The non-transitory computer-readable storage medium of claim 14 wherein identifying the one or more areas of motion in the set of frames of the video file further includes:
expanding the minimum rectangle encompassing each of the one or more areas of motion by a predetermined amount; and
applying a border feather around the expanded minimum rectangle to provide a smooth transition between the areas of motion and the still image.